2024-03-23T15:21:39.378 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T15:21:39.379 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} --owner scheduled_pdonnell@teuthology --name pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448 -- /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/orig.config.yaml 2024-03-23T15:21:39.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T15:21:39.875 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448 branch: wip-batrick-testing-20240323.003144-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} email: pdonnell@redhat.com first_in_suite: false job_id: '7618448' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240323.003144-squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 21b3bd672163092b4e542f772797fcba02d3d4de subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 21b3bd672163092b4e542f772797fcba02d3d4de kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240323.003144-squid sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4284 sha1: 21b3bd672163092b4e542f772797fcba02d3d4de sleep_before_teardown: 0 subset: 405/512 suite: fs suite_branch: wip-batrick-testing-20240323.003144-squid suite_path: /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMx8IIWK56fikEo9ZprYtOVI1zpIm3NUtHMXlwNOqNiXuGpae7usAivni1APB+pFqGQqFcd8HEmCP45W4JU36E= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/fFKzerWxP+Si6gH89i627Ly6MfnNTRjCc4sYQ3jQ0LgyClsRX5Y/VUf2tJGG21DEjLQDwJemJQt8EuJhC0hY= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmYcB6Xaa/ZMTcukp3RTyLUzGVkjjFGUVVNpJ/cchpy6nMeqsRbkWdNPLH7bKHfPDAaMP/JQUzVruu58q60J8A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-23_09:56:53 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-23T15:21:39.875 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa; will attempt to use it 2024-03-23T15:21:39.876 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks 2024-03-23T15:21:39.876 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T15:21:39.878 INFO:teuthology.task.internal:Checking packages... 2024-03-23T15:21:39.901 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '21b3bd672163092b4e542f772797fcba02d3d4de' 2024-03-23T15:21:39.901 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T15:21:39.902 INFO:teuthology.packaging:ref: None 2024-03-23T15:21:39.902 INFO:teuthology.packaging:tag: None 2024-03-23T15:21:39.902 INFO:teuthology.packaging:branch: wip-batrick-testing-20240323.003144-squid 2024-03-23T15:21:39.902 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:21:39.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240323.003144-squid 2024-03-23T15:21:40.114 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1855.g21b3bd67 2024-03-23T15:21:40.116 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T15:21:40.123 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T15:21:40.123 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T15:21:40.153 INFO:teuthology.task.internal:Saving configuration 2024-03-23T15:21:40.177 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T15:21:40.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T15:21:40.215 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 15:16:03.538390', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMx8IIWK56fikEo9ZprYtOVI1zpIm3NUtHMXlwNOqNiXuGpae7usAivni1APB+pFqGQqFcd8HEmCP45W4JU36E='} 2024-03-23T15:21:40.240 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 15:16:03.539448', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/fFKzerWxP+Si6gH89i627Ly6MfnNTRjCc4sYQ3jQ0LgyClsRX5Y/VUf2tJGG21DEjLQDwJemJQt8EuJhC0hY='} 2024-03-23T15:21:40.262 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 15:16:03.540336', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmYcB6Xaa/ZMTcukp3RTyLUzGVkjjFGUVVNpJ/cchpy6nMeqsRbkWdNPLH7bKHfPDAaMP/JQUzVruu58q60J8A='} 2024-03-23T15:21:40.262 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T15:21:40.271 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-23T15:21:40.271 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-23T15:21:40.271 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-23T15:21:40.271 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T15:21:40.387 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f219b997160>, signals=[15]) 2024-03-23T15:21:40.387 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T15:21:40.398 INFO:teuthology.task.internal:Opening connections... 2024-03-23T15:21:40.398 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:21:40.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:21:40.491 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-03-23T15:21:40.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:21:40.582 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-03-23T15:21:40.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:21:40.671 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T15:21:40.678 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-03-23T15:21:40.696 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-03-23T15:21:40.696 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-03-23T15:21:40.754 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-03-23T15:21:40.754 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-03-23T15:21:40.754 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-03-23T15:21:40.754 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-03-23T15:21:40.755 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-03-23T15:21:40.755 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-03-23T15:21:40.755 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T15:21:40.755 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-03-23T15:21:40.755 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-03-23T15:21:40.755 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T15:21:40.756 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-03-23T15:21:40.756 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T15:21:40.756 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T15:21:40.756 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T15:21:40.757 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-03-23T15:21:40.785 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-03-23T15:21:40.803 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-03-23T15:21:40.803 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T15:21:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-03-23T15:21:40.862 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T15:21:40.862 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T15:21:40.862 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T15:21:40.862 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-03-23T15:21:40.884 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-03-23T15:21:40.903 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-03-23T15:21:40.903 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-03-23T15:21:40.960 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-03-23T15:21:40.960 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-03-23T15:21:40.960 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-03-23T15:21:40.960 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-03-23T15:21:40.960 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-03-23T15:21:40.960 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-03-23T15:21:40.960 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-23T15:21:40.961 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-03-23T15:21:40.961 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-03-23T15:21:40.961 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-23T15:21:40.961 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-03-23T15:21:40.961 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T15:21:40.961 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-23T15:21:40.961 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T15:21:40.961 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-03-23T15:21:40.984 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T15:21:40.997 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T15:21:41.025 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T15:21:41.026 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-03-23T15:21:41.028 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-03-23T15:21:41.030 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-03-23T15:21:41.047 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T15:21:41.057 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T15:21:41.057 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-03-23T15:21:41.085 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-03-23T15:21:41.088 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-03-23T15:21:41.297 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T15:21:41.381 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T15:21:41.400 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-23T15:21:41.400 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-23T15:21:41.400 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-23T15:21:41.400 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T15:21:41.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:21:41.416 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-03-23T15:21:41.473 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-361.el9.x86_64 2024-03-23T15:21:41.473 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.14.0-361.el9.x86_64 2024-03-23T15:21:41.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2024-03-23T15:21:43.620 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-23T15:21:45.262 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-23T15:21:46.361 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-03-23T15:22:00.719 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-03-23T15:22:05.028 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.1 MB 00:00 2024-03-23T15:22:06.655 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-03-23T15:22:26.016 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-03-23T15:22:30.871 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-23T15:22:32.945 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-23T15:22:32.945 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-23T15:22:33.036 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-03-23T15:22:33.041 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-03-23T15:22:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-03-23T15:22:33.041 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-03-23T15:22:33.041 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-03-23T15:22:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-23T15:22:33.041 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout:Install 4 Packages 2024-03-23T15:22:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:22:33.043 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 76 M 2024-03-23T15:22:33.043 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 123 M 2024-03-23T15:22:33.043 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-03-23T15:22:33.656 INFO:teuthology.orchestra.run.smithi076.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 126 kB/s | 44 kB 00:00 2024-03-23T15:22:40.956 INFO:teuthology.orchestra.run.smithi076.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 1.9 MB/s | 15 MB 00:07 2024-03-23T15:22:45.247 INFO:teuthology.orchestra.run.smithi076.stdout:(3/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 2.8 MB/s | 33 MB 00:11 2024-03-23T15:22:45.664 INFO:teuthology.orchestra.run.smithi076.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 2.3 MB/s | 27 MB 00:12 2024-03-23T15:22:45.664 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-03-23T15:22:45.665 INFO:teuthology.orchestra.run.smithi076.stdout:Total 6.0 MB/s | 76 MB 00:12 2024-03-23T15:22:46.327 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-03-23T15:22:46.694 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-03-23T15:22:46.694 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-03-23T15:22:47.684 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-03-23T15:22:47.684 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-03-23T15:22:50.149 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-03-23T15:22:50.775 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-23T15:22:50.861 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T15:22:53.595 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T15:22:53.735 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T15:22:58.449 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T15:22:58.569 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T15:23:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T15:23:32.540 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T15:23:32.540 INFO:teuthology.orchestra.run.smithi076.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-23T15:23:32.540 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:23:32.585 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-23T15:23:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-23T15:23:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-23T15:23:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-23T15:23:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-23T15:23:33.366 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-23T15:23:33.366 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:23:33.366 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-03-23T15:23:33.367 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-23T15:23:33.367 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-23T15:23:33.367 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-23T15:23:33.367 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-23T15:23:33.367 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:23:33.367 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-03-23T15:23:33.733 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T15:23:33.826 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T15:23:33.826 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T15:23:33.826 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-23T15:23:33.826 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T15:23:33.827 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T15:23:33.827 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-23T15:23:33.827 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T15:23:33.827 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T15:23:33.905 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-23T15:23:33.906 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-23T15:23:33.906 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2024-03-23T15:23:33.939 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 68.el9 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8264254 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-23T15:23:33.940 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:23:33.941 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T15:23:33.942 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T15:23:33.942 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 5.14.0-430.el9.x86_64 2024-03-23T15:23:33.942 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T15:23:34.575 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2024-03-23T15:23:36.026 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-23T15:23:36.044 INFO:teuthology.orchestra.run.smithi076.stderr:done 2024-03-23T15:23:36.047 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-03-23T15:23:36.062 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.BtzhLAoqjV 2024-03-23T15:23:36.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BtzhLAoqjV 2024-03-23T15:23:36.130 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.BtzhLAoqjV 2024-03-23T15:23:36.268 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.BtzhLAoqjV is 6KB 2024-03-23T15:23:36.317 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.BtzhLAoqjV 2024-03-23T15:23:36.331 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T15:23:36.399 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-23T15:23:36.400 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-23T15:23:36.400 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-23T15:23:36.400 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-23T15:23:36.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-23T15:23:36.574 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2024-03-23T15:24:06.578 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-23T15:24:06.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-23T15:24:06.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:24:24.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-03-23T15:24:33.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-23T15:24:33.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:24:37.058 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-03-23T15:24:49.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-23T15:24:49.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:24:52.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-03-23T15:25:07.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-23T15:25:07.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:25:39.245 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-03-23T15:25:39.734 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-23T15:25:39.734 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-430.el9.x86_64"... 2024-03-23T15:25:39.735 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-03-23T15:25:39.758 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-430.el9.x86_64 2024-03-23T15:25:39.758 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-23T15:25:39.758 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T15:25:39.758 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T15:25:40.759 DEBUG:teuthology.parallel:result is None 2024-03-23T15:25:40.759 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T15:25:40.767 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T15:25:40.767 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T15:25:40.769 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T15:25:40.772 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T15:25:40.790 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T15:25:40.797 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T15:25:40.803 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T15:25:40.803 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T15:25:40.825 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T15:25:40.830 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T15:25:40.981 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T15:25:40.988 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T15:25:40.988 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T15:25:41.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:25:41.004 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T15:25:41.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:25:41.021 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T15:25:41.023 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T15:25:41.046 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T15:25:41.083 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T15:25:41.087 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T15:25:41.100 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T15:25:41.118 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T15:25:41.138 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T15:25:41.180 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T15:25:41.182 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T15:25:41.190 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T15:25:41.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T15:25:41.192 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T15:25:41.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T15:25:41.224 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T15:25:41.232 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T15:25:41.233 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T15:25:41.260 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T15:25:41.266 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T15:25:41.282 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T15:25:41.356 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T15:25:41.407 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T15:25:41.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T15:25:41.530 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:25:41.530 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T15:25:41.596 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T15:25:41.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T15:25:41.699 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T15:25:41.783 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T15:25:41.828 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:25:41.828 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T15:25:41.896 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T15:25:41.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T15:25:41.997 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T15:25:42.084 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T15:25:42.128 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:25:42.128 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T15:25:42.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-03-23T15:25:42.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-03-23T15:25:42.202 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-03-23T15:25:42.243 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T15:25:42.245 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T15:25:42.274 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T15:25:42.663 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T15:25:42.672 INFO:teuthology.task.internal:Starting timer... 2024-03-23T15:25:42.673 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T15:25:42.727 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T15:25:42.763 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-23T15:25:42.763 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-03-23T15:25:42.817 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-03-23T15:25:42.818 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-03-23T15:25:42.883 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-03-23T15:25:42.885 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-03-23T15:25:42.948 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-03-23T15:25:42.950 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T15:25:42.950 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-03-23T15:25:42.979 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-03-23T15:25:42.979 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-03-23T15:25:43.000 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-03-23T15:25:43.000 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-03-23T15:25:43.025 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-03-23T15:25:43.025 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-03-23T15:25:43.025 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T15:25:43.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:25:43.054 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T15:25:43.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:25:43.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T15:25:43.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:25:43.111 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T15:25:43.111 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-03-23T15:25:43.147 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-03-23T15:25:43.185 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-03-23T15:25:43.220 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T15:25:43.230 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T15:25:43.247 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T15:25:43.248 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi076.front.sepia.ceph.com,smithi183.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T15:30:19.133 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-03-23T15:30:19.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-23T15:30:19.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:30:19.223 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-03-23T15:30:19.296 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-03-23T15:30:19.297 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-03-23T15:30:19.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:30:19.396 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-03-23T15:30:19.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-03-23T15:30:19.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-03-23T15:30:19.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T15:30:19.562 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-03-23T15:30:19.630 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-03-23T15:30:19.631 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T15:30:19.641 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T15:30:19.642 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T15:30:19.642 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T15:30:19.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T15:30:19.645 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T15:30:19.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T15:30:19.647 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T15:30:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T15:30:19.677 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T15:30:19.690 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T15:30:19.694 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T15:30:19.705 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T15:30:19.719 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-03-23T15:30:19.723 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T15:30:19.724 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-03-23T15:30:19.732 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-03-23T15:30:19.738 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-03-23T15:30:19.748 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T15:30:19.753 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-03-23T15:30:19.755 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T15:30:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T15:30:19.767 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-03-23T15:30:19.772 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T15:30:19.784 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T15:30:19.801 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T15:30:19.816 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-03-23T15:30:19.818 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-03-23T15:30:19.820 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T15:30:19.820 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-03-23T15:30:19.820 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.820 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.820 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.820 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T15:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-03-23T15:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.821 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.845 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-03-23T15:30:19.848 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T15:30:19.848 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-03-23T15:30:19.848 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.848 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.848 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.848 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:30:19.849 INFO:teuthology.run_tasks:Running task install... 2024-03-23T15:30:19.857 DEBUG:teuthology.task.install:project ceph 2024-03-23T15:30:19.857 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'}} 2024-03-23T15:30:19.857 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'} 2024-03-23T15:30:19.858 INFO:teuthology.task.install:Using flavor: default 2024-03-23T15:30:19.865 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T15:30:19.865 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-23T15:30:19.866 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-23T15:30:19.866 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:30:19.868 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-23T15:30:19.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:30:19.869 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'tag': None, 'wait_for_package': False} 2024-03-23T15:30:19.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:30:20.016 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/ 2024-03-23T15:30:20.016 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1855.g21b3bd67 2024-03-23T15:30:20.086 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/ 2024-03-23T15:30:20.086 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1855.g21b3bd67 2024-03-23T15:30:20.093 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/ 2024-03-23T15:30:20.093 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1855.g21b3bd67 2024-03-23T15:30:20.193 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T15:30:20.193 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:30:20.193 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T15:30:20.195 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T15:30:20.195 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:30:20.195 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T15:30:20.206 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T15:30:20.206 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:30:20.206 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T15:30:20.223 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T15:30:20.223 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/21b3bd672163092b4e542f772797fcba02d3d4de/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T15:30:20.224 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T15:30:20.224 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/21b3bd672163092b4e542f772797fcba02d3d4de/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T15:30:20.235 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-23T15:30:20.235 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/21b3bd672163092b4e542f772797fcba02d3d4de/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T15:30:20.291 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T15:30:20.292 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T15:30:20.303 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T15:30:20.370 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T15:30:20.372 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T15:30:20.384 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T15:30:20.437 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-03-23T15:30:20.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-03-23T15:30:20.439 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-03-23T15:30:20.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-03-23T15:30:20.450 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-03-23T15:30:20.451 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-03-23T15:30:20.673 INFO:teuthology.orchestra.run.smithi192.stdout:74 files removed 2024-03-23T15:30:20.687 INFO:teuthology.orchestra.run.smithi183.stdout:74 files removed 2024-03-23T15:30:20.687 INFO:teuthology.orchestra.run.smithi076.stdout:74 files removed 2024-03-23T15:30:20.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T15:30:20.714 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T15:30:20.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-23T15:30:21.337 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 238 kB/s | 83 kB 00:00 2024-03-23T15:30:21.394 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 212 kB/s | 83 kB 00:00 2024-03-23T15:30:21.420 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 205 kB/s | 83 kB 00:00 2024-03-23T15:30:21.737 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 33 kB/s | 10 kB 00:00 2024-03-23T15:30:21.746 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-23T15:30:21.787 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-23T15:30:22.162 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-03-23T15:30:22.180 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-23T15:30:22.246 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-03-23T15:30:22.840 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-23T15:30:22.998 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - AppStream 8.3 MB/s | 6.7 MB 00:00 2024-03-23T15:30:23.100 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 7.5 MB/s | 6.7 MB 00:00 2024-03-23T15:30:24.223 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-03-23T15:30:24.475 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-23T15:30:24.674 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-23T15:30:25.666 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-23T15:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.0 MB 00:06 2024-03-23T15:30:33.721 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 896 kB/s | 8.0 MB 00:09 2024-03-23T15:30:36.239 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-23T15:30:37.299 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 19 MB 00:04 2024-03-23T15:30:40.584 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.1 MB 00:00 2024-03-23T15:30:41.367 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-23T15:30:42.801 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-03-23T15:30:43.049 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-03-23T15:30:44.167 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-23T15:30:44.264 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-23T15:30:47.595 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 924 kB/s | 19 MB 00:20 2024-03-23T15:30:49.092 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-23T15:30:49.286 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-23T15:30:51.462 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T15:30:51.463 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T15:30:51.475 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T15:30:51.475 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T15:30:51.476 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T15:30:51.477 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T15:30:51.479 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T15:30:51.481 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T15:30:51.482 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T15:30:51.561 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-03-23T15:30:51.578 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-03-23T15:30:51.578 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-03-23T15:30:51.578 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-03-23T15:30:51.578 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 6.6 k 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.3 M 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 888 k 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 145 k 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 1.5 M 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 138 k 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 1.8 M 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 7.4 M 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 49 k 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 9.8 M 2024-03-23T15:30:51.579 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 47 M 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 264 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 748 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 219 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 27 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 18 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 32 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 744 k 2024-03-23T15:30:51.580 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 127 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 161 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 320 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 298 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 99 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 85 k 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T15:30:51.581 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 170 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T15:30:51.582 INFO:teuthology.orchestra.run.smithi076.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.4 M 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T15:30:51.583 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 21 M 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 31 k 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 2.4 M 2024-03-23T15:30:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 248 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 4.7 M 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 17 M 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 15 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 25 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T15:30:51.585 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 163 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 499 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.1 M 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T15:30:51.586 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T15:30:51.587 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 45 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 131 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T15:30:51.588 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T15:30:51.589 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T15:30:51.590 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T15:30:51.591 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2024-03-23T15:30:51.592 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T15:30:51.593 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:30:51.593 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-03-23T15:30:51.593 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2024-03-23T15:30:51.593 INFO:teuthology.orchestra.run.smithi076.stdout:Install 150 Packages 2024-03-23T15:30:51.593 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 24 Packages 2024-03-23T15:30:51.593 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:30:51.593 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 224 M 2024-03-23T15:30:51.594 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-03-23T15:30:51.614 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T15:30:51.615 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T15:30:51.627 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T15:30:51.628 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T15:30:51.629 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T15:30:51.629 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T15:30:51.631 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T15:30:51.634 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T15:30:51.634 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T15:30:51.714 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-03-23T15:30:51.737 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 6.6 k 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.3 M 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 888 k 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 145 k 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 1.5 M 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 138 k 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 1.8 M 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 7.4 M 2024-03-23T15:30:51.738 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 49 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 9.8 M 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 47 M 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 264 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 748 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 219 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 27 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 18 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 32 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 744 k 2024-03-23T15:30:51.739 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 127 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 161 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 320 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 298 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 99 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 85 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 170 k 2024-03-23T15:30:51.740 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T15:30:51.741 INFO:teuthology.orchestra.run.smithi192.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.4 M 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T15:30:51.742 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 21 M 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 31 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 2.4 M 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 248 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 4.7 M 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 17 M 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 15 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 25 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T15:30:51.743 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 163 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 499 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.1 M 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T15:30:51.744 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T15:30:51.745 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 45 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 131 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T15:30:51.746 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T15:30:51.747 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T15:30:51.748 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T15:30:51.749 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T15:30:51.750 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:30:51.751 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-03-23T15:30:51.751 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-03-23T15:30:51.751 INFO:teuthology.orchestra.run.smithi192.stdout:Install 150 Packages 2024-03-23T15:30:51.751 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 24 Packages 2024-03-23T15:30:51.751 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:30:51.751 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 224 M 2024-03-23T15:30:51.751 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-03-23T15:30:52.227 INFO:teuthology.orchestra.run.smithi076.stdout:(1/174): ceph-19.0.0-1855.g21b3bd67.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-03-23T15:30:52.535 INFO:teuthology.orchestra.run.smithi076.stdout:(2/174): ceph-fuse-19.0.0-1855.g21b3bd67.el9.x8 2.8 MB/s | 888 kB 00:00 2024-03-23T15:30:52.596 INFO:teuthology.orchestra.run.smithi192.stdout:(1/174): ceph-19.0.0-1855.g21b3bd67.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-03-23T15:30:52.627 INFO:teuthology.orchestra.run.smithi076.stdout:(3/174): ceph-immutable-object-cache-19.0.0-185 1.5 MB/s | 145 kB 00:00 2024-03-23T15:30:52.869 INFO:teuthology.orchestra.run.smithi076.stdout:(4/174): ceph-base-19.0.0-1855.g21b3bd67.el9.x8 6.3 MB/s | 5.3 MB 00:00 2024-03-23T15:30:52.961 INFO:teuthology.orchestra.run.smithi076.stdout:(5/174): ceph-mds-19.0.0-1855.g21b3bd67.el9.x86 7.1 MB/s | 2.4 MB 00:00 2024-03-23T15:30:52.962 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.1 MB 00:01 2024-03-23T15:30:53.045 INFO:teuthology.orchestra.run.smithi076.stdout:(6/174): ceph-mgr-19.0.0-1855.g21b3bd67.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-03-23T15:30:53.354 INFO:teuthology.orchestra.run.smithi192.stdout:(2/174): ceph-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.1 MB/s | 888 kB 00:00 2024-03-23T15:30:53.563 INFO:teuthology.orchestra.run.smithi192.stdout:(3/174): ceph-immutable-object-cache-19.0.0-185 694 kB/s | 145 kB 00:00 2024-03-23T15:30:53.595 INFO:teuthology.orchestra.run.smithi076.stdout:(7/174): ceph-mon-19.0.0-1855.g21b3bd67.el9.x86 7.4 MB/s | 4.7 MB 00:00 2024-03-23T15:30:54.728 INFO:teuthology.orchestra.run.smithi076.stdout:(8/174): ceph-common-19.0.0-1855.g21b3bd67.el9. 7.6 MB/s | 21 MB 00:02 2024-03-23T15:30:54.845 INFO:teuthology.orchestra.run.smithi076.stdout:(9/174): ceph-selinux-19.0.0-1855.g21b3bd67.el9 216 kB/s | 25 kB 00:00 2024-03-23T15:30:54.880 INFO:teuthology.orchestra.run.smithi192.stdout:(4/174): ceph-base-19.0.0-1855.g21b3bd67.el9.x8 2.1 MB/s | 5.3 MB 00:02 2024-03-23T15:30:54.945 INFO:teuthology.orchestra.run.smithi076.stdout:(10/174): ceph-radosgw-19.0.0-1855.g21b3bd67.el 7.2 MB/s | 9.8 MB 00:01 2024-03-23T15:30:55.089 INFO:teuthology.orchestra.run.smithi192.stdout:(5/174): ceph-mgr-19.0.0-1855.g21b3bd67.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-03-23T15:30:55.112 INFO:teuthology.orchestra.run.smithi076.stdout:(11/174): ceph-osd-19.0.0-1855.g21b3bd67.el9.x8 8.1 MB/s | 17 MB 00:02 2024-03-23T15:30:55.137 INFO:teuthology.orchestra.run.smithi076.stdout:(12/174): cephfs-mirror-19.0.0-1855.g21b3bd67.e 1.1 MB/s | 219 kB 00:00 2024-03-23T15:30:55.164 INFO:teuthology.orchestra.run.smithi192.stdout:(6/174): ceph-mds-19.0.0-1855.g21b3bd67.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-03-23T15:30:55.188 INFO:teuthology.orchestra.run.smithi076.stdout:(13/174): cephfs-shell-19.0.0-1855.g21b3bd67.el 360 kB/s | 27 kB 00:00 2024-03-23T15:30:55.213 INFO:teuthology.orchestra.run.smithi076.stdout:(14/174): libcephfs-devel-19.0.0-1855.g21b3bd67 421 kB/s | 32 kB 00:00 2024-03-23T15:30:55.229 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-03-23T15:30:55.305 INFO:teuthology.orchestra.run.smithi076.stdout:(15/174): libcephfs2-19.0.0-1855.g21b3bd67.el9. 6.2 MB/s | 744 kB 00:00 2024-03-23T15:30:55.330 INFO:teuthology.orchestra.run.smithi076.stdout:(16/174): libcephsqlite-19.0.0-1855.g21b3bd67.e 1.4 MB/s | 163 kB 00:00 2024-03-23T15:30:55.380 INFO:teuthology.orchestra.run.smithi076.stdout:(17/174): librados-devel-19.0.0-1855.g21b3bd67. 1.6 MB/s | 127 kB 00:00 2024-03-23T15:30:55.447 INFO:teuthology.orchestra.run.smithi076.stdout:(18/174): libradosstriper1-19.0.0-1855.g21b3bd6 4.2 MB/s | 499 kB 00:00 2024-03-23T15:30:55.548 INFO:teuthology.orchestra.run.smithi076.stdout:(19/174): python3-ceph-argparse-19.0.0-1855.g21 450 kB/s | 45 kB 00:00 2024-03-23T15:30:55.665 INFO:teuthology.orchestra.run.smithi076.stdout:(20/174): python3-ceph-common-19.0.0-1855.g21b3 1.1 MB/s | 131 kB 00:00 2024-03-23T15:30:55.765 INFO:teuthology.orchestra.run.smithi076.stdout:(21/174): python3-cephfs-19.0.0-1855.g21b3bd67. 1.6 MB/s | 161 kB 00:00 2024-03-23T15:30:55.797 INFO:teuthology.orchestra.run.smithi192.stdout:(7/174): ceph-mon-19.0.0-1855.g21b3bd67.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-03-23T15:30:55.857 INFO:teuthology.orchestra.run.smithi076.stdout:(22/174): python3-rados-19.0.0-1855.g21b3bd67.e 3.4 MB/s | 320 kB 00:00 2024-03-23T15:30:55.966 INFO:teuthology.orchestra.run.smithi076.stdout:(23/174): librgw2-19.0.0-1855.g21b3bd67.el9.x86 8.8 MB/s | 5.1 MB 00:00 2024-03-23T15:30:55.991 INFO:teuthology.orchestra.run.smithi076.stdout:(24/174): python3-rbd-19.0.0-1855.g21b3bd67.el9 2.2 MB/s | 298 kB 00:00 2024-03-23T15:30:56.049 INFO:teuthology.orchestra.run.smithi076.stdout:(25/174): python3-rgw-19.0.0-1855.g21b3bd67.el9 1.2 MB/s | 99 kB 00:00 2024-03-23T15:30:56.074 INFO:teuthology.orchestra.run.smithi076.stdout:(26/174): rbd-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-23T15:30:56.183 INFO:teuthology.orchestra.run.smithi076.stdout:(27/174): rbd-nbd-19.0.0-1855.g21b3bd67.el9.x86 1.5 MB/s | 170 kB 00:00 2024-03-23T15:30:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1855.g 314 kB/s | 31 kB 00:00 2024-03-23T15:30:56.459 INFO:teuthology.orchestra.run.smithi076.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1855.g21b3bd6 789 kB/s | 138 kB 00:00 2024-03-23T15:30:56.526 INFO:teuthology.orchestra.run.smithi076.stdout:(30/174): rbd-mirror-19.0.0-1855.g21b3bd67.el9. 6.5 MB/s | 3.1 MB 00:00 2024-03-23T15:30:56.793 INFO:teuthology.orchestra.run.smithi076.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1855.g21b3b 5.3 MB/s | 1.8 MB 00:00 2024-03-23T15:30:56.893 INFO:teuthology.orchestra.run.smithi076.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1855.g21 2.4 MB/s | 248 kB 00:00 2024-03-23T15:30:56.968 INFO:teuthology.orchestra.run.smithi076.stdout:(33/174): ceph-mgr-rook-19.0.0-1855.g21b3bd67.e 658 kB/s | 49 kB 00:00 2024-03-23T15:30:57.052 INFO:teuthology.orchestra.run.smithi076.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1855.g2 184 kB/s | 15 kB 00:00 2024-03-23T15:30:57.089 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-23T15:30:57.152 INFO:teuthology.orchestra.run.smithi076.stdout:(35/174): ceph-volume-19.0.0-1855.g21b3bd67.el9 2.6 MB/s | 264 kB 00:00 2024-03-23T15:30:57.281 INFO:teuthology.orchestra.run.smithi192.stdout:(8/174): ceph-radosgw-19.0.0-1855.g21b3bd67.el9 6.6 MB/s | 9.8 MB 00:01 2024-03-23T15:30:57.327 INFO:teuthology.orchestra.run.smithi076.stdout:(36/174): cephadm-19.0.0-1855.g21b3bd67.el9.noa 4.2 MB/s | 748 kB 00:00 2024-03-23T15:30:57.348 INFO:teuthology.orchestra.run.smithi192.stdout:(9/174): ceph-selinux-19.0.0-1855.g21b3bd67.el9 379 kB/s | 25 kB 00:00 2024-03-23T15:30:57.411 INFO:teuthology.orchestra.run.smithi076.stdout:(37/174): cephfs-top-19.0.0-1855.g21b3bd67.el9. 219 kB/s | 18 kB 00:00 2024-03-23T15:30:57.503 INFO:teuthology.orchestra.run.smithi076.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-03-23T15:30:57.686 INFO:teuthology.orchestra.run.smithi076.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-23T15:30:57.736 INFO:teuthology.orchestra.run.smithi076.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 920 kB 00:00 2024-03-23T15:30:57.762 INFO:teuthology.orchestra.run.smithi076.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-03-23T15:30:57.787 INFO:teuthology.orchestra.run.smithi076.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-23T15:30:57.812 INFO:teuthology.orchestra.run.smithi076.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-23T15:30:57.946 INFO:teuthology.orchestra.run.smithi076.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 17 kB 00:00 2024-03-23T15:30:58.013 INFO:teuthology.orchestra.run.smithi076.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-03-23T15:30:58.046 INFO:teuthology.orchestra.run.smithi076.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 10 kB 00:00 2024-03-23T15:30:58.072 INFO:teuthology.orchestra.run.smithi076.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-23T15:30:58.097 INFO:teuthology.orchestra.run.smithi076.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-23T15:30:58.122 INFO:teuthology.orchestra.run.smithi076.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-23T15:30:58.156 INFO:teuthology.orchestra.run.smithi076.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-23T15:30:58.182 INFO:teuthology.orchestra.run.smithi076.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 802 kB/s | 46 kB 00:00 2024-03-23T15:30:58.207 INFO:teuthology.orchestra.run.smithi076.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-23T15:30:58.232 INFO:teuthology.orchestra.run.smithi076.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-23T15:30:58.257 INFO:teuthology.orchestra.run.smithi076.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-23T15:30:58.282 INFO:teuthology.orchestra.run.smithi076.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 689 kB/s | 34 kB 00:00 2024-03-23T15:30:58.408 INFO:teuthology.orchestra.run.smithi076.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-23T15:30:58.450 INFO:teuthology.orchestra.run.smithi076.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-03-23T15:30:58.475 INFO:teuthology.orchestra.run.smithi076.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-23T15:30:58.500 INFO:teuthology.orchestra.run.smithi076.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-23T15:30:58.659 INFO:teuthology.orchestra.run.smithi076.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-23T15:30:58.692 INFO:teuthology.orchestra.run.smithi076.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-03-23T15:30:58.717 INFO:teuthology.orchestra.run.smithi076.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-23T15:30:58.860 INFO:teuthology.orchestra.run.smithi076.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-23T15:30:58.885 INFO:teuthology.orchestra.run.smithi076.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 148 kB/s | 24 kB 00:00 2024-03-23T15:30:58.910 INFO:teuthology.orchestra.run.smithi076.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-23T15:30:58.935 INFO:teuthology.orchestra.run.smithi076.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-23T15:30:58.960 INFO:teuthology.orchestra.run.smithi076.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-23T15:30:59.094 INFO:teuthology.orchestra.run.smithi076.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-23T15:30:59.120 INFO:teuthology.orchestra.run.smithi076.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-23T15:30:59.145 INFO:teuthology.orchestra.run.smithi076.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-23T15:30:59.161 INFO:teuthology.orchestra.run.smithi076.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-03-23T15:30:59.189 INFO:teuthology.orchestra.run.smithi076.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 984 kB/s | 41 kB 00:00 2024-03-23T15:30:59.212 INFO:teuthology.orchestra.run.smithi076.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-23T15:30:59.237 INFO:teuthology.orchestra.run.smithi076.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-23T15:30:59.263 INFO:teuthology.orchestra.run.smithi076.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-23T15:30:59.280 INFO:teuthology.orchestra.run.smithi076.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-23T15:30:59.296 INFO:teuthology.orchestra.run.smithi076.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-23T15:30:59.485 INFO:teuthology.orchestra.run.smithi076.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 206 kB/s | 39 kB 00:00 2024-03-23T15:30:59.731 INFO:teuthology.orchestra.run.smithi076.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-23T15:30:59.747 INFO:teuthology.orchestra.run.smithi076.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 158 kB/s | 41 kB 00:00 2024-03-23T15:30:59.772 INFO:teuthology.orchestra.run.smithi076.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-23T15:30:59.798 INFO:teuthology.orchestra.run.smithi076.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-23T15:30:59.815 INFO:teuthology.orchestra.run.smithi076.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-23T15:30:59.832 INFO:teuthology.orchestra.run.smithi076.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-23T15:30:59.857 INFO:teuthology.orchestra.run.smithi076.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-23T15:30:59.891 INFO:teuthology.orchestra.run.smithi076.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-23T15:30:59.933 INFO:teuthology.orchestra.run.smithi076.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-23T15:30:59.958 INFO:teuthology.orchestra.run.smithi076.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-23T15:30:59.983 INFO:teuthology.orchestra.run.smithi076.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-23T15:31:00.009 INFO:teuthology.orchestra.run.smithi076.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-23T15:31:00.476 INFO:teuthology.orchestra.run.smithi076.stdout:(91/174): ceph-test-19.0.0-1855.g21b3bd67.el9.x 8.4 MB/s | 47 MB 00:05 2024-03-23T15:31:00.668 INFO:teuthology.orchestra.run.smithi076.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 274 kB/s | 187 kB 00:00 2024-03-23T15:31:00.693 INFO:teuthology.orchestra.run.smithi076.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 902 kB/s | 195 kB 00:00 2024-03-23T15:31:00.726 INFO:teuthology.orchestra.run.smithi076.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-23T15:31:00.776 INFO:teuthology.orchestra.run.smithi076.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-23T15:31:00.898 INFO:teuthology.orchestra.run.smithi192.stdout:(10/174): ceph-osd-19.0.0-1855.g21b3bd67.el9.x8 2.9 MB/s | 17 MB 00:05 2024-03-23T15:31:00.993 INFO:teuthology.orchestra.run.smithi076.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 72 kB/s | 21 kB 00:00 2024-03-23T15:31:01.006 INFO:teuthology.orchestra.run.smithi192.stdout:(11/174): cephfs-mirror-19.0.0-1855.g21b3bd67.e 2.0 MB/s | 219 kB 00:00 2024-03-23T15:31:01.035 INFO:teuthology.orchestra.run.smithi076.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 82 kB/s | 25 kB 00:00 2024-03-23T15:31:01.079 INFO:teuthology.orchestra.run.smithi192.stdout:(12/174): cephfs-shell-19.0.0-1855.g21b3bd67.el 372 kB/s | 27 kB 00:00 2024-03-23T15:31:01.155 INFO:teuthology.orchestra.run.smithi192.stdout:(13/174): libcephfs-devel-19.0.0-1855.g21b3bd67 422 kB/s | 32 kB 00:00 2024-03-23T15:31:01.455 INFO:teuthology.orchestra.run.smithi192.stdout:(14/174): libcephfs2-19.0.0-1855.g21b3bd67.el9. 2.4 MB/s | 744 kB 00:00 2024-03-23T15:31:01.460 INFO:teuthology.orchestra.run.smithi076.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 525 kB/s | 245 kB 00:00 2024-03-23T15:31:01.494 INFO:teuthology.orchestra.run.smithi076.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 541 kB/s | 248 kB 00:00 2024-03-23T15:31:01.603 INFO:teuthology.orchestra.run.smithi076.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 322 kB/s | 35 kB 00:00 2024-03-23T15:31:01.645 INFO:teuthology.orchestra.run.smithi076.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.2 MB/s | 216 kB 00:00 2024-03-23T15:31:01.695 INFO:teuthology.orchestra.run.smithi076.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 484 kB/s | 44 kB 00:00 2024-03-23T15:31:01.753 INFO:teuthology.orchestra.run.smithi076.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 864 kB/s | 94 kB 00:00 2024-03-23T15:31:01.772 INFO:teuthology.orchestra.run.smithi192.stdout:(15/174): ceph-common-19.0.0-1855.g21b3bd67.el9 2.2 MB/s | 21 MB 00:09 2024-03-23T15:31:01.798 INFO:teuthology.orchestra.run.smithi192.stdout:(16/174): libcephsqlite-19.0.0-1855.g21b3bd67.e 477 kB/s | 163 kB 00:00 2024-03-23T15:31:01.804 INFO:teuthology.orchestra.run.smithi076.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-23T15:31:01.873 INFO:teuthology.orchestra.run.smithi192.stdout:(17/174): librados-devel-19.0.0-1855.g21b3bd67. 1.2 MB/s | 127 kB 00:00 2024-03-23T15:31:01.904 INFO:teuthology.orchestra.run.smithi076.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.7 MB/s | 4.1 MB 00:01 2024-03-23T15:31:01.954 INFO:teuthology.orchestra.run.smithi076.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-03-23T15:31:02.156 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-03-23T15:31:02.165 INFO:teuthology.orchestra.run.smithi192.stdout:(18/174): libradosstriper1-19.0.0-1855.g21b3bd6 1.3 MB/s | 499 kB 00:00 2024-03-23T15:31:02.213 INFO:teuthology.orchestra.run.smithi076.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-03-23T15:31:02.257 INFO:teuthology.orchestra.run.smithi192.stdout:(19/174): python3-ceph-argparse-19.0.0-1855.g21 492 kB/s | 45 kB 00:00 2024-03-23T15:31:02.405 INFO:teuthology.orchestra.run.smithi076.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 617 kB/s | 309 kB 00:00 2024-03-23T15:31:02.430 INFO:teuthology.orchestra.run.smithi076.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 876 kB/s | 548 kB 00:00 2024-03-23T15:31:02.672 INFO:teuthology.orchestra.run.smithi076.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-03-23T15:31:02.699 INFO:teuthology.orchestra.run.smithi192.stdout:(20/174): ceph-test-19.0.0-1855.g21b3bd67.el9.x 8.9 MB/s | 47 MB 00:05 2024-03-23T15:31:02.724 INFO:teuthology.orchestra.run.smithi192.stdout:(21/174): python3-ceph-common-19.0.0-1855.g21b3 281 kB/s | 131 kB 00:00 2024-03-23T15:31:02.780 INFO:teuthology.orchestra.run.smithi076.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-03-23T15:31:02.783 INFO:teuthology.orchestra.run.smithi192.stdout:(22/174): python3-cephfs-19.0.0-1855.g21b3bd67. 1.9 MB/s | 161 kB 00:00 2024-03-23T15:31:02.875 INFO:teuthology.orchestra.run.smithi192.stdout:(23/174): python3-rbd-19.0.0-1855.g21b3bd67.el9 3.2 MB/s | 298 kB 00:00 2024-03-23T15:31:02.967 INFO:teuthology.orchestra.run.smithi192.stdout:(24/174): python3-rgw-19.0.0-1855.g21b3bd67.el9 1.1 MB/s | 99 kB 00:00 2024-03-23T15:31:03.000 INFO:teuthology.orchestra.run.smithi192.stdout:(25/174): python3-rados-19.0.0-1855.g21b3bd67.e 1.1 MB/s | 320 kB 00:00 2024-03-23T15:31:03.051 INFO:teuthology.orchestra.run.smithi192.stdout:(26/174): rbd-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-23T15:31:03.053 INFO:teuthology.orchestra.run.smithi076.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 177 kB/s | 67 kB 00:00 2024-03-23T15:31:03.151 INFO:teuthology.orchestra.run.smithi192.stdout:(27/174): rbd-nbd-19.0.0-1855.g21b3bd67.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-23T15:31:03.243 INFO:teuthology.orchestra.run.smithi192.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1855.g 343 kB/s | 31 kB 00:00 2024-03-23T15:31:03.289 INFO:teuthology.orchestra.run.smithi076.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 297 kB/s | 151 kB 00:00 2024-03-23T15:31:03.351 INFO:teuthology.orchestra.run.smithi192.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1855.g21b3bd6 1.2 MB/s | 138 kB 00:00 2024-03-23T15:31:03.356 INFO:teuthology.orchestra.run.smithi076.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-03-23T15:31:03.660 INFO:teuthology.orchestra.run.smithi192.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1855.g21b3b 5.8 MB/s | 1.8 MB 00:00 2024-03-23T15:31:03.706 INFO:teuthology.orchestra.run.smithi076.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-03-23T15:31:03.732 INFO:teuthology.orchestra.run.smithi076.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-03-23T15:31:03.873 INFO:teuthology.orchestra.run.smithi076.stdout:(117/174): python3-asyncssh-2.13.2-5.el9.noarch 938 kB/s | 548 kB 00:00 2024-03-23T15:31:03.982 INFO:teuthology.orchestra.run.smithi076.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 117 kB/s | 32 kB 00:00 2024-03-23T15:31:04.008 INFO:teuthology.orchestra.run.smithi076.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 54 kB/s | 15 kB 00:00 2024-03-23T15:31:04.333 INFO:teuthology.orchestra.run.smithi076.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 374 kB/s | 172 kB 00:00 2024-03-23T15:31:04.453 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T15:31:04.454 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-23T15:31:04.468 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-23T15:31:04.469 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-23T15:31:04.470 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-23T15:31:04.471 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-23T15:31:04.473 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-23T15:31:04.476 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-23T15:31:04.477 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-23T15:31:04.478 INFO:teuthology.orchestra.run.smithi192.stdout:(31/174): rbd-mirror-19.0.0-1855.g21b3bd67.el9. 2.1 MB/s | 3.1 MB 00:01 2024-03-23T15:31:04.491 INFO:teuthology.orchestra.run.smithi076.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 704 kB/s | 358 kB 00:00 2024-03-23T15:31:04.517 INFO:teuthology.orchestra.run.smithi076.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 596 kB/s | 303 kB 00:00 2024-03-23T15:31:04.545 INFO:teuthology.orchestra.run.smithi192.stdout:(32/174): librgw2-19.0.0-1855.g21b3bd67.el9.x86 1.9 MB/s | 5.1 MB 00:02 2024-03-23T15:31:04.556 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-03-23T15:31:04.577 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2024-03-23T15:31:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-03-23T15:31:04.577 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2024-03-23T15:31:04.577 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-03-23T15:31:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 6.6 k 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.3 M 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 888 k 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 145 k 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 1.5 M 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 138 k 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 1.8 M 2024-03-23T15:31:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 7.4 M 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 49 k 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 9.8 M 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 47 M 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 264 k 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 748 k 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 219 k 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 27 k 2024-03-23T15:31:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 18 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 32 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 744 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 127 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-23T15:31:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 161 k 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 320 k 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 298 k 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 99 k 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 85 k 2024-03-23T15:31:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 170 k 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-23T15:31:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-23T15:31:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-23T15:31:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-23T15:31:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-23T15:31:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-23T15:31:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-23T15:31:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-23T15:31:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-23T15:31:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.4 M 2024-03-23T15:31:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 3.1 M 2024-03-23T15:31:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-23T15:31:04.585 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 21 M 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 31 k 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 2.4 M 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 248 k 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 4.7 M 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 17 M 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1855.g21b3bd67.el9 ceph-noarch 15 k 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 25 k 2024-03-23T15:31:04.586 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 163 k 2024-03-23T15:31:04.587 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 499 k 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 5.1 M 2024-03-23T15:31:04.588 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-23T15:31:04.589 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-23T15:31:04.589 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-23T15:31:04.589 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-23T15:31:04.589 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-23T15:31:04.589 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-23T15:31:04.589 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-23T15:31:04.589 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-23T15:31:04.590 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-23T15:31:04.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-23T15:31:04.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 45 k 2024-03-23T15:31:04.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-1855.g21b3bd67.el9 ceph 131 k 2024-03-23T15:31:04.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-23T15:31:04.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-23T15:31:04.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-23T15:31:04.591 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-23T15:31:04.592 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-23T15:31:04.593 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-23T15:31:04.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-23T15:31:04.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-23T15:31:04.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-23T15:31:04.594 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-23T15:31:04.595 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-23T15:31:04.596 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-23T15:31:04.597 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout:Install 150 Packages 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 24 Packages 2024-03-23T15:31:04.598 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:04.599 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 224 M 2024-03-23T15:31:04.599 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-03-23T15:31:04.600 INFO:teuthology.orchestra.run.smithi076.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-03-23T15:31:04.620 INFO:teuthology.orchestra.run.smithi192.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1855.g21 1.7 MB/s | 248 kB 00:00 2024-03-23T15:31:04.645 INFO:teuthology.orchestra.run.smithi192.stdout:(34/174): ceph-mgr-rook-19.0.0-1855.g21b3bd67.e 492 kB/s | 49 kB 00:00 2024-03-23T15:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1855.g2 306 kB/s | 15 kB 00:00 2024-03-23T15:31:04.754 INFO:teuthology.orchestra.run.smithi192.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-03-23T15:31:04.779 INFO:teuthology.orchestra.run.smithi192.stdout:(37/174): ceph-volume-19.0.0-1855.g21b3bd67.el9 1.9 MB/s | 264 kB 00:00 2024-03-23T15:31:04.796 INFO:teuthology.orchestra.run.smithi192.stdout:(38/174): cephfs-top-19.0.0-1855.g21b3bd67.el9. 437 kB/s | 18 kB 00:00 2024-03-23T15:31:04.938 INFO:teuthology.orchestra.run.smithi192.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 4.1 MB/s | 662 kB 00:00 2024-03-23T15:31:04.972 INFO:teuthology.orchestra.run.smithi192.stdout:(40/174): cephadm-19.0.0-1855.g21b3bd67.el9.noa 2.4 MB/s | 748 kB 00:00 2024-03-23T15:31:04.992 INFO:teuthology.orchestra.run.smithi076.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 405 kB/s | 203 kB 00:00 2024-03-23T15:31:04.997 INFO:teuthology.orchestra.run.smithi192.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 4.5 MB/s | 920 kB 00:00 2024-03-23T15:31:05.022 INFO:teuthology.orchestra.run.smithi192.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 106 kB 00:00 2024-03-23T15:31:05.034 INFO:teuthology.orchestra.run.smithi076.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-03-23T15:31:05.047 INFO:teuthology.orchestra.run.smithi192.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-03-23T15:31:05.073 INFO:teuthology.orchestra.run.smithi192.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 639 kB/s | 32 kB 00:00 2024-03-23T15:31:05.156 INFO:teuthology.orchestra.run.smithi192.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-03-23T15:31:05.176 INFO:teuthology.orchestra.run.smithi076.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-03-23T15:31:05.181 INFO:teuthology.orchestra.run.smithi192.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 17 kB 00:00 2024-03-23T15:31:05.207 INFO:teuthology.orchestra.run.smithi192.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 76 kB/s | 10 kB 00:00 2024-03-23T15:31:05.232 INFO:teuthology.orchestra.run.smithi192.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-23T15:31:05.257 INFO:teuthology.orchestra.run.smithi192.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-23T15:31:05.283 INFO:teuthology.orchestra.run.smithi192.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-23T15:31:05.309 INFO:teuthology.orchestra.run.smithi076.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-03-23T15:31:05.316 INFO:teuthology.orchestra.run.smithi192.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-23T15:31:05.335 INFO:teuthology.orchestra.run.smithi076.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 59 kB/s | 18 kB 00:00 2024-03-23T15:31:05.342 INFO:teuthology.orchestra.run.smithi192.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 808 kB/s | 46 kB 00:00 2024-03-23T15:31:05.375 INFO:teuthology.orchestra.run.smithi192.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-03-23T15:31:05.409 INFO:teuthology.orchestra.run.smithi192.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-03-23T15:31:05.434 INFO:teuthology.orchestra.run.smithi192.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 590 kB/s | 34 kB 00:00 2024-03-23T15:31:05.443 INFO:teuthology.orchestra.run.smithi076.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2024-03-23T15:31:05.459 INFO:teuthology.orchestra.run.smithi192.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2024-03-23T15:31:05.560 INFO:teuthology.orchestra.run.smithi192.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 31 MB/s | 4.7 MB 00:00 2024-03-23T15:31:05.585 INFO:teuthology.orchestra.run.smithi076.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 71 kB/s | 19 kB 00:00 2024-03-23T15:31:05.602 INFO:teuthology.orchestra.run.smithi192.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-03-23T15:31:05.627 INFO:teuthology.orchestra.run.smithi076.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 67 kB/s | 20 kB 00:00 2024-03-23T15:31:05.635 INFO:teuthology.orchestra.run.smithi192.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 4.6 MB/s | 811 kB 00:00 2024-03-23T15:31:05.669 INFO:teuthology.orchestra.run.smithi192.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2024-03-23T15:31:05.811 INFO:teuthology.orchestra.run.smithi192.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-03-23T15:31:05.836 INFO:teuthology.orchestra.run.smithi192.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 473 kB/s | 78 kB 00:00 2024-03-23T15:31:05.870 INFO:teuthology.orchestra.run.smithi192.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-03-23T15:31:05.932 INFO:teuthology.orchestra.run.smithi183.stdout:(1/174): ceph-19.0.0-1855.g21b3bd67.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-03-23T15:31:05.935 INFO:teuthology.orchestra.run.smithi076.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-03-23T15:31:05.969 INFO:teuthology.orchestra.run.smithi076.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 230 kB/s | 79 kB 00:00 2024-03-23T15:31:06.037 INFO:teuthology.orchestra.run.smithi192.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-23T15:31:06.058 INFO:teuthology.orchestra.run.smithi192.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 112 kB/s | 24 kB 00:00 2024-03-23T15:31:06.083 INFO:teuthology.orchestra.run.smithi192.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 86 kB/s | 18 kB 00:00 2024-03-23T15:31:06.108 INFO:teuthology.orchestra.run.smithi192.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 144 kB 00:00 2024-03-23T15:31:06.128 INFO:teuthology.orchestra.run.smithi076.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-03-23T15:31:06.158 INFO:teuthology.orchestra.run.smithi192.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 423 kB/s | 32 kB 00:00 2024-03-23T15:31:06.275 INFO:teuthology.orchestra.run.smithi192.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-23T15:31:06.294 INFO:teuthology.orchestra.run.smithi076.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 161 kB/s | 58 kB 00:00 2024-03-23T15:31:06.301 INFO:teuthology.orchestra.run.smithi192.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-23T15:31:06.326 INFO:teuthology.orchestra.run.smithi192.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-23T15:31:06.351 INFO:teuthology.orchestra.run.smithi192.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 921 kB/s | 69 kB 00:00 2024-03-23T15:31:06.376 INFO:teuthology.orchestra.run.smithi192.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 825 kB/s | 41 kB 00:00 2024-03-23T15:31:06.395 INFO:teuthology.orchestra.run.smithi076.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-03-23T15:31:06.402 INFO:teuthology.orchestra.run.smithi192.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-23T15:31:06.427 INFO:teuthology.orchestra.run.smithi192.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-23T15:31:06.452 INFO:teuthology.orchestra.run.smithi192.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-23T15:31:06.465 INFO:teuthology.orchestra.run.smithi183.stdout:(2/174): ceph-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.6 MB/s | 888 kB 00:00 2024-03-23T15:31:06.477 INFO:teuthology.orchestra.run.smithi192.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 631 kB/s | 47 kB 00:00 2024-03-23T15:31:06.478 INFO:teuthology.orchestra.run.smithi076.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-03-23T15:31:06.503 INFO:teuthology.orchestra.run.smithi192.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 572 kB/s | 43 kB 00:00 2024-03-23T15:31:06.528 INFO:teuthology.orchestra.run.smithi192.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 775 kB/s | 39 kB 00:00 2024-03-23T15:31:06.635 INFO:teuthology.orchestra.run.smithi183.stdout:(3/174): ceph-base-19.0.0-1855.g21b3bd67.el9.x8 4.1 MB/s | 5.3 MB 00:01 2024-03-23T15:31:06.666 INFO:teuthology.orchestra.run.smithi183.stdout:(4/174): ceph-immutable-object-cache-19.0.0-185 723 kB/s | 145 kB 00:00 2024-03-23T15:31:06.689 INFO:teuthology.orchestra.run.smithi076.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 97 kB/s | 28 kB 00:00 2024-03-23T15:31:06.712 INFO:teuthology.orchestra.run.smithi076.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 215 kB/s | 90 kB 00:00 2024-03-23T15:31:06.746 INFO:teuthology.orchestra.run.smithi076.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-03-23T15:31:06.895 INFO:teuthology.orchestra.run.smithi192.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 104 kB/s | 41 kB 00:00 2024-03-23T15:31:06.908 INFO:teuthology.orchestra.run.smithi183.stdout:(5/174): ceph-mds-19.0.0-1855.g21b3bd67.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-03-23T15:31:06.920 INFO:teuthology.orchestra.run.smithi192.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 768 kB/s | 300 kB 00:00 2024-03-23T15:31:07.021 INFO:teuthology.orchestra.run.smithi076.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-03-23T15:31:07.054 INFO:teuthology.orchestra.run.smithi076.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-03-23T15:31:07.113 INFO:teuthology.orchestra.run.smithi076.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 443 kB/s | 188 kB 00:00 2024-03-23T15:31:07.170 INFO:teuthology.orchestra.run.smithi192.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-03-23T15:31:07.187 INFO:teuthology.orchestra.run.smithi192.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 237 kB/s | 63 kB 00:00 2024-03-23T15:31:07.204 INFO:teuthology.orchestra.run.smithi192.stdout:(84/174): uuid-1.6.2-55.el9.x86_64.rpm 201 kB/s | 56 kB 00:00 2024-03-23T15:31:07.221 INFO:teuthology.orchestra.run.smithi192.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-03-23T15:31:07.246 INFO:teuthology.orchestra.run.smithi192.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-23T15:31:07.280 INFO:teuthology.orchestra.run.smithi192.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-23T15:31:07.283 INFO:teuthology.orchestra.run.smithi183.stdout:(6/174): ceph-mon-19.0.0-1855.g21b3bd67.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-23T15:31:07.322 INFO:teuthology.orchestra.run.smithi192.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-23T15:31:07.364 INFO:teuthology.orchestra.run.smithi192.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 879 kB/s | 103 kB 00:00 2024-03-23T15:31:07.374 INFO:teuthology.orchestra.run.smithi076.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-03-23T15:31:07.389 INFO:teuthology.orchestra.run.smithi192.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-03-23T15:31:07.415 INFO:teuthology.orchestra.run.smithi192.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-23T15:31:07.433 INFO:teuthology.orchestra.run.smithi183.stdout:(7/174): ceph-mgr-19.0.0-1855.g21b3bd67.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-03-23T15:31:07.455 INFO:teuthology.orchestra.run.smithi076.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-03-23T15:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-23T15:31:07.572 INFO:teuthology.orchestra.run.smithi076.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 444 kB/s | 230 kB 00:00 2024-03-23T15:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-23T15:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-23T15:31:07.632 INFO:teuthology.orchestra.run.smithi192.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-03-23T15:31:07.714 INFO:teuthology.orchestra.run.smithi076.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 77 kB/s | 20 kB 00:00 2024-03-23T15:31:07.973 INFO:teuthology.orchestra.run.smithi076.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 711 kB/s | 427 kB 00:00 2024-03-23T15:31:07.998 INFO:teuthology.orchestra.run.smithi076.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-03-23T15:31:08.299 INFO:teuthology.orchestra.run.smithi192.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 30 kB/s | 21 kB 00:00 2024-03-23T15:31:08.341 INFO:teuthology.orchestra.run.smithi192.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 34 kB/s | 25 kB 00:00 2024-03-23T15:31:08.350 INFO:teuthology.orchestra.run.smithi183.stdout:(8/174): ceph-osd-19.0.0-1855.g21b3bd67.el9.x86 16 MB/s | 17 MB 00:01 2024-03-23T15:31:08.442 INFO:teuthology.orchestra.run.smithi183.stdout:(9/174): ceph-selinux-19.0.0-1855.g21b3bd67.el9 275 kB/s | 25 kB 00:00 2024-03-23T15:31:08.473 INFO:teuthology.orchestra.run.smithi076.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-03-23T15:31:08.532 INFO:teuthology.orchestra.run.smithi076.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-23T15:31:08.716 INFO:teuthology.orchestra.run.smithi076.stdout:(152/174): librados2-19.0.0-1855.g21b3bd67.el9. 4.6 MB/s | 3.4 MB 00:00 2024-03-23T15:31:08.924 INFO:teuthology.orchestra.run.smithi076.stdout:(153/174): librbd1-19.0.0-1855.g21b3bd67.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-03-23T15:31:09.124 INFO:teuthology.orchestra.run.smithi076.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 210 kB/s | 42 kB 00:00 2024-03-23T15:31:09.358 INFO:teuthology.orchestra.run.smithi076.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.0 MB/s | 248 kB 00:00 2024-03-23T15:31:09.458 INFO:teuthology.orchestra.run.smithi076.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 714 kB/s | 71 kB 00:00 2024-03-23T15:31:09.508 INFO:teuthology.orchestra.run.smithi192.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 203 kB/s | 245 kB 00:01 2024-03-23T15:31:09.533 INFO:teuthology.orchestra.run.smithi192.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 208 kB/s | 248 kB 00:01 2024-03-23T15:31:09.541 INFO:teuthology.orchestra.run.smithi076.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 437 kB/s | 36 kB 00:00 2024-03-23T15:31:09.742 INFO:teuthology.orchestra.run.smithi076.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 962 kB/s | 192 kB 00:00 2024-03-23T15:31:09.942 INFO:teuthology.orchestra.run.smithi076.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.2 MB/s | 256 kB 00:00 2024-03-23T15:31:10.001 INFO:teuthology.orchestra.run.smithi076.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 395 kB/s | 23 kB 00:00 2024-03-23T15:31:10.059 INFO:teuthology.orchestra.run.smithi076.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-23T15:31:10.101 INFO:teuthology.orchestra.run.smithi076.stdout:(162/174): NetworkManager-libnm-1.47.2-1.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-03-23T15:31:10.168 INFO:teuthology.orchestra.run.smithi076.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-23T15:31:10.193 INFO:teuthology.orchestra.run.smithi076.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 136 kB 00:00 2024-03-23T15:31:10.235 INFO:teuthology.orchestra.run.smithi076.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 955 kB/s | 64 kB 00:00 2024-03-23T15:31:10.242 INFO:teuthology.orchestra.run.smithi192.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 49 kB/s | 35 kB 00:00 2024-03-23T15:31:10.260 INFO:teuthology.orchestra.run.smithi076.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 428 kB/s | 29 kB 00:00 2024-03-23T15:31:10.402 INFO:teuthology.orchestra.run.smithi076.stdout:(167/174): NetworkManager-1.47.2-1.el9.x86_64.r 1.2 MB/s | 2.3 MB 00:01 2024-03-23T15:31:10.525 INFO:teuthology.orchestra.run.smithi192.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 213 kB/s | 216 kB 00:01 2024-03-23T15:31:10.575 INFO:teuthology.orchestra.run.smithi183.stdout:(10/174): ceph-common-19.0.0-1855.g21b3bd67.el9 3.9 MB/s | 21 MB 00:05 2024-03-23T15:31:10.611 INFO:teuthology.orchestra.run.smithi076.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.2 MB/s | 1.2 MB 00:00 2024-03-23T15:31:10.658 INFO:teuthology.orchestra.run.smithi183.stdout:(11/174): cephfs-mirror-19.0.0-1855.g21b3bd67.e 2.6 MB/s | 219 kB 00:00 2024-03-23T15:31:10.719 INFO:teuthology.orchestra.run.smithi076.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.7 MB/s | 2.2 MB 00:00 2024-03-23T15:31:10.734 INFO:teuthology.orchestra.run.smithi183.stdout:(12/174): cephfs-shell-19.0.0-1855.g21b3bd67.el 361 kB/s | 27 kB 00:00 2024-03-23T15:31:10.778 INFO:teuthology.orchestra.run.smithi076.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.7 MB/s | 465 kB 00:00 2024-03-23T15:31:10.801 INFO:teuthology.orchestra.run.smithi183.stdout:(13/174): libcephfs-devel-19.0.0-1855.g21b3bd67 475 kB/s | 32 kB 00:00 2024-03-23T15:31:10.803 INFO:teuthology.orchestra.run.smithi076.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 373 kB/s | 31 kB 00:00 2024-03-23T15:31:10.867 INFO:teuthology.orchestra.run.smithi192.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 71 kB/s | 44 kB 00:00 2024-03-23T15:31:10.968 INFO:teuthology.orchestra.run.smithi183.stdout:(14/174): libcephfs2-19.0.0-1855.g21b3bd67.el9. 4.4 MB/s | 744 kB 00:00 2024-03-23T15:31:11.003 INFO:teuthology.orchestra.run.smithi076.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.8 MB/s | 1.1 MB 00:00 2024-03-23T15:31:11.026 INFO:teuthology.orchestra.run.smithi192.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 142 kB/s | 22 kB 00:00 2024-03-23T15:31:11.051 INFO:teuthology.orchestra.run.smithi183.stdout:(15/174): libcephsqlite-19.0.0-1855.g21b3bd67.e 1.9 MB/s | 163 kB 00:00 2024-03-23T15:31:11.135 INFO:teuthology.orchestra.run.smithi183.stdout:(16/174): librados-devel-19.0.0-1855.g21b3bd67. 1.5 MB/s | 127 kB 00:00 2024-03-23T15:31:11.145 INFO:teuthology.orchestra.run.smithi076.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 217 kB/s | 74 kB 00:00 2024-03-23T15:31:11.168 INFO:teuthology.orchestra.run.smithi192.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-03-23T15:31:11.268 INFO:teuthology.orchestra.run.smithi192.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 126 kB/s | 94 kB 00:00 2024-03-23T15:31:11.269 INFO:teuthology.orchestra.run.smithi183.stdout:(17/174): libradosstriper1-19.0.0-1855.g21b3bd6 3.7 MB/s | 499 kB 00:00 2024-03-23T15:31:11.387 INFO:teuthology.orchestra.run.smithi076.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 2.3 MB 00:00 2024-03-23T15:31:11.391 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-03-23T15:31:11.391 INFO:teuthology.orchestra.run.smithi076.stdout:Total 11 MB/s | 224 MB 00:19 2024-03-23T15:31:11.669 INFO:teuthology.orchestra.run.smithi183.stdout:(18/174): ceph-test-19.0.0-1855.g21b3bd67.el9.x 15 MB/s | 47 MB 00:03 2024-03-23T15:31:11.719 INFO:teuthology.orchestra.run.smithi183.stdout:(19/174): python3-ceph-argparse-19.0.0-1855.g21 911 kB/s | 45 kB 00:00 2024-03-23T15:31:11.751 INFO:teuthology.orchestra.run.smithi192.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 940 kB/s | 548 kB 00:00 2024-03-23T15:31:11.769 INFO:teuthology.orchestra.run.smithi183.stdout:(20/174): python3-ceph-common-19.0.0-1855.g21b3 2.6 MB/s | 131 kB 00:00 2024-03-23T15:31:11.836 INFO:teuthology.orchestra.run.smithi183.stdout:(21/174): python3-cephfs-19.0.0-1855.g21b3bd67. 2.4 MB/s | 161 kB 00:00 2024-03-23T15:31:11.853 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-03-23T15:31:11.903 INFO:teuthology.orchestra.run.smithi183.stdout:(22/174): python3-rados-19.0.0-1855.g21b3bd67.e 4.7 MB/s | 320 kB 00:00 2024-03-23T15:31:11.943 INFO:teuthology.orchestra.run.smithi192.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 458 kB/s | 309 kB 00:00 2024-03-23T15:31:11.960 INFO:teuthology.orchestra.run.smithi192.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 93 kB/s | 19 kB 00:00 2024-03-23T15:31:11.978 INFO:teuthology.orchestra.run.smithi183.stdout:(23/174): python3-rbd-19.0.0-1855.g21b3bd67.el9 3.9 MB/s | 298 kB 00:00 2024-03-23T15:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-03-23T15:31:12.045 INFO:teuthology.orchestra.run.smithi183.stdout:(24/174): python3-rgw-19.0.0-1855.g21b3bd67.el9 1.5 MB/s | 99 kB 00:00 2024-03-23T15:31:12.103 INFO:teuthology.orchestra.run.smithi183.stdout:(25/174): rbd-fuse-19.0.0-1855.g21b3bd67.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-23T15:31:12.119 INFO:teuthology.orchestra.run.smithi192.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2024-03-23T15:31:12.129 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-03-23T15:31:12.129 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-03-23T15:31:12.211 INFO:teuthology.orchestra.run.smithi192.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 735 kB/s | 67 kB 00:00 2024-03-23T15:31:12.212 INFO:teuthology.orchestra.run.smithi183.stdout:(26/174): librgw2-19.0.0-1855.g21b3bd67.el9.x86 5.4 MB/s | 5.1 MB 00:00 2024-03-23T15:31:12.279 INFO:teuthology.orchestra.run.smithi183.stdout:(27/174): rbd-nbd-19.0.0-1855.g21b3bd67.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-23T15:31:12.311 INFO:teuthology.orchestra.run.smithi192.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-03-23T15:31:12.370 INFO:teuthology.orchestra.run.smithi192.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-23T15:31:12.421 INFO:teuthology.orchestra.run.smithi183.stdout:(28/174): rbd-mirror-19.0.0-1855.g21b3bd67.el9. 9.7 MB/s | 3.1 MB 00:00 2024-03-23T15:31:12.446 INFO:teuthology.orchestra.run.smithi183.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1855.g 188 kB/s | 31 kB 00:00 2024-03-23T15:31:12.470 INFO:teuthology.orchestra.run.smithi192.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-03-23T15:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-03-23T15:31:12.547 INFO:teuthology.orchestra.run.smithi183.stdout:(30/174): ceph-radosgw-19.0.0-1855.g21b3bd67.el 1.9 MB/s | 9.8 MB 00:05 2024-03-23T15:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-03-23T15:31:12.572 INFO:teuthology.orchestra.run.smithi183.stdout:(31/174): ceph-mgr-cephadm-19.0.0-1855.g21b3bd6 918 kB/s | 138 kB 00:00 2024-03-23T15:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-03-23T15:31:12.647 INFO:teuthology.orchestra.run.smithi183.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1855.g21 3.2 MB/s | 248 kB 00:00 2024-03-23T15:31:12.689 INFO:teuthology.orchestra.run.smithi192.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-23T15:31:12.714 INFO:teuthology.orchestra.run.smithi183.stdout:(33/174): ceph-mgr-rook-19.0.0-1855.g21b3bd67.e 740 kB/s | 49 kB 00:00 2024-03-23T15:31:12.789 INFO:teuthology.orchestra.run.smithi192.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-23T15:31:12.797 INFO:teuthology.orchestra.run.smithi183.stdout:(34/174): ceph-mgr-dashboard-19.0.0-1855.g21b3b 5.1 MB/s | 1.8 MB 00:00 2024-03-23T15:31:12.823 INFO:teuthology.orchestra.run.smithi183.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1855.g2 141 kB/s | 15 kB 00:00 2024-03-23T15:31:12.865 INFO:teuthology.orchestra.run.smithi183.stdout:(36/174): ceph-volume-19.0.0-1855.g21b3bd67.el9 3.9 MB/s | 264 kB 00:00 2024-03-23T15:31:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-23T15:31:12.932 INFO:teuthology.orchestra.run.smithi183.stdout:(37/174): cephfs-top-19.0.0-1855.g21b3bd67.el9. 274 kB/s | 18 kB 00:00 2024-03-23T15:31:12.965 INFO:teuthology.orchestra.run.smithi183.stdout:(38/174): cephadm-19.0.0-1855.g21b3bd67.el9.noa 5.1 MB/s | 748 kB 00:00 2024-03-23T15:31:13.115 INFO:teuthology.orchestra.run.smithi183.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-03-23T15:31:13.149 INFO:teuthology.orchestra.run.smithi183.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2024-03-23T15:31:13.174 INFO:teuthology.orchestra.run.smithi183.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-23T15:31:13.181 INFO:teuthology.orchestra.run.smithi192.stdout:(121/174): python3-certifi-2021.10.8-2.el9.noar 24 kB/s | 15 kB 00:00 2024-03-23T15:31:13.200 INFO:teuthology.orchestra.run.smithi183.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-23T15:31:13.217 INFO:teuthology.orchestra.run.smithi183.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 764 kB/s | 32 kB 00:00 2024-03-23T15:31:13.239 INFO:teuthology.orchestra.run.smithi192.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 101 kB/s | 35 kB 00:00 2024-03-23T15:31:13.265 INFO:teuthology.orchestra.run.smithi192.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-23T15:31:13.317 INFO:teuthology.orchestra.run.smithi183.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-23T15:31:13.340 INFO:teuthology.orchestra.run.smithi183.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 140 kB/s | 17 kB 00:00 2024-03-23T15:31:13.365 INFO:teuthology.orchestra.run.smithi192.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-03-23T15:31:13.366 INFO:teuthology.orchestra.run.smithi183.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 207 kB/s | 10 kB 00:00 2024-03-23T15:31:13.391 INFO:teuthology.orchestra.run.smithi183.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-23T15:31:13.407 INFO:teuthology.orchestra.run.smithi192.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-23T15:31:13.416 INFO:teuthology.orchestra.run.smithi183.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-03-23T15:31:13.442 INFO:teuthology.orchestra.run.smithi183.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-23T15:31:13.457 INFO:teuthology.orchestra.run.smithi192.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-03-23T15:31:13.467 INFO:teuthology.orchestra.run.smithi183.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-23T15:31:13.484 INFO:teuthology.orchestra.run.smithi183.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-23T15:31:13.502 INFO:teuthology.orchestra.run.smithi192.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-03-23T15:31:13.509 INFO:teuthology.orchestra.run.smithi183.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-23T15:31:13.535 INFO:teuthology.orchestra.run.smithi183.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-23T15:31:13.550 INFO:teuthology.orchestra.run.smithi192.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-03-23T15:31:13.560 INFO:teuthology.orchestra.run.smithi183.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-23T15:31:13.577 INFO:teuthology.orchestra.run.smithi183.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 825 kB/s | 34 kB 00:00 2024-03-23T15:31:13.583 INFO:teuthology.orchestra.run.smithi192.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-23T15:31:13.634 INFO:teuthology.orchestra.run.smithi192.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-03-23T15:31:13.686 INFO:teuthology.orchestra.run.smithi183.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-23T15:31:13.692 INFO:teuthology.orchestra.run.smithi192.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-03-23T15:31:13.717 INFO:teuthology.orchestra.run.smithi192.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-03-23T15:31:13.719 INFO:teuthology.orchestra.run.smithi183.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-23T15:31:13.744 INFO:teuthology.orchestra.run.smithi183.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2024-03-23T15:31:13.769 INFO:teuthology.orchestra.run.smithi183.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-23T15:31:13.784 INFO:teuthology.orchestra.run.smithi192.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 857 kB/s | 79 kB 00:00 2024-03-23T15:31:13.833 INFO:teuthology.orchestra.run.smithi192.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 502 kB/s | 58 kB 00:00 2024-03-23T15:31:13.887 INFO:teuthology.orchestra.run.smithi183.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2024-03-23T15:31:13.893 INFO:teuthology.orchestra.run.smithi192.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-23T15:31:13.919 INFO:teuthology.orchestra.run.smithi192.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 192 kB/s | 16 kB 00:00 2024-03-23T15:31:13.928 INFO:teuthology.orchestra.run.smithi183.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-03-23T15:31:13.953 INFO:teuthology.orchestra.run.smithi183.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-23T15:31:13.977 INFO:teuthology.orchestra.run.smithi192.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-23T15:31:14.011 INFO:teuthology.orchestra.run.smithi192.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-03-23T15:31:14.069 INFO:teuthology.orchestra.run.smithi192.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-03-23T15:31:14.087 INFO:teuthology.orchestra.run.smithi183.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-23T15:31:14.103 INFO:teuthology.orchestra.run.smithi192.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-03-23T15:31:14.104 INFO:teuthology.orchestra.run.smithi183.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-23T15:31:14.121 INFO:teuthology.orchestra.run.smithi183.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 541 kB/s | 18 kB 00:00 2024-03-23T15:31:14.145 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-03-23T15:31:14.145 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-03-23T15:31:14.146 INFO:teuthology.orchestra.run.smithi183.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-23T15:31:14.162 INFO:teuthology.orchestra.run.smithi192.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-03-23T15:31:14.163 INFO:teuthology.orchestra.run.smithi183.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-03-23T15:31:14.195 INFO:teuthology.orchestra.run.smithi192.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-03-23T15:31:14.254 INFO:teuthology.orchestra.run.smithi192.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-03-23T15:31:14.280 INFO:teuthology.orchestra.run.smithi183.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-23T15:31:14.287 INFO:teuthology.orchestra.run.smithi192.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-23T15:31:14.305 INFO:teuthology.orchestra.run.smithi183.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-23T15:31:14.331 INFO:teuthology.orchestra.run.smithi183.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-23T15:31:14.354 INFO:teuthology.orchestra.run.smithi192.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 894 kB/s | 90 kB 00:00 2024-03-23T15:31:14.356 INFO:teuthology.orchestra.run.smithi183.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-23T15:31:14.381 INFO:teuthology.orchestra.run.smithi183.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-03-23T15:31:14.407 INFO:teuthology.orchestra.run.smithi183.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-23T15:31:14.432 INFO:teuthology.orchestra.run.smithi183.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-23T15:31:14.447 INFO:teuthology.orchestra.run.smithi192.stdout:(146/174): openssl-devel-3.0.7-27.el9.x86_64.rp 621 kB/s | 4.1 MB 00:06 2024-03-23T15:31:14.457 INFO:teuthology.orchestra.run.smithi183.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-23T15:31:14.464 INFO:teuthology.orchestra.run.smithi192.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-03-23T15:31:14.482 INFO:teuthology.orchestra.run.smithi183.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 943 kB/s | 47 kB 00:00 2024-03-23T15:31:14.508 INFO:teuthology.orchestra.run.smithi183.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 854 kB/s | 43 kB 00:00 2024-03-23T15:31:14.589 INFO:teuthology.orchestra.run.smithi192.stdout:(148/174): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-03-23T15:31:14.648 INFO:teuthology.orchestra.run.smithi192.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.2 MB/s | 427 kB 00:00 2024-03-23T15:31:14.691 INFO:teuthology.orchestra.run.smithi183.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 213 kB/s | 39 kB 00:00 2024-03-23T15:31:14.842 INFO:teuthology.orchestra.run.smithi183.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-23T15:31:14.867 INFO:teuthology.orchestra.run.smithi183.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 235 kB/s | 41 kB 00:00 2024-03-23T15:31:14.881 INFO:teuthology.orchestra.run.smithi192.stdout:(150/174): re2-20211101-3.el9.x86_64.rpm 442 kB/s | 192 kB 00:00 2024-03-23T15:31:14.892 INFO:teuthology.orchestra.run.smithi183.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-23T15:31:14.917 INFO:teuthology.orchestra.run.smithi183.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-23T15:31:14.940 INFO:teuthology.orchestra.run.smithi192.stdout:(151/174): librados2-19.0.0-1855.g21b3bd67.el9. 9.7 MB/s | 3.4 MB 00:00 2024-03-23T15:31:14.943 INFO:teuthology.orchestra.run.smithi183.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-23T15:31:14.965 INFO:teuthology.orchestra.run.smithi192.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-03-23T15:31:14.968 INFO:teuthology.orchestra.run.smithi183.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-03-23T15:31:14.993 INFO:teuthology.orchestra.run.smithi183.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-23T15:31:15.019 INFO:teuthology.orchestra.run.smithi183.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-23T15:31:15.060 INFO:teuthology.orchestra.run.smithi183.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-23T15:31:15.086 INFO:teuthology.orchestra.run.smithi183.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-23T15:31:15.111 INFO:teuthology.orchestra.run.smithi183.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-23T15:31:15.123 INFO:teuthology.orchestra.run.smithi192.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-23T15:31:15.145 INFO:teuthology.orchestra.run.smithi183.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-23T15:31:15.174 INFO:teuthology.orchestra.run.smithi192.stdout:(154/174): NetworkManager-libnm-1.47.2-1.el9.x8 8.8 MB/s | 1.8 MB 00:00 2024-03-23T15:31:15.199 INFO:teuthology.orchestra.run.smithi192.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 561 kB/s | 42 kB 00:00 2024-03-23T15:31:15.224 INFO:teuthology.orchestra.run.smithi192.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-03-23T15:31:15.266 INFO:teuthology.orchestra.run.smithi192.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-23T15:31:15.278 INFO:teuthology.orchestra.run.smithi183.stdout:(91/174): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-03-23T15:31:15.291 INFO:teuthology.orchestra.run.smithi192.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 544 kB/s | 36 kB 00:00 2024-03-23T15:31:15.342 INFO:teuthology.orchestra.run.smithi192.stdout:(159/174): librbd1-19.0.0-1855.g21b3bd67.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-03-23T15:31:15.367 INFO:teuthology.orchestra.run.smithi192.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.9 MB/s | 192 kB 00:00 2024-03-23T15:31:15.392 INFO:teuthology.orchestra.run.smithi192.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-23T15:31:15.418 INFO:teuthology.orchestra.run.smithi192.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 305 kB/s | 23 kB 00:00 2024-03-23T15:31:15.443 INFO:teuthology.orchestra.run.smithi192.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-23T15:31:15.445 INFO:teuthology.orchestra.run.smithi183.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 561 kB/s | 187 kB 00:00 2024-03-23T15:31:15.468 INFO:teuthology.orchestra.run.smithi192.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-23T15:31:15.494 INFO:teuthology.orchestra.run.smithi192.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-23T15:31:15.503 INFO:teuthology.orchestra.run.smithi183.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-23T15:31:15.519 INFO:teuthology.orchestra.run.smithi192.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-03-23T15:31:15.529 INFO:teuthology.orchestra.run.smithi183.stdout:(94/174): libquadmath-11.4.1-3.el9.x86_64.rpm 964 kB/s | 195 kB 00:00 2024-03-23T15:31:15.544 INFO:teuthology.orchestra.run.smithi192.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-23T15:31:15.562 INFO:teuthology.orchestra.run.smithi183.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-03-23T15:31:15.586 INFO:teuthology.orchestra.run.smithi192.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-23T15:31:15.589 INFO:teuthology.orchestra.run.smithi183.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 253 kB/s | 21 kB 00:00 2024-03-23T15:31:15.622 INFO:teuthology.orchestra.run.smithi183.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 273 kB/s | 25 kB 00:00 2024-03-23T15:31:15.636 INFO:teuthology.orchestra.run.smithi192.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-23T15:31:15.662 INFO:teuthology.orchestra.run.smithi192.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-23T15:31:15.687 INFO:teuthology.orchestra.run.smithi192.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 622 kB/s | 31 kB 00:00 2024-03-23T15:31:15.723 INFO:teuthology.orchestra.run.smithi183.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-03-23T15:31:15.729 INFO:teuthology.orchestra.run.smithi192.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-23T15:31:15.748 INFO:teuthology.orchestra.run.smithi183.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-23T15:31:15.773 INFO:teuthology.orchestra.run.smithi183.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-23T15:31:15.779 INFO:teuthology.orchestra.run.smithi192.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.7 MB/s | 2.3 MB 00:00 2024-03-23T15:31:15.798 INFO:teuthology.orchestra.run.smithi183.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 693 kB/s | 35 kB 00:00 2024-03-23T15:31:15.823 INFO:teuthology.orchestra.run.smithi183.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 889 kB/s | 44 kB 00:00 2024-03-23T15:31:15.849 INFO:teuthology.orchestra.run.smithi183.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-23T15:31:15.916 INFO:teuthology.orchestra.run.smithi183.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-03-23T15:31:15.983 INFO:teuthology.orchestra.run.smithi183.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.8 MB/s | 4.1 MB 00:00 2024-03-23T15:31:16.075 INFO:teuthology.orchestra.run.smithi183.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-03-23T15:31:16.325 INFO:teuthology.orchestra.run.smithi183.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-03-23T15:31:16.404 INFO:teuthology.orchestra.run.smithi192.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 104 kB/s | 74 kB 00:00 2024-03-23T15:31:16.409 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-03-23T15:31:16.409 INFO:teuthology.orchestra.run.smithi192.stdout:Total 9.1 MB/s | 224 MB 00:24 2024-03-23T15:31:16.475 INFO:teuthology.orchestra.run.smithi183.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 980 kB/s | 548 kB 00:00 2024-03-23T15:31:16.501 INFO:teuthology.orchestra.run.smithi183.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-03-23T15:31:16.567 INFO:teuthology.orchestra.run.smithi183.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-03-23T15:31:16.626 INFO:teuthology.orchestra.run.smithi183.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-03-23T15:31:16.676 INFO:teuthology.orchestra.run.smithi183.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-03-23T15:31:16.768 INFO:teuthology.orchestra.run.smithi183.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-23T15:31:16.893 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-03-23T15:31:16.894 INFO:teuthology.orchestra.run.smithi183.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-03-23T15:31:17.036 INFO:teuthology.orchestra.run.smithi183.stdout:(115/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-03-23T15:31:17.061 INFO:teuthology.orchestra.run.smithi183.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-03-23T15:31:17.086 INFO:teuthology.orchestra.run.smithi183.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 226 kB/s | 43 kB 00:00 2024-03-23T15:31:17.128 INFO:teuthology.orchestra.run.smithi183.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-03-23T15:31:17.170 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-03-23T15:31:17.170 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-03-23T15:31:17.228 INFO:teuthology.orchestra.run.smithi183.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-03-23T15:31:17.262 INFO:teuthology.orchestra.run.smithi183.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 980 kB/s | 172 kB 00:00 2024-03-23T15:31:17.295 INFO:teuthology.orchestra.run.smithi183.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-03-23T15:31:17.354 INFO:teuthology.orchestra.run.smithi183.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-23T15:31:17.379 INFO:teuthology.orchestra.run.smithi183.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-03-23T15:31:17.421 INFO:teuthology.orchestra.run.smithi183.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-23T15:31:17.488 INFO:teuthology.orchestra.run.smithi183.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-23T15:31:17.513 INFO:teuthology.orchestra.run.smithi183.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-03-23T15:31:17.572 INFO:teuthology.orchestra.run.smithi183.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-03-23T15:31:17.597 INFO:teuthology.orchestra.run.smithi183.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-03-23T15:31:17.623 INFO:teuthology.orchestra.run.smithi183.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 212 kB/s | 23 kB 00:00 2024-03-23T15:31:17.673 INFO:teuthology.orchestra.run.smithi183.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-03-23T15:31:17.690 INFO:teuthology.orchestra.run.smithi183.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-23T15:31:17.782 INFO:teuthology.orchestra.run.smithi183.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-03-23T15:31:17.807 INFO:teuthology.orchestra.run.smithi183.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 675 kB/s | 79 kB 00:00 2024-03-23T15:31:17.865 INFO:teuthology.orchestra.run.smithi183.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-03-23T15:31:17.899 INFO:teuthology.orchestra.run.smithi183.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-03-23T15:31:17.966 INFO:teuthology.orchestra.run.smithi183.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-03-23T15:31:17.983 INFO:teuthology.orchestra.run.smithi183.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-03-23T15:31:18.008 INFO:teuthology.orchestra.run.smithi183.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-03-23T15:31:18.058 INFO:teuthology.orchestra.run.smithi183.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-03-23T15:31:18.084 INFO:teuthology.orchestra.run.smithi183.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-03-23T15:31:18.126 INFO:teuthology.orchestra.run.smithi183.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-03-23T15:31:18.151 INFO:teuthology.orchestra.run.smithi183.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-03-23T15:31:18.176 INFO:teuthology.orchestra.run.smithi183.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-03-23T15:31:18.235 INFO:teuthology.orchestra.run.smithi183.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-03-23T15:31:18.277 INFO:teuthology.orchestra.run.smithi183.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-03-23T15:31:18.310 INFO:teuthology.orchestra.run.smithi183.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-03-23T15:31:18.361 INFO:teuthology.orchestra.run.smithi183.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-23T15:31:18.386 INFO:teuthology.orchestra.run.smithi183.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-03-23T15:31:18.453 INFO:teuthology.orchestra.run.smithi183.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-23T15:31:18.562 INFO:teuthology.orchestra.run.smithi183.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-03-23T15:31:18.679 INFO:teuthology.orchestra.run.smithi183.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.7 MB/s | 321 kB 00:00 2024-03-23T15:31:18.896 INFO:teuthology.orchestra.run.smithi183.stdout:(152/174): librados2-19.0.0-1855.g21b3bd67.el9. 6.6 MB/s | 3.4 MB 00:00 2024-03-23T15:31:19.054 INFO:teuthology.orchestra.run.smithi183.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2024-03-23T15:31:19.113 INFO:teuthology.orchestra.run.smithi183.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 723 kB/s | 42 kB 00:00 2024-03-23T15:31:19.134 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-03-23T15:31:19.135 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-03-23T15:31:19.164 INFO:teuthology.orchestra.run.smithi183.stdout:(155/174): librbd1-19.0.0-1855.g21b3bd67.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-03-23T15:31:19.213 INFO:teuthology.orchestra.run.smithi183.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.4 MB/s | 248 kB 00:00 2024-03-23T15:31:19.289 INFO:teuthology.orchestra.run.smithi183.stdout:(157/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.7 MB/s | 1.8 MB 00:00 2024-03-23T15:31:19.322 INFO:teuthology.orchestra.run.smithi183.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 432 kB/s | 36 kB 00:00 2024-03-23T15:31:19.364 INFO:teuthology.orchestra.run.smithi183.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 357 kB/s | 71 kB 00:00 2024-03-23T15:31:19.381 INFO:teuthology.orchestra.run.smithi183.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.1 MB/s | 192 kB 00:00 2024-03-23T15:31:19.406 INFO:teuthology.orchestra.run.smithi183.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-03-23T15:31:19.423 INFO:teuthology.orchestra.run.smithi183.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 392 kB/s | 23 kB 00:00 2024-03-23T15:31:19.465 INFO:teuthology.orchestra.run.smithi183.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-23T15:31:19.490 INFO:teuthology.orchestra.run.smithi183.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 995 kB/s | 108 kB 00:00 2024-03-23T15:31:19.516 INFO:teuthology.orchestra.run.smithi183.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 136 kB 00:00 2024-03-23T15:31:19.533 INFO:teuthology.orchestra.run.smithi183.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 955 kB/s | 64 kB 00:00 2024-03-23T15:31:19.550 INFO:teuthology.orchestra.run.smithi183.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 488 kB/s | 29 kB 00:00 2024-03-23T15:31:19.625 INFO:teuthology.orchestra.run.smithi183.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-03-23T15:31:19.667 INFO:teuthology.orchestra.run.smithi183.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 16 MB/s | 2.2 MB 00:00 2024-03-23T15:31:19.717 INFO:teuthology.orchestra.run.smithi183.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 4.9 MB/s | 465 kB 00:00 2024-03-23T15:31:19.742 INFO:teuthology.orchestra.run.smithi183.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-03-23T15:31:19.793 INFO:teuthology.orchestra.run.smithi183.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.4 MB/s | 2.3 MB 00:00 2024-03-23T15:31:19.834 INFO:teuthology.orchestra.run.smithi183.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.2 MB/s | 1.1 MB 00:00 2024-03-23T15:31:19.876 INFO:teuthology.orchestra.run.smithi183.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 556 kB/s | 74 kB 00:00 2024-03-23T15:31:19.882 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-03-23T15:31:19.882 INFO:teuthology.orchestra.run.smithi183.stdout:Total 15 MB/s | 224 MB 00:15 2024-03-23T15:31:20.361 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-03-23T15:31:20.631 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-03-23T15:31:20.631 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-03-23T15:31:21.145 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-03-23T15:31:21.242 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T15:31:21.276 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T15:31:21.314 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T15:31:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T15:31:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T15:31:21.426 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T15:31:21.459 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T15:31:21.560 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T15:31:21.607 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T15:31:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T15:31:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:31:22.120 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:31:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:31:22.266 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:31:22.305 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T15:31:22.329 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-03-23T15:31:22.329 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-03-23T15:31:22.351 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T15:31:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T15:31:22.593 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T15:31:22.621 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:31:22.724 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:31:22.747 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T15:31:22.795 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T15:31:22.842 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T15:31:22.865 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T15:31:22.880 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T15:31:23.024 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T15:31:23.055 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T15:31:23.095 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T15:31:23.278 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T15:31:23.423 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T15:31:23.483 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T15:31:23.579 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:23.634 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:23.826 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:23.932 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T15:31:24.026 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T15:31:24.056 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T15:31:24.084 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T15:31:24.127 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T15:31:24.200 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T15:31:24.263 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T15:31:24.301 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T15:31:24.365 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T15:31:24.401 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T15:31:24.444 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T15:31:24.477 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T15:31:24.513 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T15:31:24.582 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T15:31:24.618 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 39/198 2024-03-23T15:31:24.661 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 40/198 2024-03-23T15:31:24.690 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 41/198 2024-03-23T15:31:24.759 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T15:31:24.804 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T15:31:24.878 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T15:31:25.018 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T15:31:25.073 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T15:31:25.589 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T15:31:25.668 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T15:31:25.756 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T15:31:25.798 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T15:31:25.805 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T15:31:25.847 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T15:31:25.895 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T15:31:25.932 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 53/198 2024-03-23T15:31:25.981 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T15:31:26.077 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-03-23T15:31:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T15:31:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T15:31:26.246 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T15:31:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T15:31:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T15:31:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T15:31:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T15:31:26.393 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T15:31:26.419 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T15:31:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T15:31:26.461 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T15:31:26.490 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T15:31:26.534 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T15:31:26.780 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T15:31:26.828 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:31:26.846 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T15:31:26.895 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T15:31:26.940 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:31:26.951 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T15:31:26.963 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:31:27.023 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T15:31:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:31:27.054 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T15:31:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T15:31:27.091 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T15:31:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T15:31:27.138 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T15:31:27.174 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T15:31:27.216 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T15:31:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T15:31:27.288 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T15:31:27.347 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T15:31:27.376 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:31:27.391 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T15:31:27.439 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T15:31:27.470 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T15:31:27.478 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:31:27.502 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T15:31:27.503 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T15:31:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T15:31:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T15:31:27.598 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T15:31:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T15:31:27.619 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T15:31:27.636 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T15:31:27.677 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T15:31:27.702 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T15:31:27.727 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T15:31:27.761 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T15:31:27.792 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T15:31:27.805 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T15:31:27.824 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T15:31:27.928 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T15:31:27.929 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T15:31:28.018 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T15:31:28.026 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T15:31:28.068 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T15:31:28.078 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T15:31:28.116 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T15:31:28.146 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:28.189 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T15:31:28.196 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:28.372 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T15:31:28.405 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:28.414 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T15:31:28.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T15:31:28.550 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T15:31:28.595 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T15:31:28.596 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T15:31:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T15:31:28.634 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T15:31:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T15:31:28.669 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T15:31:28.687 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T15:31:28.704 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T15:31:28.721 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T15:31:28.760 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T15:31:28.779 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T15:31:28.791 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T15:31:28.822 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T15:31:28.833 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T15:31:28.871 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T15:31:28.898 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T15:31:28.933 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T15:31:28.934 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T15:31:28.966 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T15:31:28.978 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T15:31:28.998 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T15:31:29.030 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T15:31:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T15:31:29.062 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T15:31:29.091 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T15:31:29.121 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T15:31:29.159 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T15:31:29.183 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-03-23T15:31:29.212 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T15:31:29.220 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 39/198 2024-03-23T15:31:29.256 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T15:31:29.263 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 40/198 2024-03-23T15:31:29.272 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-23T15:31:29.288 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T15:31:29.292 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 41/198 2024-03-23T15:31:29.298 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-23T15:31:29.329 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T15:31:29.334 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T15:31:29.338 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-23T15:31:29.360 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T15:31:29.378 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-23T15:31:29.407 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T15:31:29.431 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T15:31:29.441 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-23T15:31:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-23T15:31:29.480 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T15:31:29.488 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T15:31:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-23T15:31:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-23T15:31:29.629 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T15:31:29.684 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T15:31:29.872 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-23T15:31:29.957 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:31:30.079 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:31:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:31:30.197 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T15:31:30.201 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:31:30.201 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T15:31:30.230 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T15:31:30.245 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T15:31:30.271 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-23T15:31:30.278 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T15:31:30.296 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T15:31:30.326 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T15:31:30.386 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T15:31:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T15:31:30.433 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T15:31:30.440 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T15:31:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T15:31:30.449 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-23T15:31:30.474 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T15:31:30.494 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-23T15:31:30.523 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:31:30.523 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T15:31:30.559 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 53/198 2024-03-23T15:31:30.589 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T15:31:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T15:31:30.625 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:31:30.649 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T15:31:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T15:31:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 17/198 2024-03-23T15:31:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T15:31:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-23T15:31:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T15:31:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-23T15:31:30.825 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T15:31:30.926 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T15:31:30.956 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-23T15:31:30.996 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-23T15:31:30.997 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T15:31:31.087 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T15:31:31.102 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-23T15:31:31.127 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T15:31:31.199 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T15:31:31.242 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-23T15:31:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T15:31:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:31.322 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T15:31:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T15:31:31.376 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T15:31:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T15:31:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T15:31:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T15:31:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T15:31:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T15:31:31.589 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-23T15:31:31.608 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T15:31:31.652 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T15:31:31.694 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T15:31:31.699 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-23T15:31:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-23T15:31:31.823 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-23T15:31:31.825 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T15:31:31.850 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-23T15:31:31.874 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T15:31:31.886 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-23T15:31:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T15:31:31.955 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T15:31:31.959 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-23T15:31:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-23T15:31:32.048 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T15:31:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-23T15:31:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T15:31:32.116 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-23T15:31:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T15:31:32.150 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-23T15:31:32.153 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T15:31:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T15:31:32.194 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-23T15:31:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-23T15:31:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T15:31:32.271 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-23T15:31:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-23T15:31:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T15:31:32.384 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 39/198 2024-03-23T15:31:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 40/198 2024-03-23T15:31:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 41/198 2024-03-23T15:31:32.485 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T15:31:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-23T15:31:32.528 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T15:31:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-23T15:31:32.576 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T15:31:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T15:31:32.644 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-23T15:31:32.688 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T15:31:32.771 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T15:31:32.777 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-23T15:31:32.802 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T15:31:32.822 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T15:31:32.823 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-23T15:31:32.834 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T15:31:32.861 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T15:31:32.873 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T15:31:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T15:31:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T15:31:33.095 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T15:31:33.134 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T15:31:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T15:31:33.180 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T15:31:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T15:31:33.208 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T15:31:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T15:31:33.267 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T15:31:33.298 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T15:31:33.314 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T15:31:33.321 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 125/198 2024-03-23T15:31:33.342 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-23T15:31:33.357 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T15:31:33.390 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T15:31:33.418 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-23T15:31:33.424 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T15:31:33.458 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T15:31:33.488 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-23T15:31:33.499 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T15:31:33.539 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T15:31:33.542 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T15:31:33.547 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-23T15:31:33.579 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-23T15:31:33.619 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-23T15:31:33.631 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T15:31:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 53/198 2024-03-23T15:31:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-23T15:31:33.711 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T15:31:33.756 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T15:31:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T15:31:33.810 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T15:31:33.851 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T15:31:33.898 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-23T15:31:33.947 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T15:31:33.997 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T15:31:34.039 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T15:31:34.072 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-23T15:31:34.131 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-23T15:31:34.157 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T15:31:34.175 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-23T15:31:34.188 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 129/198 2024-03-23T15:31:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-23T15:31:34.502 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-23T15:31:34.678 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-23T15:31:34.714 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T15:31:34.749 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-23T15:31:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T15:31:34.789 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-23T15:31:34.804 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T15:31:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-23T15:31:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-23T15:31:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-23T15:31:34.925 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T15:31:34.951 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-23T15:31:34.992 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-23T15:31:35.127 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-23T15:31:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T15:31:35.182 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-23T15:31:35.212 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-23T15:31:35.253 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-23T15:31:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T15:31:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-23T15:31:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-23T15:31:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-23T15:31:35.437 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T15:31:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-23T15:31:35.468 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T15:31:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-23T15:31:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-23T15:31:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T15:31:35.661 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-23T15:31:35.760 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-23T15:31:35.802 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-23T15:31:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-23T15:31:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-23T15:31:36.090 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-23T15:31:36.131 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-23T15:31:36.168 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T15:31:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-23T15:31:36.279 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T15:31:36.305 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-23T15:31:36.337 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-23T15:31:36.371 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-23T15:31:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:36.404 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-23T15:31:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:36.438 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-23T15:31:36.478 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-23T15:31:36.509 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-23T15:31:36.547 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-23T15:31:36.640 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-23T15:31:36.674 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-23T15:31:36.699 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-23T15:31:36.740 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-23T15:31:36.776 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-23T15:31:36.864 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-23T15:31:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:36.963 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-23T15:31:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T15:31:37.006 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-23T15:31:37.038 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-23T15:31:37.070 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T15:31:37.070 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T15:31:37.071 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T15:31:37.071 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:37.085 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-23T15:31:37.105 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T15:31:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-23T15:31:37.231 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-23T15:31:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T15:31:37.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T15:31:37.599 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T15:31:37.631 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T15:31:37.658 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T15:31:37.958 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-23T15:31:37.967 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T15:31:37.996 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T15:31:38.003 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-23T15:31:38.037 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-23T15:31:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-23T15:31:38.085 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T15:31:38.117 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 125/198 2024-03-23T15:31:38.153 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T15:31:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-23T15:31:38.204 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-23T15:31:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-23T15:31:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-23T15:31:38.480 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-23T15:31:38.606 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T15:31:38.641 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T15:31:38.712 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T15:31:38.742 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 129/198 2024-03-23T15:31:39.120 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-23T15:31:39.225 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-23T15:31:40.438 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-23T15:31:40.511 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-23T15:31:40.543 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-23T15:31:40.574 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-23T15:31:40.602 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-23T15:31:40.913 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-23T15:31:40.940 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T15:31:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:41.030 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 124/198 2024-03-23T15:31:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 125/198 2024-03-23T15:31:41.097 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-23T15:31:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-23T15:31:41.550 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T15:31:41.594 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T15:31:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 128/198 2024-03-23T15:31:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 129/198 2024-03-23T15:31:41.684 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T15:31:41.684 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T15:31:41.684 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T15:31:41.684 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:41.712 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T15:31:43.914 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 130/198 2024-03-23T15:31:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T15:31:44.600 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 131/198 2024-03-23T15:31:44.601 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-23T15:31:44.601 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-23T15:31:44.601 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:44.629 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T15:31:48.760 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T15:31:48.760 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-03-23T15:31:48.760 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-03-23T15:31:48.760 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-03-23T15:31:48.761 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-03-23T15:31:48.761 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-03-23T15:31:48.761 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-03-23T15:31:48.761 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-03-23T15:31:48.761 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:48.928 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T15:31:48.943 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T15:31:48.943 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:48.943 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T15:31:48.944 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:50.432 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T15:31:50.458 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T15:31:50.458 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:50.458 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T15:31:50.458 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T15:31:50.458 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T15:31:50.458 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T15:31:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T15:31:50.706 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:50.706 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T15:31:50.706 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T15:31:50.706 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T15:31:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:51.141 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T15:31:51.164 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T15:31:51.164 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:51.164 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T15:31:51.164 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T15:31:51.164 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T15:31:51.164 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T15:31:51.652 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T15:31:51.675 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T15:31:51.706 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 139/198 2024-03-23T15:31:51.766 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 140/198 2024-03-23T15:31:51.790 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T15:31:52.021 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T15:31:52.049 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T15:31:53.013 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T15:31:53.041 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T15:31:53.141 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T15:31:53.303 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 144/198 2024-03-23T15:31:53.326 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T15:31:53.349 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T15:31:53.349 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:53.350 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T15:31:53.350 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T15:31:53.350 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T15:31:53.350 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:53.386 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T15:31:53.396 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T15:31:53.436 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 147/198 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-03-23T15:31:53.457 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:53.625 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T15:31:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T15:31:53.640 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:53.640 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T15:31:53.640 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:54.289 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T15:31:54.321 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T15:31:54.344 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T15:31:54.344 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:54.344 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T15:31:54.344 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T15:31:54.344 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T15:31:54.344 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:54.388 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T15:31:54.410 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T15:31:54.410 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:54.410 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T15:31:54.410 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:54.464 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:31:54.486 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:31:54.486 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:54.487 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T15:31:54.487 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T15:31:54.487 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T15:31:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T15:31:54.812 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T15:31:54.812 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:54.813 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T15:31:54.813 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T15:31:54.813 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T15:31:54.813 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:31:55.244 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T15:31:55.270 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T15:31:55.270 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:55.270 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T15:31:55.270 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T15:31:55.270 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T15:31:55.270 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T15:31:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T15:31:55.517 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:55.518 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T15:31:55.518 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T15:31:55.518 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T15:31:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:55.954 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T15:31:55.976 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T15:31:55.977 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:55.977 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T15:31:55.977 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T15:31:55.977 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T15:31:55.977 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:56.064 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T15:31:56.314 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 132/198 2024-03-23T15:31:56.314 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-03-23T15:31:56.314 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-03-23T15:31:56.314 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-03-23T15:31:56.314 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-03-23T15:31:56.314 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-03-23T15:31:56.314 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-03-23T15:31:56.315 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-03-23T15:31:56.315 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T15:31:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 133/198 2024-03-23T15:31:56.493 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:56.493 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-23T15:31:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:56.499 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T15:31:56.546 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T15:31:56.578 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 139/198 2024-03-23T15:31:56.637 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 140/198 2024-03-23T15:31:56.661 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T15:31:56.889 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T15:31:56.920 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T15:31:57.877 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T15:31:57.912 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T15:31:57.947 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T15:31:57.972 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 134/198 2024-03-23T15:31:57.972 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:57.972 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-23T15:31:57.972 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T15:31:57.972 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-23T15:31:57.972 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:58.010 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T15:31:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 144/198 2024-03-23T15:31:58.198 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T15:31:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T15:31:58.220 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 135/198 2024-03-23T15:31:58.220 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:58.220 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-23T15:31:58.220 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T15:31:58.220 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-23T15:31:58.220 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:58.228 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T15:31:58.228 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:58.228 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T15:31:58.228 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T15:31:58.228 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T15:31:58.228 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:58.272 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T15:31:58.286 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T15:31:58.328 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 147/198 2024-03-23T15:31:58.657 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T15:31:58.679 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 136/198 2024-03-23T15:31:58.679 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:58.679 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-23T15:31:58.679 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T15:31:58.679 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-23T15:31:58.679 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:31:58.767 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-23T15:31:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T15:31:59.202 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T15:31:59.225 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T15:31:59.232 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 138/198 2024-03-23T15:31:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T15:31:59.248 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T15:31:59.248 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:59.248 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T15:31:59.248 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T15:31:59.248 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T15:31:59.248 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:59.263 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 139/198 2024-03-23T15:31:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 154/198 2024-03-23T15:31:59.292 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T15:31:59.314 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T15:31:59.314 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:59.314 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T15:31:59.314 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:59.323 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 140/198 2024-03-23T15:31:59.325 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 155/198 2024-03-23T15:31:59.346 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T15:31:59.351 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T15:31:59.360 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:31:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:31:59.382 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:59.382 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T15:31:59.382 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T15:31:59.382 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T15:31:59.382 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:59.530 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 157/198 2024-03-23T15:31:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 141/198 2024-03-23T15:31:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T15:31:59.644 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T15:31:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 159/198 2024-03-23T15:31:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T15:31:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T15:31:59.707 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:31:59.707 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T15:31:59.708 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T15:31:59.708 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T15:31:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:31:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T15:31:59.793 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T15:31:59.862 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T15:31:59.894 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 163/198 2024-03-23T15:31:59.935 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T15:31:59.964 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T15:32:00.029 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T15:32:00.097 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T15:32:00.121 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T15:32:00.566 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 142/198 2024-03-23T15:32:00.590 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T15:32:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 143/198 2024-03-23T15:32:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:00.803 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:00.822 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:00.852 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 144/198 2024-03-23T15:32:00.875 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T15:32:00.898 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 145/198 2024-03-23T15:32:00.898 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:32:00.898 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-23T15:32:00.898 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T15:32:00.898 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-23T15:32:00.898 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:32:00.943 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T15:32:00.953 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 146/198 2024-03-23T15:32:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 147/198 2024-03-23T15:32:01.627 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 170/198 2024-03-23T15:32:01.715 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T15:32:01.755 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T15:32:01.834 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-23T15:32:01.844 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T15:32:01.861 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T15:32:01.877 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T15:32:01.883 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 149/198 2024-03-23T15:32:01.883 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:32:01.884 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-23T15:32:01.884 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T15:32:01.884 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-23T15:32:01.884 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:32:01.902 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T15:32:01.927 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T15:32:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T15:32:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 150/198 2024-03-23T15:32:01.950 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:32:01.950 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-23T15:32:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:32:01.953 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T15:32:01.953 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T15:32:01.996 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:32:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:32:02.018 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:32:02.018 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-23T15:32:02.018 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T15:32:02.018 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-23T15:32:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:32:02.041 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T15:32:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T15:32:02.071 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T15:32:02.147 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T15:32:02.178 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T15:32:02.207 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T15:32:02.232 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T15:32:02.257 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T15:32:02.290 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T15:32:02.329 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T15:32:02.338 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T15:32:02.361 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 152/198 2024-03-23T15:32:02.361 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-23T15:32:02.361 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-23T15:32:02.362 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T15:32:02.362 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-23T15:32:02.362 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:32:02.370 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T15:32:02.395 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T15:32:02.421 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T15:32:02.446 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T15:32:02.471 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T15:32:02.477 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:02.477 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:02.658 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:02.658 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T15:32:02.724 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T15:32:02.748 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T15:32:02.772 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T15:32:02.800 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T15:32:02.825 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T15:32:02.891 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T15:32:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T15:32:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 154/198 2024-03-23T15:32:04.223 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 155/198 2024-03-23T15:32:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T15:32:04.434 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 157/198 2024-03-23T15:32:04.546 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T15:32:04.586 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 159/198 2024-03-23T15:32:04.625 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T15:32:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T15:32:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T15:32:04.771 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 163/198 2024-03-23T15:32:04.803 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T15:32:04.833 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T15:32:04.887 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T15:32:04.948 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T15:32:04.982 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T15:32:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 170/198 2024-03-23T15:32:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T15:32:06.598 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T15:32:06.679 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T15:32:06.712 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T15:32:06.745 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T15:32:06.770 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T15:32:06.784 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T15:32:06.797 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T15:32:06.797 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T15:32:06.826 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 154/198 2024-03-23T15:32:06.856 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 155/198 2024-03-23T15:32:06.868 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T15:32:06.883 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-23T15:32:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T15:32:06.898 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T15:32:06.974 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T15:32:07.001 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T15:32:07.042 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T15:32:07.071 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 157/198 2024-03-23T15:32:07.076 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T15:32:07.101 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T15:32:07.134 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T15:32:07.173 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T15:32:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-23T15:32:07.206 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T15:32:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 159/198 2024-03-23T15:32:07.231 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T15:32:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-23T15:32:07.265 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T15:32:07.290 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T15:32:07.315 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T15:32:07.321 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:07.322 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:07.323 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-23T15:32:07.375 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-23T15:32:07.408 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 163/198 2024-03-23T15:32:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-23T15:32:07.477 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-23T15:32:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:07.502 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T15:32:07.531 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-23T15:32:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T15:32:07.584 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-23T15:32:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T15:32:07.617 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-23T15:32:07.640 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T15:32:07.659 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T15:32:07.659 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 1/198 2024-03-23T15:32:07.659 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2/198 2024-03-23T15:32:07.659 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 3/198 2024-03-23T15:32:07.659 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 4/198 2024-03-23T15:32:07.659 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 5/198 2024-03-23T15:32:07.660 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 6/198 2024-03-23T15:32:07.660 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 7/198 2024-03-23T15:32:07.660 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 8/198 2024-03-23T15:32:07.660 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 9/198 2024-03-23T15:32:07.660 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:32:07.660 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:32:07.660 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 13/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 14/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 15/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 17/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 18/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 19/198 2024-03-23T15:32:07.661 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 20/198 2024-03-23T15:32:07.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 21/198 2024-03-23T15:32:07.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 22/198 2024-03-23T15:32:07.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 23/198 2024-03-23T15:32:07.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 24/198 2024-03-23T15:32:07.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 25/198 2024-03-23T15:32:07.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 26/198 2024-03-23T15:32:07.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 27/198 2024-03-23T15:32:07.671 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T15:32:07.684 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 28/198 2024-03-23T15:32:07.684 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 29/198 2024-03-23T15:32:07.684 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 30/198 2024-03-23T15:32:07.684 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 31/198 2024-03-23T15:32:07.684 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 32/198 2024-03-23T15:32:07.685 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 33/198 2024-03-23T15:32:07.685 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 34/198 2024-03-23T15:32:07.685 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 35/198 2024-03-23T15:32:07.685 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 36/198 2024-03-23T15:32:07.685 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 37/198 2024-03-23T15:32:07.685 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 38/198 2024-03-23T15:32:07.686 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 39/198 2024-03-23T15:32:07.686 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T15:32:07.686 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T15:32:07.686 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T15:32:07.686 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T15:32:07.686 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T15:32:07.687 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T15:32:07.687 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T15:32:07.687 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T15:32:07.687 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T15:32:07.687 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T15:32:07.687 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T15:32:07.687 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T15:32:07.688 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T15:32:07.688 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T15:32:07.688 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T15:32:07.688 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T15:32:07.688 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T15:32:07.688 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T15:32:07.689 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T15:32:07.689 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T15:32:07.689 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T15:32:07.689 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T15:32:07.689 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T15:32:07.689 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T15:32:07.690 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T15:32:07.691 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T15:32:07.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T15:32:07.693 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T15:32:07.694 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T15:32:07.695 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T15:32:07.696 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T15:32:07.697 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T15:32:07.698 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T15:32:07.698 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T15:32:07.698 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T15:32:07.698 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T15:32:07.698 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T15:32:07.698 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T15:32:07.698 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T15:32:07.701 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T15:32:07.702 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T15:32:07.703 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T15:32:07.704 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T15:32:07.704 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T15:32:07.766 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T15:32:08.226 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:08.266 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:08.285 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-23T15:32:09.091 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 170/198 2024-03-23T15:32:09.169 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-23T15:32:09.209 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-23T15:32:09.289 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-23T15:32:09.348 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-23T15:32:09.372 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-23T15:32:09.398 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-23T15:32:09.424 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-23T15:32:09.425 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T15:32:09.512 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-23T15:32:09.542 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-23T15:32:09.542 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T15:32:09.627 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-23T15:32:09.653 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-23T15:32:09.686 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-23T15:32:09.711 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-23T15:32:09.736 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-23T15:32:09.769 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-23T15:32:09.809 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-23T15:32:09.842 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-23T15:32:09.867 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-23T15:32:09.892 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-23T15:32:09.934 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-23T15:32:09.959 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-23T15:32:09.965 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:09.965 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:10.145 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-23T15:32:10.146 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T15:32:10.211 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-23T15:32:10.237 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-23T15:32:10.268 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-23T15:32:10.304 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-23T15:32:10.329 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-23T15:32:10.393 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T15:32:10.655 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T15:32:10.655 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T15:32:10.656 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.657 INFO:teuthology.orchestra.run.smithi076.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T15:32:10.658 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.659 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.660 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.661 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: jq-1.6-16.el9.x86_64 2024-03-23T15:32:10.662 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.663 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T15:32:10.664 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T15:32:10.665 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T15:32:10.666 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T15:32:10.667 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T15:32:10.668 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T15:32:10.669 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T15:32:10.670 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:32:10.671 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-03-23T15:32:11.173 DEBUG:teuthology.parallel:result is None 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 1/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 3/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 4/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 5/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 6/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 7/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 8/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 9/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:32:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 13/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 14/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 15/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 17/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 18/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 19/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 20/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 21/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 22/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 23/198 2024-03-23T15:32:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 24/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 25/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 26/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 27/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 28/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 29/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 30/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 31/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 32/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 33/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 34/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 35/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 36/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 37/198 2024-03-23T15:32:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 38/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 39/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T15:32:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T15:32:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T15:32:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T15:32:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T15:32:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T15:32:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T15:32:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T15:32:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T15:32:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T15:32:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T15:32:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T15:32:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T15:32:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T15:32:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T15:32:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T15:32:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T15:32:15.165 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-23T15:32:15.165 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 1/198 2024-03-23T15:32:15.165 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2/198 2024-03-23T15:32:15.165 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 3/198 2024-03-23T15:32:15.165 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 4/198 2024-03-23T15:32:15.165 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1855.g21b3b 5/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 6/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 7/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 8/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 9/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 10/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 11/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 12/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 13/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 14/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_ 15/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 16/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 17/198 2024-03-23T15:32:15.166 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_6 18/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86 19/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 20/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el 21/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9. 22/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_6 23/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 24/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 25/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 26/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 27/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 28/198 2024-03-23T15:32:15.167 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 29/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67. 30/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noa 31/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.n 32/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b 33/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el 34/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 35/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.e 36/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 37/198 2024-03-23T15:32:15.189 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 38/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 39/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-23T15:32:15.190 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-23T15:32:15.191 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-23T15:32:15.192 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-23T15:32:15.193 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-23T15:32:15.194 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-23T15:32:15.194 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-23T15:32:15.194 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-23T15:32:15.194 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-23T15:32:15.194 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-23T15:32:15.195 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-23T15:32:15.195 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-23T15:32:15.195 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-23T15:32:15.195 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-23T15:32:15.195 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-23T15:32:15.195 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-23T15:32:15.196 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-23T15:32:15.196 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-23T15:32:15.196 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-23T15:32:15.196 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-23T15:32:15.196 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-23T15:32:15.196 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-23T15:32:15.197 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-23T15:32:15.198 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-23T15:32:15.199 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-23T15:32:15.200 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 151/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 153/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-23T15:32:15.201 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-23T15:32:15.202 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-23T15:32:15.203 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-23T15:32:15.204 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-23T15:32:15.204 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-23T15:32:15.204 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-23T15:32:15.204 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-23T15:32:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T15:32:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:32:15.744 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-03-23T15:32:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T15:32:15.746 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.747 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:15.748 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-16.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.749 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T15:32:15.750 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T15:32:15.751 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T15:32:15.752 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T15:32:15.753 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T15:32:15.754 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.755 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T15:32:15.756 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T15:32:15.757 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T15:32:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T15:32:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T15:32:15.758 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:32:15.758 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-03-23T15:32:16.269 DEBUG:teuthology.parallel:result is None 2024-03-23T15:32:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: acl-2.3.1-4.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-23T15:32:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-23T15:32:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-1855.g21b3bd67.el9.noarch 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: jq-1.6-16.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-23T15:32:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.11-1.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-23T15:32:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-23T15:32:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-23T15:32:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-23T15:32:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-23T15:32:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-1855.g21b3bd67.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-23T15:32:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-23T15:32:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-23T15:32:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-23T15:32:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-23T15:32:18.182 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-03-23T15:32:18.640 DEBUG:teuthology.parallel:result is None 2024-03-23T15:32:18.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:32:18.855 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T15:32:18.881 INFO:teuthology.orchestra.run.smithi076.stdout:19.0.0-1855.g21b3bd67.el9 2024-03-23T15:32:18.882 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855.g21b3bd67.el9 2024-03-23T15:32:18.882 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855.g21b3bd67 is installed. 2024-03-23T15:32:18.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:32:19.096 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T15:32:19.117 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-1855.g21b3bd67.el9 2024-03-23T15:32:19.117 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855.g21b3bd67.el9 2024-03-23T15:32:19.118 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855.g21b3bd67 is installed. 2024-03-23T15:32:19.119 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:32:19.331 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T15:32:19.356 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-1855.g21b3bd67.el9 2024-03-23T15:32:19.357 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855.g21b3bd67.el9 2024-03-23T15:32:19.357 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855.g21b3bd67 is installed. 2024-03-23T15:32:19.359 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T15:32:19.359 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:32:19.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T15:32:19.392 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:32:19.392 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T15:32:19.422 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:32:19.422 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T15:32:19.456 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T15:32:19.457 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:32:19.457 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T15:32:19.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T15:32:19.556 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:32:19.556 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T15:32:19.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T15:32:19.651 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:32:19.651 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T15:32:19.681 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T15:32:19.750 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T15:32:19.750 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:32:19.750 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T15:32:19.779 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T15:32:19.845 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:32:19.845 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T15:32:19.872 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T15:32:19.939 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:32:19.939 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T15:32:19.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T15:32:20.038 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T15:32:20.038 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:32:20.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T15:32:20.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T15:32:20.140 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:32:20.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T15:32:20.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T15:32:20.233 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:32:20.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T15:32:20.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T15:32:20.331 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-23T15:32:20.427 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-23T15:32:20.427 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:32:20.428 INFO:tasks.cephadm:Cluster fsid is 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:32:20.428 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-23T15:32:20.428 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.76', 'mon.b': '172.21.15.183', 'mon.c': '172.21.15.192'} 2024-03-23T15:32:20.428 INFO:tasks.cephadm:First mon is mon.a on smithi076 2024-03-23T15:32:20.428 INFO:tasks.cephadm:First mgr is x 2024-03-23T15:32:20.428 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-23T15:32:20.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo hostname $(hostname -s) 2024-03-23T15:32:20.456 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-03-23T15:32:20.482 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-03-23T15:32:20.512 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-23T15:32:20.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:32:20.653 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77900/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1855-g21b3bd67', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1855.g21b3bd67'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-23 04:48:23.862772', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240323.003144-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-23T15:32:20.789 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20240323.003144-squid, sha1 21b3bd672163092b4e542f772797fcba02d3d4de from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:32:20.790 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm 2024-03-23T15:32:20.791 DEBUG:teuthology.orchestra.run.smithi076:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:21.252 INFO:teuthology.orchestra.run.smithi076.stdout:-rw-r--r--. 1 ubuntu ubuntu 766828 Mar 23 15:32 /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:21.252 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:21.725 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 766828 Mar 23 15:32 /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:21.726 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:22.198 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 766828 Mar 23 15:32 /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:22.199 DEBUG:teuthology.orchestra.run.smithi076:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:22.218 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:22.236 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T15:32:22.270 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de on all hosts... 2024-03-23T15:32:22.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de pull 2024-03-23T15:32:22.272 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de pull 2024-03-23T15:32:22.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de pull 2024-03-23T15:32:23.125 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T15:32:23.177 INFO:teuthology.orchestra.run.smithi076.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T15:32:23.205 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T15:32:56.354 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-23T15:32:56.355 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T15:32:56.355 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12", 2024-03-23T15:32:56.355 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-03-23T15:32:56.355 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919" 2024-03-23T15:32:56.355 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-03-23T15:32:56.355 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-23T15:33:09.167 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-03-23T15:33:09.167 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T15:33:09.168 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12", 2024-03-23T15:33:09.168 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-03-23T15:33:09.168 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919" 2024-03-23T15:33:09.168 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-03-23T15:33:09.168 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-03-23T15:33:09.719 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:33:09.719 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T15:33:09.719 INFO:teuthology.orchestra.run.smithi076.stdout: "image_id": "553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12", 2024-03-23T15:33:09.719 INFO:teuthology.orchestra.run.smithi076.stdout: "repo_digests": [ 2024-03-23T15:33:09.719 INFO:teuthology.orchestra.run.smithi076.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919" 2024-03-23T15:33:09.719 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:33:09.720 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:33:09.746 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph 2024-03-23T15:33:09.804 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-03-23T15:33:09.835 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-03-23T15:33:09.871 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /etc/ceph 2024-03-23T15:33:09.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-03-23T15:33:09.926 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-03-23T15:33:09.955 INFO:tasks.cephadm:Writing seed config... 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-23T15:33:09.956 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-23T15:33:09.957 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-23T15:33:09.958 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-23T15:33:09.959 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-23T15:33:09.960 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:33:09.960 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-23T15:33:09.976 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 89c3e582-e92a-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-23T15:33:09.976 DEBUG:teuthology.orchestra.run.smithi076:mon.a> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a.service 2024-03-23T15:33:10.018 DEBUG:teuthology.orchestra.run.smithi076:mgr.x> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.x.service 2024-03-23T15:33:10.061 INFO:tasks.cephadm:Bootstrapping... 2024-03-23T15:33:10.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de -v bootstrap --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.76 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:33:10.255 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-03-23T15:33:10.256 INFO:teuthology.orchestra.run.smithi076.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de', '-v', 'bootstrap', '--fsid', '89c3e582-e92a-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.76', '--skip-admin-label'] 2024-03-23T15:33:10.280 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T15:33:10.280 INFO:teuthology.orchestra.run.smithi076.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-23T15:33:10.280 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying podman|docker is present... 2024-03-23T15:33:10.303 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T15:33:10.303 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying lvm2 is present... 2024-03-23T15:33:10.303 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying time synchronization is in place... 2024-03-23T15:33:10.311 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T15:33:10.311 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T15:33:10.316 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T15:33:10.317 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-03-23T15:33:10.323 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout enabled 2024-03-23T15:33:10.329 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout active 2024-03-23T15:33:10.329 INFO:teuthology.orchestra.run.smithi076.stdout:Unit chronyd.service is enabled and running 2024-03-23T15:33:10.329 INFO:teuthology.orchestra.run.smithi076.stdout:Repeating the final host check... 2024-03-23T15:33:10.352 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-23T15:33:10.352 INFO:teuthology.orchestra.run.smithi076.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-23T15:33:10.352 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl is present 2024-03-23T15:33:10.352 INFO:teuthology.orchestra.run.smithi076.stdout:lvcreate is present 2024-03-23T15:33:10.358 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T15:33:10.358 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T15:33:10.364 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T15:33:10.364 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-03-23T15:33:10.370 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout enabled 2024-03-23T15:33:10.376 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout active 2024-03-23T15:33:10.376 INFO:teuthology.orchestra.run.smithi076.stdout:Unit chronyd.service is enabled and running 2024-03-23T15:33:10.376 INFO:teuthology.orchestra.run.smithi076.stdout:Host looks OK 2024-03-23T15:33:10.376 INFO:teuthology.orchestra.run.smithi076.stdout:Cluster fsid: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:10.376 INFO:teuthology.orchestra.run.smithi076.stdout:Acquiring lock 139635779028592 on /run/cephadm/89c3e582-e92a-11ee-95cd-87774f69a715.lock 2024-03-23T15:33:10.377 INFO:teuthology.orchestra.run.smithi076.stdout:Lock 139635779028592 acquired on /run/cephadm/89c3e582-e92a-11ee-95cd-87774f69a715.lock 2024-03-23T15:33:10.377 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying IP 172.21.15.76 port 3300 ... 2024-03-23T15:33:10.377 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying IP 172.21.15.76 port 6789 ... 2024-03-23T15:33:10.377 INFO:teuthology.orchestra.run.smithi076.stdout:Base mon IP(s) is [172.21.15.76:3300, 172.21.15.76:6789], mon addrv is [v2:172.21.15.76:3300,v1:172.21.15.76:6789] 2024-03-23T15:33:10.380 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.76 metric 100 2024-03-23T15:33:10.380 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.76 metric 100 2024-03-23T15:33:10.383 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-23T15:33:10.383 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-23T15:33:10.383 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1788sec hoplimit 64 pref medium 2024-03-23T15:33:10.385 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-23T15:33:10.385 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-23T15:33:10.385 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T15:33:10.385 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-23T15:33:10.386 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-03-23T15:33:10.386 INFO:teuthology.orchestra.run.smithi076.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T15:33:10.386 INFO:teuthology.orchestra.run.smithi076.stdout:Mon IP `172.21.15.76` is in CIDR network `172.21.0.0/20` 2024-03-23T15:33:10.386 INFO:teuthology.orchestra.run.smithi076.stdout:Mon IP `172.21.15.76` is in CIDR network `172.21.0.0/20` 2024-03-23T15:33:10.386 INFO:teuthology.orchestra.run.smithi076.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-23T15:33:10.387 INFO:teuthology.orchestra.run.smithi076.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-23T15:33:10.387 INFO:teuthology.orchestra.run.smithi076.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T15:33:10.829 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stdout 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 2024-03-23T15:33:10.829 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de... 2024-03-23T15:33:10.829 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Getting image source signatures 2024-03-23T15:33:10.829 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Copying blob sha256:30dc77a9f705c02d26e1a1f9e2bfdf9d08cd4086b08b221e8a27d0dd2909924d 2024-03-23T15:33:10.829 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-23T15:33:10.829 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Copying config sha256:553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 2024-03-23T15:33:10.830 INFO:teuthology.orchestra.run.smithi076.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-23T15:33:11.683 INFO:teuthology.orchestra.run.smithi076.stdout:ceph: stdout ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev) 2024-03-23T15:33:11.683 INFO:teuthology.orchestra.run.smithi076.stdout:Ceph version: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev) 2024-03-23T15:33:11.684 INFO:teuthology.orchestra.run.smithi076.stdout:Extracting ceph user uid/gid from container image... 2024-03-23T15:33:12.700 INFO:teuthology.orchestra.run.smithi076.stdout:stat: stdout 167 167 2024-03-23T15:33:12.700 INFO:teuthology.orchestra.run.smithi076.stdout:Creating initial keys... 2024-03-23T15:33:13.580 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph-authtool: stdout AQA59v5lRl3wABAAP1Wz6K+giK77nFhDgzTxow== 2024-03-23T15:33:14.484 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph-authtool: stdout AQA59v5lKPb1NRAA3bnM4C3bnkEOmHPFOZXE6w== 2024-03-23T15:33:15.481 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph-authtool: stdout AQA69v5l8TU2MRAA4kv4dnD1KGSJ9gQHXzFHIg== 2024-03-23T15:33:15.482 INFO:teuthology.orchestra.run.smithi076.stdout:Creating initial monmap... 2024-03-23T15:33:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T15:33:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-23T15:33:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T15:33:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool for a [v2:172.21.15.76:3300,v1:172.21.15.76:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T15:33:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:setting min_mon_release = quincy 2024-03-23T15:33:16.487 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: set fsid to 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T15:33:16.488 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:33:16.488 INFO:teuthology.orchestra.run.smithi076.stdout:Creating mon... 2024-03-23T15:33:17.359 INFO:teuthology.orchestra.run.smithi076.stdout:create mon.a on 2024-03-23T15:33:17.559 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-23T15:33:17.694 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-23T15:33:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-89c3e582-e92a-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-89c3e582-e92a-11ee-95cd-87774f69a715.target. 2024-03-23T15:33:17.869 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-89c3e582-e92a-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-89c3e582-e92a-11ee-95cd-87774f69a715.target. 2024-03-23T15:33:18.114 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a 2024-03-23T15:33:18.115 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to reset failed state of unit ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-23T15:33:18.278 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-89c3e582-e92a-11ee-95cd-87774f69a715.target.wants/ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-89c3e582-e92a-11ee-95cd-87774f69a715@.service. 2024-03-23T15:33:18.790 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T15:33:18.790 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-03-23T15:33:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T15:33:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-03-23T15:33:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-03-23T15:33:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T15:33:18.798 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mon to start... 2024-03-23T15:33:18.799 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mon... 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout cluster: 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout id: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout services: 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.533088s) 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-23T15:33:19.842 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout data: 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout pgs: 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.234+0000 7fb049bf3700 1 Processor -- start 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.234+0000 7fb049bf3700 1 -- start start 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.234+0000 7fb049bf3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.234+0000 7fb049bf3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb044106980 con 0x7fb044105f90 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.234+0000 7fb0437fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:19.843 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.234+0000 7fb0437fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36100/0 (socket says 172.21.15.76:36100) 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.234+0000 7fb0437fe700 1 -- 172.21.15.76:0/1174236895 learned_addr learned my addr 172.21.15.76:0/1174236895 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.235+0000 7fb0437fe700 1 -- 172.21.15.76:0/1174236895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb044108f20 con 0x7fb044105f90 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.235+0000 7fb0437fe700 1 --2- 172.21.15.76:0/1174236895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441063b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb0340098a0 tx=0x7fb034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd5da0dc269fa32 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.235+0000 7fb0427fc700 1 -- 172.21.15.76:0/1174236895 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb034004160 con 0x7fb044105f90 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.235+0000 7fb0427fc700 1 -- 172.21.15.76:0/1174236895 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0340042c0 con 0x7fb044105f90 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.235+0000 7fb0427fc700 1 -- 172.21.15.76:0/1174236895 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0340045a0 con 0x7fb044105f90 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.235+0000 7fb049bf3700 1 -- 172.21.15.76:0/1174236895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 msgr2=0x7fb0441063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.235+0000 7fb049bf3700 1 --2- 172.21.15.76:0/1174236895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441063b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb0340098a0 tx=0x7fb034009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.236+0000 7fb049bf3700 1 -- 172.21.15.76:0/1174236895 shutdown_connections 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.236+0000 7fb049bf3700 1 --2- 172.21.15.76:0/1174236895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441063b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.236+0000 7fb049bf3700 1 -- 172.21.15.76:0/1174236895 >> 172.21.15.76:0/1174236895 conn(0x7fb044101900 msgr2=0x7fb044103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.236+0000 7fb049bf3700 1 -- 172.21.15.76:0/1174236895 shutdown_connections 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.236+0000 7fb049bf3700 1 -- 172.21.15.76:0/1174236895 wait complete. 2024-03-23T15:33:19.844 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.236+0000 7fb049bf3700 1 Processor -- start 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.236+0000 7fb049bf3700 1 -- start start 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb049bf3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441087a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb049bf3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb044108ce0 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb0437fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb0437fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36116/0 (socket says 172.21.15.76:36116) 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb0437fe700 1 -- 172.21.15.76:0/1720769828 learned_addr learned my addr 172.21.15.76:0/1720769828 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb0437fe700 1 -- 172.21.15.76:0/1720769828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb034009550 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb0437fe700 1 --2- 172.21.15.76:0/1720769828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441087a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb034004000 tx=0x7fb034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb040ff9700 1 -- 172.21.15.76:0/1720769828 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb034011630 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb040ff9700 1 -- 172.21.15.76:0/1720769828 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb034011790 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb044106ed0 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb040ff9700 1 -- 172.21.15.76:0/1720769828 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb034011a30 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb044107370 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.237+0000 7fb040ff9700 1 -- 172.21.15.76:0/1720769828 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb034022070 con 0x7fb044105f90 2024-03-23T15:33:19.845 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.238+0000 7fb040ff9700 1 -- 172.21.15.76:0/1720769828 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb03401c450 con 0x7fb044105f90 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.238+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb044107d80 con 0x7fb044105f90 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.241+0000 7fb040ff9700 1 -- 172.21.15.76:0/1720769828 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb034011b90 con 0x7fb044105f90 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.310+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb044069300 con 0x7fb044105f90 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.310+0000 7fb040ff9700 1 -- 172.21.15.76:0/1720769828 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb034011d30 con 0x7fb044105f90 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.311+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 msgr2=0x7fb0441087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.311+0000 7fb049bf3700 1 --2- 172.21.15.76:0/1720769828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441087a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb034004000 tx=0x7fb034004750 comp rx=0 tx=0).stop 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.311+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 shutdown_connections 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.311+0000 7fb049bf3700 1 --2- 172.21.15.76:0/1720769828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb044105f90 0x7fb0441087a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.311+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 >> 172.21.15.76:0/1720769828 conn(0x7fb044101900 msgr2=0x7fb044191ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.312+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 shutdown_connections 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:19.312+0000 7fb049bf3700 1 -- 172.21.15.76:0/1720769828 wait complete. 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:mon is available 2024-03-23T15:33:19.846 INFO:teuthology.orchestra.run.smithi076.stdout:Assimilating anything we can from ceph.conf... 2024-03-23T15:33:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T15:33:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout fsid = 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:20.831 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.76:3300,v1:172.21.15.76:6789] 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T15:33:20.832 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.206+0000 7f480591c700 1 Processor -- start 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.206+0000 7f480591c700 1 -- start start 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.206+0000 7f480591c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f48001080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.206+0000 7f480591c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4800108690 con 0x7f4800107ca0 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.206+0000 7f47feffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f48001080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.206+0000 7f47feffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f48001080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36124/0 (socket says 172.21.15.76:36124) 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.206+0000 7f47feffd700 1 -- 172.21.15.76:0/1471050028 learned_addr learned my addr 172.21.15.76:0/1471050028 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.207+0000 7f47feffd700 1 -- 172.21.15.76:0/1471050028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4800108ea0 con 0x7f4800107ca0 2024-03-23T15:33:20.833 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.207+0000 7f47feffd700 1 --2- 172.21.15.76:0/1471050028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f48001080c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47f00098a0 tx=0x7f47f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87167b6f8ddfdfe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.207+0000 7f47fdffb700 1 -- 172.21.15.76:0/1471050028 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f0004160 con 0x7f4800107ca0 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.207+0000 7f47fdffb700 1 -- 172.21.15.76:0/1471050028 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f00042c0 con 0x7f4800107ca0 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.207+0000 7f47fdffb700 1 -- 172.21.15.76:0/1471050028 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f00045a0 con 0x7f4800107ca0 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.207+0000 7f480591c700 1 -- 172.21.15.76:0/1471050028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 msgr2=0x7f48001080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.207+0000 7f480591c700 1 --2- 172.21.15.76:0/1471050028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f48001080c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47f00098a0 tx=0x7f47f0009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.208+0000 7f480591c700 1 -- 172.21.15.76:0/1471050028 shutdown_connections 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.208+0000 7f480591c700 1 --2- 172.21.15.76:0/1471050028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f48001080c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.208+0000 7f480591c700 1 -- 172.21.15.76:0/1471050028 >> 172.21.15.76:0/1471050028 conn(0x7f4800103280 msgr2=0x7f48001056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.208+0000 7f480591c700 1 -- 172.21.15.76:0/1471050028 shutdown_connections 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.208+0000 7f480591c700 1 -- 172.21.15.76:0/1471050028 wait complete. 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.208+0000 7f480591c700 1 Processor -- start 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.208+0000 7f480591c700 1 -- start start 2024-03-23T15:33:20.834 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f480591c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f480019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f480591c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f480019b600 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f47feffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f480019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f47feffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f480019b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36140/0 (socket says 172.21.15.76:36140) 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f47feffd700 1 -- 172.21.15.76:0/3899614339 learned_addr learned my addr 172.21.15.76:0/3899614339 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f47feffd700 1 -- 172.21.15.76:0/3899614339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f0009550 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f47feffd700 1 --2- 172.21.15.76:0/3899614339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f480019b0c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f47f0004000 tx=0x7f47f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f0011630 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f0011790 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480019b800 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f0011a30 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.209+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480019bca0 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.210+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f47f0022070 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.210+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f47f001c450 con 0x7f4800107ca0 2024-03-23T15:33:20.835 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.210+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4800194d80 con 0x7f4800107ca0 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.213+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f47f0011b90 con 0x7f4800107ca0 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.280+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4800109480 con 0x7f4800107ca0 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.283+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f47f0011de0 con 0x7f4800107ca0 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.284+0000 7f480491a700 1 -- 172.21.15.76:0/3899614339 <== mon.0 v2:172.21.15.76:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47f001c660 con 0x7f4800107ca0 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.284+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 msgr2=0x7f480019b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.284+0000 7f480591c700 1 --2- 172.21.15.76:0/3899614339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f480019b0c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f47f0004000 tx=0x7f47f0004750 comp rx=0 tx=0).stop 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.285+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 shutdown_connections 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.285+0000 7f480591c700 1 --2- 172.21.15.76:0/3899614339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4800107ca0 0x7f480019b0c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.285+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 >> 172.21.15.76:0/3899614339 conn(0x7f4800103280 msgr2=0x7f4800191ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.285+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 shutdown_connections 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:20.285+0000 7f480591c700 1 -- 172.21.15.76:0/3899614339 wait complete. 2024-03-23T15:33:20.836 INFO:teuthology.orchestra.run.smithi076.stdout:Generating new minimal ceph.conf... 2024-03-23T15:33:21.927 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.253+0000 7fafd9341700 1 Processor -- start 2024-03-23T15:33:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.253+0000 7fafd9341700 1 -- start start 2024-03-23T15:33:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.253+0000 7fafd9341700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd4106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd9341700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41075b0 con 0x7fafd4106bc0 2024-03-23T15:33:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd2ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd4106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd2ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd4106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36148/0 (socket says 172.21.15.76:36148) 2024-03-23T15:33:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd2ffd700 1 -- 172.21.15.76:0/612935174 learned_addr learned my addr 172.21.15.76:0/612935174 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:21.928 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd2ffd700 1 -- 172.21.15.76:0/612935174 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd4107dc0 con 0x7fafd4106bc0 2024-03-23T15:33:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd2ffd700 1 --2- 172.21.15.76:0/612935174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd4106fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009e80 tx=0x7fafcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef50300f35f21f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd1ffb700 1 -- 172.21.15.76:0/612935174 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc00e070 con 0x7fafd4106bc0 2024-03-23T15:33:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd1ffb700 1 -- 172.21.15.76:0/612935174 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fafcc00b8c0 con 0x7fafd4106bc0 2024-03-23T15:33:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.254+0000 7fafd1ffb700 1 -- 172.21.15.76:0/612935174 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc004550 con 0x7fafd4106bc0 2024-03-23T15:33:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 -- 172.21.15.76:0/612935174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 msgr2=0x7fafd4106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 --2- 172.21.15.76:0/612935174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd4106fe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009e80 tx=0x7fafcc00b330 comp rx=0 tx=0).stop 2024-03-23T15:33:21.929 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 -- 172.21.15.76:0/612935174 shutdown_connections 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 --2- 172.21.15.76:0/612935174 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd4106fe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 -- 172.21.15.76:0/612935174 >> 172.21.15.76:0/612935174 conn(0x7fafd4102160 msgr2=0x7fafd41045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 -- 172.21.15.76:0/612935174 shutdown_connections 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 -- 172.21.15.76:0/612935174 wait complete. 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.255+0000 7fafd9341700 1 Processor -- start 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd9341700 1 -- start start 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd9341700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd9341700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fafd419be70 con 0x7fafd4106bc0 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd2ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd2ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd419b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36162/0 (socket says 172.21.15.76:36162) 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd2ffd700 1 -- 172.21.15.76:0/121256038 learned_addr learned my addr 172.21.15.76:0/121256038 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd2ffd700 1 -- 172.21.15.76:0/121256038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc009b30 con 0x7fafd4106bc0 2024-03-23T15:33:21.930 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd2ffd700 1 --2- 172.21.15.76:0/121256038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd419b930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00b7b0 tx=0x7fafcc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafbbfff700 1 -- 172.21.15.76:0/121256038 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc00e040 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafbbfff700 1 -- 172.21.15.76:0/121256038 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fafcc00bd70 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafbbfff700 1 -- 172.21.15.76:0/121256038 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc01c770 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.256+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd419c070 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.257+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd419c4f0 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.257+0000 7fafbbfff700 1 -- 172.21.15.76:0/121256038 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fafcc01c970 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.257+0000 7fafbbfff700 1 -- 172.21.15.76:0/121256038 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fafcc007800 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.257+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd4195540 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.260+0000 7fafbbfff700 1 -- 172.21.15.76:0/121256038 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fafcc02a060 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.326+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fafd410b220 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.326+0000 7fafbbfff700 1 -- 172.21.15.76:0/121256038 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fafcc020070 con 0x7fafd4106bc0 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.327+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 msgr2=0x7fafd419b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:21.931 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.327+0000 7fafd9341700 1 --2- 172.21.15.76:0/121256038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd419b930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00b7b0 tx=0x7fafcc003d00 comp rx=0 tx=0).stop 2024-03-23T15:33:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.327+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 shutdown_connections 2024-03-23T15:33:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.327+0000 7fafd9341700 1 --2- 172.21.15.76:0/121256038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafd4106bc0 0x7fafd419b930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.327+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 >> 172.21.15.76:0/121256038 conn(0x7fafd4102160 msgr2=0x7fafd4192680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.327+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 shutdown_connections 2024-03-23T15:33:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:21.327+0000 7fafd9341700 1 -- 172.21.15.76:0/121256038 wait complete. 2024-03-23T15:33:21.932 INFO:teuthology.orchestra.run.smithi076.stdout:Restarting the monitor... 2024-03-23T15:33:22.280 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:22 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[31801]: 2024-03-23T15:33:22.196+0000 7ffaf0240700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-23T15:33:22.540 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:22 smithi076 podman[32050]: 2024-03-23 15:33:22.278780142 +0000 UTC m=+0.191332852 container died 17db328a4ef950747bbe34928da186a35774566d60f29626273132aba60aea0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:33:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:22 smithi076 podman[32050]: 2024-03-23 15:33:22.637852012 +0000 UTC m=+0.550404728 container cleanup 17db328a4ef950747bbe34928da186a35774566d60f29626273132aba60aea0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-23T15:33:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:22 smithi076 bash[32050]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a 2024-03-23T15:33:23.308 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:22 smithi076 podman[32064]: 2024-03-23 15:33:22.938811017 +0000 UTC m=+0.664016937 container remove 17db328a4ef950747bbe34928da186a35774566d60f29626273132aba60aea0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-23T15:33:23.308 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-23T15:33:23.308 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 systemd[1]: Stopped Ceph mon.a for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:33:23.308 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 systemd[1]: Starting Ceph mon.a for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:33:23.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 podman[32157]: 2024-03-23 15:33:23.306211439 +0000 UTC m=+0.101537203 container create c68c8b1c86ce1ee38c431a34ef451ed00d50c6a528970cf74c4d08a5fdc19731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 podman[32157]: 2024-03-23 15:33:23.222261962 +0000 UTC m=+0.017587694 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 podman[32157]: 2024-03-23 15:33:23.508999576 +0000 UTC m=+0.304325304 container init c68c8b1c86ce1ee38c431a34ef451ed00d50c6a528970cf74c4d08a5fdc19731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 podman[32157]: 2024-03-23 15:33:23.512043512 +0000 UTC m=+0.307369239 container start c68c8b1c86ce1ee38c431a34ef451ed00d50c6a528970cf74c4d08a5fdc19731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux ) 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev), process ceph-mon, pid 2 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: pidfile_write: ignore empty --pid-file 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: load: jerasure load: lrc 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: RocksDB version: 7.9.2 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Git sha 0 2024-03-23T15:33:23.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Compile date 2024-03-23 03:28:38 2024-03-23T15:33:23.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: DB SUMMARY 2024-03-23T15:33:23.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: DB Session ID: QJI4N0VVWJQUNJAHC70N 2024-03-23T15:33:23.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: CURRENT file: CURRENT 2024-03-23T15:33:23.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T15:33:23.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-23T15:33:23.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-23T15:33:23.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105314 ; 2024-03-23T15:33:23.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.error_if_exists: 0 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.create_if_missing: 0 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.paranoid_checks: 1 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.env: 0x55bb94d27c20 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.info_log: 0x55bb96f7fac0 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.statistics: (nil) 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.use_fsync: 0 2024-03-23T15:33:23.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_log_file_size: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.allow_fallocate: 1 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.use_direct_reads: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.db_log_dir: 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.wal_dir: 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T15:33:23.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.write_buffer_manager: 0x55bb96f521e0 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T15:33:23.581 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.unordered_write: 0 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.row_cache: None 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.wal_filter: None 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T15:33:23.582 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.two_write_queues: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.wal_compression: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.atomic_flush: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.log_readahead_size: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T15:33:23.583 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_background_jobs: 2 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_background_compactions: -1 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_subcompactions: 1 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_open_files: -1 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T15:33:23.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_background_flushes: -1 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Compression algorithms supported: 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kZSTD supported: 0 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kXpressCompression supported: 0 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kZlibCompression supported: 1 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kSnappyCompression supported: 1 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kLZ4Compression supported: 1 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: kBZip2Compression supported: 0 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-23T15:33:23.585 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.merge_operator: 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_filter: None 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bb96f7fbc0) 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout: cache_index_and_filter_blocks: 1 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout: pin_top_level_index_and_filter: 1 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout: index_type: 0 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout: data_block_index_type: 0 2024-03-23T15:33:23.586 INFO:journalctl@ceph.mon.a.smithi076.stdout: index_shortening: 1 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: checksum: 4 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: no_block_cache: 0 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache: 0x55bb96f83090 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache_name: BinnedLRUCache 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache_options: 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: capacity : 536870912 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: num_shard_bits : 4 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: strict_capacity_limit : 0 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: high_pri_pool_ratio: 0.000 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_cache_compressed: (nil) 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: persistent_cache: (nil) 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_size: 4096 2024-03-23T15:33:23.587 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_size_deviation: 10 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_restart_interval: 16 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: index_block_restart_interval: 1 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: metadata_block_size: 4096 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: partition_filters: 0 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: use_delta_encoding: 1 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: filter_policy: bloomfilter 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: whole_key_filtering: 1 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: verify_compression: 0 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: read_amp_bytes_per_bit: 0 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: format_version: 5 2024-03-23T15:33:23.588 INFO:journalctl@ceph.mon.a.smithi076.stdout: enable_index_compression: 1 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout: block_align: 0 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout: max_auto_readahead_size: 262144 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout: prepopulate_block_cache: 0 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout: initial_auto_readahead_size: 8192 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression: NoCompression 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.num_levels: 7 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T15:33:23.589 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T15:33:23.590 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T15:33:23.591 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T15:33:23.592 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.table_properties_collectors: 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.inplace_update_support: 0 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T15:33:23.593 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.bloom_locality: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.max_successive_merges: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.ttl: 2592000 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.enable_blob_files: false 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.min_blob_size: 0 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T15:33:23.594 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c56ac4ff-816a-4d80-9377-c89fc30e466f 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208003540933, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208003541950, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100357, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98277, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14306, "raw_average_key_size": 49, "raw_value_size": 90523, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711208003, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c56ac4ff-816a-4d80-9377-c89fc30e466f", "db_session_id": "QJI4N0VVWJQUNJAHC70N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208003542030, "job": 1, "event": "recovery_finished"} 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T15:33:23.595 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bb97074000 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: DB pointer 0x55bb9705e000 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** DB Stats ** 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** Compaction Stats [default] ** 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T15:33:23.596 INFO:journalctl@ceph.mon.a.smithi076.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: L0 2/0 99.84 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: Sum 2/0 99.84 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** Compaction Stats [default] ** 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T15:33:23.597 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: Cumulative compaction: 0.00 GB write, 32.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: Interval compaction: 0.00 GB write, 32.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: Block cache BinnedLRUCache@0x55bb96f83090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 7e-06 secs_since: 0 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: starting mon.a rank 0 at public addrs [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] at bind addrs [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???) e1 preinit fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???).mds e1 new map 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???).mds e1 print_map 2024-03-23T15:33:23.598 INFO:journalctl@ceph.mon.a.smithi076.stdout: e1 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout: legacy client fscid: -1 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout: 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout: No filesystems configured 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:33:23.599 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-23T15:33:23.607 INFO:teuthology.orchestra.run.smithi076.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-23T15:33:23.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 bash[32157]: c68c8b1c86ce1ee38c431a34ef451ed00d50c6a528970cf74c4d08a5fdc19731 2024-03-23T15:33:23.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 systemd[1]: Started Ceph mon.a for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:33:23.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-23T15:33:23.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: monmap epoch 1 2024-03-23T15:33:23.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:23.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: last_changed 2024-03-23T15:33:15.904675+0000 2024-03-23T15:33:23.914 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:33:23.914 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: min_mon_release 19 (squid) 2024-03-23T15:33:23.914 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: election_strategy: 1 2024-03-23T15:33:23.914 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:33:23.914 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: fsmap 2024-03-23T15:33:23.914 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: osdmap e1: 0 total, 0 up, 0 in 2024-03-23T15:33:23.914 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:23 smithi076 ceph-mon[32176]: mgrmap e1: no daemons active 2024-03-23T15:33:25.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2448584834' entity='client.admin' 2024-03-23T15:33:28.198 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.013+0000 7f7a4afc6700 1 Processor -- start 2024-03-23T15:33:28.198 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.013+0000 7f7a4afc6700 1 -- start start 2024-03-23T15:33:28.198 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.013+0000 7f7a4afc6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a441080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:28.198 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.013+0000 7f7a4afc6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441086b0 con 0x7f7a44107cc0 2024-03-23T15:33:28.198 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.013+0000 7f7a48d62700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a441080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.013+0000 7f7a48d62700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a441080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36164/0 (socket says 172.21.15.76:36164) 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.013+0000 7f7a48d62700 1 -- 172.21.15.76:0/4016554699 learned_addr learned my addr 172.21.15.76:0/4016554699 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a48d62700 1 -- 172.21.15.76:0/4016554699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a44108ec0 con 0x7f7a44107cc0 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a48d62700 1 --2- 172.21.15.76:0/4016554699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a441080e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7a380098a0 tx=0x7f7a38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631a969040bce386 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a437fe700 1 -- 172.21.15.76:0/4016554699 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a38004160 con 0x7f7a44107cc0 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a437fe700 1 -- 172.21.15.76:0/4016554699 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a380042c0 con 0x7f7a44107cc0 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a437fe700 1 -- 172.21.15.76:0/4016554699 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a38011ac0 con 0x7f7a44107cc0 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a4afc6700 1 -- 172.21.15.76:0/4016554699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 msgr2=0x7f7a441080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a4afc6700 1 --2- 172.21.15.76:0/4016554699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a441080e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7a380098a0 tx=0x7f7a38009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a4afc6700 1 -- 172.21.15.76:0/4016554699 shutdown_connections 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a4afc6700 1 --2- 172.21.15.76:0/4016554699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a441080e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:28.199 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a4afc6700 1 -- 172.21.15.76:0/4016554699 >> 172.21.15.76:0/4016554699 conn(0x7f7a44103280 msgr2=0x7f7a441056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:28.200 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.014+0000 7f7a4afc6700 1 -- 172.21.15.76:0/4016554699 shutdown_connections 2024-03-23T15:33:28.200 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a4afc6700 1 -- 172.21.15.76:0/4016554699 wait complete. 2024-03-23T15:33:28.200 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a4afc6700 1 Processor -- start 2024-03-23T15:33:28.200 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a4afc6700 1 -- start start 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a4afc6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a4419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a4afc6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4419bf20 con 0x7f7a44107cc0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a48d62700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a4419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a48d62700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a4419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36166/0 (socket says 172.21.15.76:36166) 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.015+0000 7f7a48d62700 1 -- 172.21.15.76:0/2448584834 learned_addr learned my addr 172.21.15.76:0/2448584834 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a48d62700 1 -- 172.21.15.76:0/2448584834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a38009550 con 0x7f7a44107cc0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a48d62700 1 --2- 172.21.15.76:0/2448584834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a4419b9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7a38011780 tx=0x7f7a38011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a41ffb700 1 -- 172.21.15.76:0/2448584834 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a38003af0 con 0x7f7a44107cc0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a41ffb700 1 -- 172.21.15.76:0/2448584834 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a38003c50 con 0x7f7a44107cc0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4419c120 con 0x7f7a44107cc0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a41ffb700 1 -- 172.21.15.76:0/2448584834 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a380197e0 con 0x7f7a44107cc0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4419c5c0 con 0x7f7a44107cc0 2024-03-23T15:33:28.201 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a41ffb700 1 -- 172.21.15.76:0/2448584834 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a38025460 con 0x7f7a44107cc0 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.016+0000 7f7a41ffb700 1 -- 172.21.15.76:0/2448584834 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a3800bb20 con 0x7f7a44107cc0 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.017+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4410c0f0 con 0x7f7a44107cc0 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.019+0000 7f7a41ffb700 1 -- 172.21.15.76:0/2448584834 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7a3802a060 con 0x7f7a44107cc0 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.086+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7a4410c300 con 0x7f7a44107cc0 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.088+0000 7f7a41ffb700 1 -- 172.21.15.76:0/2448584834 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7a38019d60 con 0x7f7a44107cc0 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.089+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 msgr2=0x7f7a4419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.089+0000 7f7a4afc6700 1 --2- 172.21.15.76:0/2448584834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a4419b9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7a38011780 tx=0x7f7a38011b40 comp rx=0 tx=0).stop 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.089+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 shutdown_connections 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.089+0000 7f7a4afc6700 1 --2- 172.21.15.76:0/2448584834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a44107cc0 0x7f7a4419b9e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.089+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 >> 172.21.15.76:0/2448584834 conn(0x7f7a44103280 msgr2=0x7f7a44103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.089+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 shutdown_connections 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:24.090+0000 7f7a4afc6700 1 -- 172.21.15.76:0/2448584834 wait complete. 2024-03-23T15:33:28.202 INFO:teuthology.orchestra.run.smithi076.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-23T15:33:28.470 INFO:teuthology.orchestra.run.smithi076.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:33:28.471 INFO:teuthology.orchestra.run.smithi076.stdout:Creating mgr... 2024-03-23T15:33:28.471 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-23T15:33:28.471 INFO:teuthology.orchestra.run.smithi076.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-23T15:33:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.x 2024-03-23T15:33:29.442 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Failed to reset failed state of unit ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-23T15:33:29.649 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-89c3e582-e92a-11ee-95cd-87774f69a715.target.wants/ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-89c3e582-e92a-11ee-95cd-87774f69a715@.service. 2024-03-23T15:33:29.826 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:29 smithi076 systemd[1]: Starting Ceph mgr.x for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:33:30.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:29 smithi076 podman[32395]: 2024-03-23 15:33:29.78277808 +0000 UTC m=+0.017548175 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T15:33:31.076 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:30 smithi076 podman[32395]: 2024-03-23 15:33:30.721872618 +0000 UTC m=+0.956642710 container create 333621a8ab1baeb008f528fd543a8a3b5f91a73245dc4ef50e76d654cc0a2282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-23T15:33:33.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:32 smithi076 podman[32395]: 2024-03-23 15:33:32.620996005 +0000 UTC m=+2.855766098 container init 333621a8ab1baeb008f528fd543a8a3b5f91a73245dc4ef50e76d654cc0a2282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-23T15:33:33.076 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:32 smithi076 podman[32395]: 2024-03-23 15:33:32.624584404 +0000 UTC m=+2.859354497 container start 333621a8ab1baeb008f528fd543a8a3b5f91a73245dc4ef50e76d654cc0a2282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:33:33.523 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:33.207+0000 7fb7ae9eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:33:33.523 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:33.300+0000 7fb7ae9eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:33:33.536 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T15:33:33.537 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-03-23T15:33:33.543 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T15:33:33.544 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-03-23T15:33:33.544 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-03-23T15:33:33.544 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T15:33:33.552 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T15:33:33.552 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-03-23T15:33:33.559 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T15:33:33.559 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-03-23T15:33:33.560 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-03-23T15:33:33.560 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-23T15:33:33.560 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr to start... 2024-03-23T15:33:33.560 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr... 2024-03-23T15:33:33.806 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:33 smithi076 bash[32395]: 333621a8ab1baeb008f528fd543a8a3b5f91a73245dc4ef50e76d654cc0a2282 2024-03-23T15:33:33.806 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:33 smithi076 systemd[1]: Started Ceph mgr.x for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:33:34.066 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:33.949+0000 7fb7ae9eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:33:34.066 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.044+0000 7fb7ae9eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:33:34.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1589037371' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T15:33:34.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.192+0000 7fb7ae9eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:33:34.326 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.291+0000 7fb7ae9eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:33:34.641 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.374+0000 7fb7ae9eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:33:34.768 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 0 2024-03-23T15:33:34.769 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T15:33:34.770 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T15:33:34.771 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T15:33:34.772 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T15:33:18.792864+0000", 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.092+0000 7fceb10b9700 1 Processor -- start 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.092+0000 7fceb10b9700 1 -- start start 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.092+0000 7fceb10b9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:34.773 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.092+0000 7fceb10b9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fceac107580 con 0x7fceac106b90 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.092+0000 7fceaad9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.092+0000 7fceaad9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45674/0 (socket says 172.21.15.76:45674) 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.092+0000 7fceaad9d700 1 -- 172.21.15.76:0/3216605157 learned_addr learned my addr 172.21.15.76:0/3216605157 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.093+0000 7fceaad9d700 1 -- 172.21.15.76:0/3216605157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac107d90 con 0x7fceac106b90 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.093+0000 7fceaad9d700 1 --2- 172.21.15.76:0/3216605157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac106fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcea00098a0 tx=0x7fcea0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16030df9ed3d5f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.093+0000 7fcea9d9b700 1 -- 172.21.15.76:0/3216605157 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea0004160 con 0x7fceac106b90 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.093+0000 7fcea9d9b700 1 -- 172.21.15.76:0/3216605157 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcea00042c0 con 0x7fceac106b90 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.093+0000 7fcea9d9b700 1 -- 172.21.15.76:0/3216605157 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea0011ac0 con 0x7fceac106b90 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.094+0000 7fceb10b9700 1 -- 172.21.15.76:0/3216605157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 msgr2=0x7fceac106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.094+0000 7fceb10b9700 1 --2- 172.21.15.76:0/3216605157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac106fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcea00098a0 tx=0x7fcea0009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:34.774 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.094+0000 7fceb10b9700 1 -- 172.21.15.76:0/3216605157 shutdown_connections 2024-03-23T15:33:34.775 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.094+0000 7fceb10b9700 1 --2- 172.21.15.76:0/3216605157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac106fb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:34.775 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.094+0000 7fceb10b9700 1 -- 172.21.15.76:0/3216605157 >> 172.21.15.76:0/3216605157 conn(0x7fceac102110 msgr2=0x7fceac104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:34.775 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.094+0000 7fceb10b9700 1 -- 172.21.15.76:0/3216605157 shutdown_connections 2024-03-23T15:33:34.775 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.094+0000 7fceb10b9700 1 -- 172.21.15.76:0/3216605157 wait complete. 2024-03-23T15:33:34.775 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.095+0000 7fceb10b9700 1 Processor -- start 2024-03-23T15:33:34.775 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.095+0000 7fceb10b9700 1 -- start start 2024-03-23T15:33:34.775 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.095+0000 7fceb10b9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:34.776 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.095+0000 7fceb10b9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fceac19beb0 con 0x7fceac106b90 2024-03-23T15:33:34.776 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.095+0000 7fceaad9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:34.776 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.095+0000 7fceaad9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45690/0 (socket says 172.21.15.76:45690) 2024-03-23T15:33:34.776 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.095+0000 7fceaad9d700 1 -- 172.21.15.76:0/1589037371 learned_addr learned my addr 172.21.15.76:0/1589037371 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:34.776 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fceaad9d700 1 -- 172.21.15.76:0/1589037371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea0009550 con 0x7fceac106b90 2024-03-23T15:33:34.776 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fceaad9d700 1 --2- 172.21.15.76:0/1589037371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac19b970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000c00 tx=0x7fcea0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fce93fff700 1 -- 172.21.15.76:0/1589037371 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea0003aa0 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fce93fff700 1 -- 172.21.15.76:0/1589037371 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcea0003c00 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fce93fff700 1 -- 172.21.15.76:0/1589037371 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea00248a0 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac19c0b0 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac19c490 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fce93fff700 1 -- 172.21.15.76:0/1589037371 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcea002d3f0 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.096+0000 7fce93fff700 1 -- 172.21.15.76:0/1589037371 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcea0019770 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.097+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceac195590 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.099+0000 7fce93fff700 1 -- 172.21.15.76:0/1589037371 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcea001ac60 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.171+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fceac10b1f0 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.172+0000 7fce93fff700 1 -- 172.21.15.76:0/1589037371 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fcea0035090 con 0x7fceac106b90 2024-03-23T15:33:34.777 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.172+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 msgr2=0x7fceac19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:34.778 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.172+0000 7fceb10b9700 1 --2- 172.21.15.76:0/1589037371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac19b970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000c00 tx=0x7fcea0011b40 comp rx=0 tx=0).stop 2024-03-23T15:33:34.778 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.172+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 shutdown_connections 2024-03-23T15:33:34.778 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.172+0000 7fceb10b9700 1 --2- 172.21.15.76:0/1589037371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fceac106b90 0x7fceac19b970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:34.778 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.172+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 >> 172.21.15.76:0/1589037371 conn(0x7fceac102110 msgr2=0x7fceac1926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:34.778 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.173+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 shutdown_connections 2024-03-23T15:33:34.778 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:34.173+0000 7fceb10b9700 1 -- 172.21.15.76:0/1589037371 wait complete. 2024-03-23T15:33:34.778 INFO:teuthology.orchestra.run.smithi076.stdout:mgr not available, waiting (1/15)... 2024-03-23T15:33:34.932 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.639+0000 7fb7ae9eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:33:34.932 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.749+0000 7fb7ae9eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:33:34.932 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.836+0000 7fb7ae9eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:33:34.932 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:34.930+0000 7fb7ae9eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:33:35.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:35.102+0000 7fb7ae9eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:33:36.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:36.059+0000 7fb7ae9eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:33:36.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:36.189+0000 7fb7ae9eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:33:36.807 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:36.518+0000 7fb7ae9eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:33:36.807 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:36.606+0000 7fb7ae9eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:33:36.807 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:36.804+0000 7fb7ae9eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:33:37.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:36.990+0000 7fb7ae9eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:33:37.551 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:37 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3048681226' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T15:33:37.551 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:37.284+0000 7fb7ae9eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:33:37.551 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:37.547+0000 7fb7ae9eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:33:37.825 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:37.786+0000 7fb7ae9eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T15:33:37.907 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 0 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T15:33:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T15:33:37.910 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T15:33:18.792864+0000", 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.198+0000 7f6720b93700 1 Processor -- start 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.198+0000 7f6720b93700 1 -- start start 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.198+0000 7f6720b93700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c10ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:37.911 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.198+0000 7f6720b93700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f671c07bdc0 con 0x7f671c108810 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.198+0000 7f671a59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c10ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.198+0000 7f671a59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c10ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46750/0 (socket says 172.21.15.76:46750) 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.198+0000 7f671a59c700 1 -- 172.21.15.76:0/2063374160 learned_addr learned my addr 172.21.15.76:0/2063374160 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.199+0000 7f671a59c700 1 -- 172.21.15.76:0/2063374160 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c07bf00 con 0x7f671c108810 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.199+0000 7f671a59c700 1 --2- 172.21.15.76:0/2063374160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c10ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098a0 tx=0x7f670c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9248515a4a8b3541 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.199+0000 7f671959a700 1 -- 172.21.15.76:0/2063374160 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c004160 con 0x7f671c108810 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.199+0000 7f671959a700 1 -- 172.21.15.76:0/2063374160 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f670c0042c0 con 0x7f671c108810 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.199+0000 7f671959a700 1 -- 172.21.15.76:0/2063374160 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c011ac0 con 0x7f671c108810 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 -- 172.21.15.76:0/2063374160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 msgr2=0x7f671c10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 --2- 172.21.15.76:0/2063374160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c10ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098a0 tx=0x7f670c009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 -- 172.21.15.76:0/2063374160 shutdown_connections 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 --2- 172.21.15.76:0/2063374160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c10ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 -- 172.21.15.76:0/2063374160 >> 172.21.15.76:0/2063374160 conn(0x7f671c102160 msgr2=0x7f671c1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 -- 172.21.15.76:0/2063374160 shutdown_connections 2024-03-23T15:33:37.912 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 -- 172.21.15.76:0/2063374160 wait complete. 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.200+0000 7f6720b93700 1 Processor -- start 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f6720b93700 1 -- start start 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f6720b93700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f6720b93700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f671c19bec0 con 0x7f671c108810 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f671a59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f671a59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46760/0 (socket says 172.21.15.76:46760) 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f671a59c700 1 -- 172.21.15.76:0/3048681226 learned_addr learned my addr 172.21.15.76:0/3048681226 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f671a59c700 1 -- 172.21.15.76:0/3048681226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c009550 con 0x7f671c108810 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f671a59c700 1 --2- 172.21.15.76:0/3048681226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c19b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f670c009fb0 tx=0x7f670c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f670b7fe700 1 -- 172.21.15.76:0/3048681226 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c0038b0 con 0x7f671c108810 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.201+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c19c0c0 con 0x7f671c108810 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.202+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c19c560 con 0x7f671c108810 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.202+0000 7f670b7fe700 1 -- 172.21.15.76:0/3048681226 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f670c003a10 con 0x7f671c108810 2024-03-23T15:33:37.913 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.202+0000 7f670b7fe700 1 -- 172.21.15.76:0/3048681226 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f670c0195e0 con 0x7f671c108810 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.202+0000 7f670b7fe700 1 -- 172.21.15.76:0/3048681226 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f670c019c60 con 0x7f671c108810 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.202+0000 7f670b7fe700 1 -- 172.21.15.76:0/3048681226 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f670c0249b0 con 0x7f671c108810 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.203+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc005360 con 0x7f671c108810 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.205+0000 7f670b7fe700 1 -- 172.21.15.76:0/3048681226 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f670c003b80 con 0x7f671c108810 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.277+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f66fc0058e0 con 0x7f671c108810 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.277+0000 7f670b7fe700 1 -- 172.21.15.76:0/3048681226 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f670c01a9f0 con 0x7f671c108810 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.278+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 msgr2=0x7f671c19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.278+0000 7f6720b93700 1 --2- 172.21.15.76:0/3048681226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c19b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f670c009fb0 tx=0x7f670c011b20 comp rx=0 tx=0).stop 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.278+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 shutdown_connections 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.278+0000 7f6720b93700 1 --2- 172.21.15.76:0/3048681226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f671c108810 0x7f671c19b980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.278+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 >> 172.21.15.76:0/3048681226 conn(0x7f671c102160 msgr2=0x7f671c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.278+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 shutdown_connections 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:37.278+0000 7f6720b93700 1 -- 172.21.15.76:0/3048681226 wait complete. 2024-03-23T15:33:37.914 INFO:teuthology.orchestra.run.smithi076.stdout:mgr not available, waiting (2/15)... 2024-03-23T15:33:39.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:38 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:38.585+0000 7fb7ae9eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:33:39.476 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:39.153+0000 7fb7ae9eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:33:39.476 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:39.251+0000 7fb7ae9eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:33:39.476 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:39.473+0000 7fb7ae9eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:33:39.768 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: Activating manager daemon x 2024-03-23T15:33:39.768 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: mgrmap e2: x(active, starting, since 0.00291581s) 2024-03-23T15:33:39.768 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T15:33:39.768 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T15:33:39.768 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T15:33:39.768 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:33:39.769 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T15:33:39.769 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: Manager daemon x is now available 2024-03-23T15:33:39.769 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T15:33:39.769 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T15:33:39.769 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' 2024-03-23T15:33:39.769 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' 2024-03-23T15:33:39.769 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:39 smithi076 ceph-mon[32176]: from='mgr.14100 172.21.15.76:0/2265594580' entity='mgr.x' 2024-03-23T15:33:41.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 0 2024-03-23T15:33:41.443 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "a" 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T15:33:41.444 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T15:33:41.445 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ], 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T15:33:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T15:33:18.792864+0000", 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout }, 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.579+0000 7f4985d98700 1 Processor -- start 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f4985d98700 1 -- start start 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f4985d98700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f49801080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:41.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f4985d98700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f49801086a0 con 0x7f4980107cb0 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f497f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f49801080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f497f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f49801080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46832/0 (socket says 172.21.15.76:46832) 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f497f7fe700 1 -- 172.21.15.76:0/3536051329 learned_addr learned my addr 172.21.15.76:0/3536051329 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f497f7fe700 1 -- 172.21.15.76:0/3536051329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4980108eb0 con 0x7f4980107cb0 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.580+0000 7f497f7fe700 1 --2- 172.21.15.76:0/3536051329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f49801080d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49700098a0 tx=0x7f4970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340ceea329db1cce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f497effd700 1 -- 172.21.15.76:0/3536051329 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970004160 con 0x7f4980107cb0 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f497effd700 1 -- 172.21.15.76:0/3536051329 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49700042c0 con 0x7f4980107cb0 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f497effd700 1 -- 172.21.15.76:0/3536051329 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970011ac0 con 0x7f4980107cb0 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f4985d98700 1 -- 172.21.15.76:0/3536051329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 msgr2=0x7f49801080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f4985d98700 1 --2- 172.21.15.76:0/3536051329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f49801080d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49700098a0 tx=0x7f4970009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f4985d98700 1 -- 172.21.15.76:0/3536051329 shutdown_connections 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f4985d98700 1 --2- 172.21.15.76:0/3536051329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f49801080d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:41.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f4985d98700 1 -- 172.21.15.76:0/3536051329 >> 172.21.15.76:0/3536051329 conn(0x7f4980103250 msgr2=0x7f4980105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f4985d98700 1 -- 172.21.15.76:0/3536051329 shutdown_connections 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.581+0000 7f4985d98700 1 -- 172.21.15.76:0/3536051329 wait complete. 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f4985d98700 1 Processor -- start 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f4985d98700 1 -- start start 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f4985d98700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f498019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f4985d98700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f498019bf40 con 0x7f4980107cb0 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f497f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f498019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f497f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f498019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46840/0 (socket says 172.21.15.76:46840) 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f497f7fe700 1 -- 172.21.15.76:0/3082845699 learned_addr learned my addr 172.21.15.76:0/3082845699 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.582+0000 7f497f7fe700 1 -- 172.21.15.76:0/3082845699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4970009550 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.583+0000 7f497f7fe700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f498019ba00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4970011780 tx=0x7f4970011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.583+0000 7f497d7fa700 1 -- 172.21.15.76:0/3082845699 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970003af0 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.583+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498019c140 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.583+0000 7f497d7fa700 1 -- 172.21.15.76:0/3082845699 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4970003c50 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.583+0000 7f497d7fa700 1 -- 172.21.15.76:0/3082845699 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49700197e0 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.583+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498019c5e0 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.584+0000 7f497d7fa700 1 -- 172.21.15.76:0/3082845699 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 3) ==== 45811+0+0 (secure 0 0 0) 0x7f4970019a20 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.584+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498010c0e0 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.584+0000 7f497d7fa700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f496c038e30 0x7f496c03b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.584+0000 7f497d7fa700 1 -- 172.21.15.76:0/3082845699 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f497004d140 con 0x7f4980107cb0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.584+0000 7f4977fff700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f496c038e30 0x7f496c03b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.584+0000 7f4977fff700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f496c038e30 0x7f496c03b2f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4968006fd0 tx=0x7f4968006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.589+0000 7f497d7fa700 1 -- 172.21.15.76:0/3082845699 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4970018710 con 0x7f4980107cb0 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.863+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f498019f3b0 con 0x7f4980107cb0 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.863+0000 7f497d7fa700 1 -- 172.21.15.76:0/3082845699 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4970018350 con 0x7f4980107cb0 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f496c038e30 msgr2=0x7f496c03b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f496c038e30 0x7f496c03b2f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4968006fd0 tx=0x7f4968006e40 comp rx=0 tx=0).stop 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 msgr2=0x7f498019ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f498019ba00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4970011780 tx=0x7f4970011b40 comp rx=0 tx=0).stop 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 shutdown_connections 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f496c038e30 0x7f496c03b2f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 --2- 172.21.15.76:0/3082845699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4980107cb0 0x7f498019ba00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 >> 172.21.15.76:0/3082845699 conn(0x7f4980103250 msgr2=0x7f4980103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 shutdown_connections 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:40.865+0000 7f4985d98700 1 -- 172.21.15.76:0/3082845699 wait complete. 2024-03-23T15:33:41.451 INFO:teuthology.orchestra.run.smithi076.stdout:mgr is available 2024-03-23T15:33:41.560 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:41 smithi076 ceph-mon[32176]: mgrmap e3: x(active, since 1.00593s) 2024-03-23T15:33:41.560 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:41 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3082845699' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T15:33:42.602 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout fsid = 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T15:33:42.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f68517eb700 1 Processor -- start 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f68517eb700 1 -- start start 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f68517eb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c1080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f68517eb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f684c108690 con 0x7f684c107ca0 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f684affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c1080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f684affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c1080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46852/0 (socket says 172.21.15.76:46852) 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f684affd700 1 -- 172.21.15.76:0/2467991819 learned_addr learned my addr 172.21.15.76:0/2467991819 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.818+0000 7f684affd700 1 -- 172.21.15.76:0/2467991819 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c108ea0 con 0x7f684c107ca0 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.819+0000 7f684affd700 1 --2- 172.21.15.76:0/2467991819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c1080c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f683c0098a0 tx=0x7f683c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c918bc4fc9867c3e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.819+0000 7f6849ffb700 1 -- 172.21.15.76:0/2467991819 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f683c004160 con 0x7f684c107ca0 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.819+0000 7f6849ffb700 1 -- 172.21.15.76:0/2467991819 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f683c0042c0 con 0x7f684c107ca0 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.819+0000 7f6849ffb700 1 -- 172.21.15.76:0/2467991819 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f683c011ac0 con 0x7f684c107ca0 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 -- 172.21.15.76:0/2467991819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 msgr2=0x7f684c1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 --2- 172.21.15.76:0/2467991819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c1080c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f683c0098a0 tx=0x7f683c009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 -- 172.21.15.76:0/2467991819 shutdown_connections 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 --2- 172.21.15.76:0/2467991819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c1080c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 -- 172.21.15.76:0/2467991819 >> 172.21.15.76:0/2467991819 conn(0x7f684c103280 msgr2=0x7f684c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 -- 172.21.15.76:0/2467991819 shutdown_connections 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 -- 172.21.15.76:0/2467991819 wait complete. 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 Processor -- start 2024-03-23T15:33:42.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.820+0000 7f68517eb700 1 -- start start 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f68517eb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f68517eb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f684c19bf40 con 0x7f684c107ca0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f684affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f684affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46860/0 (socket says 172.21.15.76:46860) 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f684affd700 1 -- 172.21.15.76:0/603365326 learned_addr learned my addr 172.21.15.76:0/603365326 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f684affd700 1 -- 172.21.15.76:0/603365326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c009550 con 0x7f684c107ca0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f684affd700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c19ba00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f683c011780 tx=0x7f683c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f6833fff700 1 -- 172.21.15.76:0/603365326 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f683c003af0 con 0x7f684c107ca0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c19c140 con 0x7f684c107ca0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f6833fff700 1 -- 172.21.15.76:0/603365326 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f683c003c50 con 0x7f684c107ca0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f6833fff700 1 -- 172.21.15.76:0/603365326 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f683c0197e0 con 0x7f684c107ca0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.821+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c19c5e0 con 0x7f684c107ca0 2024-03-23T15:33:42.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.822+0000 7f6833fff700 1 -- 172.21.15.76:0/603365326 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 3) ==== 45811+0+0 (secure 0 0 0) 0x7f683c019a20 con 0x7f684c107ca0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.822+0000 7f6833fff700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6834038e30 0x7f683403b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.822+0000 7f6833fff700 1 -- 172.21.15.76:0/603365326 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f683c04cfd0 con 0x7f684c107ca0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.822+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f684c10c0d0 con 0x7f684c107ca0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.822+0000 7f684a7fc700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6834038e30 0x7f683403b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.823+0000 7f684a7fc700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6834038e30 0x7f683403b2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6840006fd0 tx=0x7f6840006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:41.827+0000 7f6833fff700 1 -- 172.21.15.76:0/603365326 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f683c018c20 con 0x7f684c107ca0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.019+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f684c19efd0 con 0x7f684c107ca0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.020+0000 7f6833fff700 1 -- 172.21.15.76:0/603365326 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f683c018350 con 0x7f684c107ca0 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6834038e30 msgr2=0x7f683403b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6834038e30 0x7f683403b2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6840006fd0 tx=0x7f6840006e40 comp rx=0 tx=0).stop 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 msgr2=0x7f684c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c19ba00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f683c011780 tx=0x7f683c011b40 comp rx=0 tx=0).stop 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 shutdown_connections 2024-03-23T15:33:42.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6834038e30 0x7f683403b2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:42.608 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 --2- 172.21.15.76:0/603365326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f684c107ca0 0x7f684c19ba00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:42.608 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.022+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 >> 172.21.15.76:0/603365326 conn(0x7f684c103280 msgr2=0x7f684c103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:42.608 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.023+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 shutdown_connections 2024-03-23T15:33:42.608 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:42.023+0000 7f68517eb700 1 -- 172.21.15.76:0/603365326 wait complete. 2024-03-23T15:33:42.608 INFO:teuthology.orchestra.run.smithi076.stdout:Enabling cephadm module... 2024-03-23T15:33:42.719 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:42 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/603365326' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T15:33:43.512 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:43 smithi076 ceph-mon[32176]: mgrmap e4: x(active, since 3s) 2024-03-23T15:33:43.512 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:43 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3922443658' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-23T15:33:43.816 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:43 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: ignoring --setuser ceph since I am not root 2024-03-23T15:33:43.816 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:43 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: ignoring --setgroup ceph since I am not root 2024-03-23T15:33:44.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:44 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:44.053+0000 7f4e24baf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.000+0000 7f6bfca69700 1 Processor -- start 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.000+0000 7f6bfca69700 1 -- start start 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.000+0000 7f6bfca69700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf810ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.000+0000 7f6bfca69700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf807be40 con 0x7f6bf81087a0 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.000+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf810ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.000+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf810ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46868/0 (socket says 172.21.15.76:46868) 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.000+0000 7f6bf659c700 1 -- 172.21.15.76:0/3676678154 learned_addr learned my addr 172.21.15.76:0/3676678154 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bf659c700 1 -- 172.21.15.76:0/3676678154 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf810b0d0 con 0x7f6bf81087a0 2024-03-23T15:33:44.090 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bf659c700 1 --2- 172.21.15.76:0/3676678154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf810ab90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6be80098a0 tx=0x7f6be8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e93f317ac2b04f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bf559a700 1 -- 172.21.15.76:0/3676678154 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be8004160 con 0x7f6bf81087a0 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bf559a700 1 -- 172.21.15.76:0/3676678154 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6be80042c0 con 0x7f6bf81087a0 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bf559a700 1 -- 172.21.15.76:0/3676678154 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be8011ac0 con 0x7f6bf81087a0 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bfca69700 1 -- 172.21.15.76:0/3676678154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 msgr2=0x7f6bf810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bfca69700 1 --2- 172.21.15.76:0/3676678154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf810ab90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6be80098a0 tx=0x7f6be8009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bfca69700 1 -- 172.21.15.76:0/3676678154 shutdown_connections 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bfca69700 1 --2- 172.21.15.76:0/3676678154 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf810ab90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.001+0000 7f6bfca69700 1 -- 172.21.15.76:0/3676678154 >> 172.21.15.76:0/3676678154 conn(0x7f6bf8102170 msgr2=0x7f6bf81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.002+0000 7f6bfca69700 1 -- 172.21.15.76:0/3676678154 shutdown_connections 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.002+0000 7f6bfca69700 1 -- 172.21.15.76:0/3676678154 wait complete. 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.002+0000 7f6bfca69700 1 Processor -- start 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.002+0000 7f6bfca69700 1 -- start start 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.002+0000 7f6bfca69700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.002+0000 7f6bfca69700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf819bf10 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf819b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46874/0 (socket says 172.21.15.76:46874) 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6bf659c700 1 -- 172.21.15.76:0/3922443658 learned_addr learned my addr 172.21.15.76:0/3922443658 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6bf659c700 1 -- 172.21.15.76:0/3922443658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be8009550 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6bf659c700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf819b9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009fb0 tx=0x7f6be8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be80038b0 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf819c110 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6be8003a10 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be80195e0 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.003+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf819c5b0 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.004+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 45917+0+0 (secure 0 0 0) 0x7f6be80198e0 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.004+0000 7f6be77fe700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6be0038f00 0x7f6be003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.004+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf810bed0 con 0x7f6bf81087a0 2024-03-23T15:33:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.004+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6be804d030 con 0x7f6bf81087a0 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.004+0000 7f6bf5d9b700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6be0038f00 0x7f6be003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.005+0000 7f6bf5d9b700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6be0038f00 0x7f6be003b3c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006fd0 tx=0x7f6bec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.009+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6be8081280 con 0x7f6bf81087a0 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.255+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f6bf819f2c0 con 0x7f6bf81087a0 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.493+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f6be8019b90 con 0x7f6bf81087a0 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.495+0000 7f6be77fe700 1 -- 172.21.15.76:0/3922443658 <== mon.0 v2:172.21.15.76:3300/0 8 ==== mgrmap(e 5) ==== 45928+0+0 (secure 0 0 0) 0x7f6be802d960 con 0x7f6bf81087a0 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6be0038f00 msgr2=0x7f6be003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6be0038f00 0x7f6be003b3c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006fd0 tx=0x7f6bec006e40 comp rx=0 tx=0).stop 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 msgr2=0x7f6bf819b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf819b9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009fb0 tx=0x7f6be8011b20 comp rx=0 tx=0).stop 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 shutdown_connections 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f6be0038f00 0x7f6be003b3c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:44.093 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 --2- 172.21.15.76:0/3922443658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bf81087a0 0x7f6bf819b9d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:44.094 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.496+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 >> 172.21.15.76:0/3922443658 conn(0x7f6bf8102170 msgr2=0x7f6bf8102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:44.094 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.497+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 shutdown_connections 2024-03-23T15:33:44.094 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:43.497+0000 7f6bfca69700 1 -- 172.21.15.76:0/3922443658 wait complete. 2024-03-23T15:33:44.427 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:44 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:44.152+0000 7f4e24baf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:33:44.734 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3922443658' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-23T15:33:44.734 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:44 smithi076 ceph-mon[32176]: mgrmap e5: x(active, since 4s) 2024-03-23T15:33:45.046 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:44 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:44.802+0000 7f4e24baf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:33:45.046 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:44 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:44.900+0000 7f4e24baf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:33:45.046 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.043+0000 7f4e24baf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:33:45.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.139+0000 7f4e24baf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:33:45.326 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.224+0000 7f4e24baf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:33:45.501 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-03-23T15:33:45.501 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-23T15:33:45.501 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T15:33:45.501 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.525+0000 7f3b0ba2d700 1 Processor -- start 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.525+0000 7f3b0ba2d700 1 -- start start 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.525+0000 7f3b0ba2d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b041080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.525+0000 7f3b0ba2d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b04108670 con 0x7f3b04107c80 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.525+0000 7f3b097c9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b041080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.525+0000 7f3b097c9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b041080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55874/0 (socket says 172.21.15.76:55874) 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.525+0000 7f3b097c9700 1 -- 172.21.15.76:0/3316320714 learned_addr learned my addr 172.21.15.76:0/3316320714 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.526+0000 7f3b097c9700 1 -- 172.21.15.76:0/3316320714 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b04108e80 con 0x7f3b04107c80 2024-03-23T15:33:45.502 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.526+0000 7f3b097c9700 1 --2- 172.21.15.76:0/3316320714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b041080a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b000098a0 tx=0x7f3b00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf64ba83dec9abc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.526+0000 7f3afbfff700 1 -- 172.21.15.76:0/3316320714 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b00004160 con 0x7f3b04107c80 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.526+0000 7f3afbfff700 1 -- 172.21.15.76:0/3316320714 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b000042c0 con 0x7f3b04107c80 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.526+0000 7f3afbfff700 1 -- 172.21.15.76:0/3316320714 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b00011ac0 con 0x7f3b04107c80 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.526+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/3316320714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 msgr2=0x7f3b041080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.526+0000 7f3b0ba2d700 1 --2- 172.21.15.76:0/3316320714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b041080a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3b000098a0 tx=0x7f3b00009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.527+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/3316320714 shutdown_connections 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.527+0000 7f3b0ba2d700 1 --2- 172.21.15.76:0/3316320714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b041080a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.527+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/3316320714 >> 172.21.15.76:0/3316320714 conn(0x7f3b04103220 msgr2=0x7f3b04105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.527+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/3316320714 shutdown_connections 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.527+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/3316320714 wait complete. 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.527+0000 7f3b0ba2d700 1 Processor -- start 2024-03-23T15:33:45.503 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.527+0000 7f3b0ba2d700 1 -- start start 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b0ba2d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b0419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b0ba2d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0419beb0 con 0x7f3b04107c80 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b097c9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b0419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b097c9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b0419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55888/0 (socket says 172.21.15.76:55888) 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b097c9700 1 -- 172.21.15.76:0/1249000865 learned_addr learned my addr 172.21.15.76:0/1249000865 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b097c9700 1 -- 172.21.15.76:0/1249000865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b00009550 con 0x7f3b04107c80 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b097c9700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b0419b970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b00000c00 tx=0x7f3b00011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:45.504 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3afa7fc700 1 -- 172.21.15.76:0/1249000865 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b00003aa0 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0419c0b0 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3afa7fc700 1 -- 172.21.15.76:0/1249000865 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b00003c00 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3afa7fc700 1 -- 172.21.15.76:0/1249000865 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b000248a0 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.528+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0419c550 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.529+0000 7f3afa7fc700 1 -- 172.21.15.76:0/1249000865 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 45928+0+0 (secure 0 0 0) 0x7f3b0002d3f0 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.529+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0410c0b0 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.529+0000 7f3afa7fc700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 0x7f3af003b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.529+0000 7f3afa7fc700 1 -- 172.21.15.76:0/1249000865 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b0004cd10 con 0x7f3b04107c80 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.529+0000 7f3b08fc8700 1 -- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 msgr2=0x7f3af003b4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/3268079315 2024-03-23T15:33:45.505 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.530+0000 7f3b08fc8700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 0x7f3af003b4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.534+0000 7f3afa7fc700 1 -- 172.21.15.76:0/1249000865 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f3b00010740 con 0x7f3b04107c80 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.731+0000 7f3b08fc8700 1 -- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 msgr2=0x7f3af003b4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/3268079315 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.731+0000 7f3b08fc8700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 0x7f3af003b4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.783+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3b0419f320 con 0x7f3b04107c80 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.784+0000 7f3afa7fc700 1 -- 172.21.15.76:0/1249000865 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3b0000b3e0 con 0x7f3b04107c80 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 msgr2=0x7f3af003b4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 0x7f3af003b4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 msgr2=0x7f3b0419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b0419b970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b00000c00 tx=0x7f3b00011b40 comp rx=0 tx=0).stop 2024-03-23T15:33:45.506 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 shutdown_connections 2024-03-23T15:33:45.507 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7f3af0038ff0 0x7f3af003b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:45.507 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 --2- 172.21.15.76:0/1249000865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b04107c80 0x7f3b0419b970 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:45.507 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 >> 172.21.15.76:0/1249000865 conn(0x7f3b04103220 msgr2=0x7f3b04103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:45.507 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 shutdown_connections 2024-03-23T15:33:45.507 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:44.786+0000 7f3b0ba2d700 1 -- 172.21.15.76:0/1249000865 wait complete. 2024-03-23T15:33:45.507 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for the mgr to restart... 2024-03-23T15:33:45.507 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr epoch 5... 2024-03-23T15:33:45.740 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1249000865' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T15:33:45.741 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.490+0000 7f4e24baf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:33:45.741 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.606+0000 7f4e24baf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:33:45.741 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.694+0000 7f4e24baf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:33:46.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.791+0000 7f4e24baf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:33:46.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:45.973+0000 7f4e24baf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:33:47.320 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:46 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:46.930+0000 7f4e24baf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:33:47.320 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:47 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:47.059+0000 7f4e24baf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:33:47.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:47 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:47.389+0000 7f4e24baf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:33:47.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:47 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:47.478+0000 7f4e24baf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:33:48.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:47 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:47.675+0000 7f4e24baf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:33:48.076 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:47 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:47.859+0000 7f4e24baf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:33:48.406 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:48 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:48.144+0000 7f4e24baf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:33:48.406 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:48 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:48.402+0000 7f4e24baf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:33:48.825 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:48 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:48.638+0000 7f4e24baf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:33:49.825 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:49 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:49.440+0000 7f4e24baf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:33:50.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:50.005+0000 7f4e24baf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:33:50.326 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:50.107+0000 7f4e24baf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:33:50.616 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:33:50.326+0000 7f4e24baf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:33:50.616 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: Active manager daemon x restarted 2024-03-23T15:33:50.616 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: Activating manager daemon x 2024-03-23T15:33:50.616 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: osdmap e2: 0 total, 0 up, 0 in 2024-03-23T15:33:50.616 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: mgrmap e6: x(active, starting, since 0.00364841s) 2024-03-23T15:33:50.616 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: Manager daemon x is now available 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:33:50.617 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:50 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:33:51.695 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:51 smithi076 ceph-mon[32176]: Found migration_current of "None". Setting to last migration. 2024-03-23T15:33:51.695 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:51 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:33:51.695 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:51 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T15:33:51.695 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:51 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T15:33:51.695 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:51 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:33:51.695 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:51 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:33:51.696 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:51 smithi076 ceph-mon[32176]: mgrmap e7: x(active, since 1.00531s) 2024-03-23T15:33:51.959 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.909+0000 7fa2b243e700 1 Processor -- start 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.909+0000 7fa2b243e700 1 -- start start 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.909+0000 7fa2b243e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.909+0000 7fa2b243e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac107aa0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.960 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55904/0 (socket says 172.21.15.76:55904) 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2abfff700 1 -- 172.21.15.76:0/1319697334 learned_addr learned my addr 172.21.15.76:0/1319697334 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2abfff700 1 -- 172.21.15.76:0/1319697334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac107dd0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2abfff700 1 --2- 172.21.15.76:0/1319697334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1074d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00098a0 tx=0x7fa2a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f92d33302a2eb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2aaffd700 1 -- 172.21.15.76:0/1319697334 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a0004160 con 0x7fa2ac1070b0 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2aaffd700 1 -- 172.21.15.76:0/1319697334 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2a00042c0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.910+0000 7fa2aaffd700 1 -- 172.21.15.76:0/1319697334 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a0011ac0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.911+0000 7fa2b243e700 1 -- 172.21.15.76:0/1319697334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 msgr2=0x7fa2ac1074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.911+0000 7fa2b243e700 1 --2- 172.21.15.76:0/1319697334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1074d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00098a0 tx=0x7fa2a0009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.911+0000 7fa2b243e700 1 -- 172.21.15.76:0/1319697334 shutdown_connections 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.911+0000 7fa2b243e700 1 --2- 172.21.15.76:0/1319697334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1074d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.911+0000 7fa2b243e700 1 -- 172.21.15.76:0/1319697334 >> 172.21.15.76:0/1319697334 conn(0x7fa2ac1018b0 msgr2=0x7fa2ac103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:51.961 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.911+0000 7fa2b243e700 1 -- 172.21.15.76:0/1319697334 shutdown_connections 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.911+0000 7fa2b243e700 1 -- 172.21.15.76:0/1319697334 wait complete. 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2b243e700 1 Processor -- start 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2b243e700 1 -- start start 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2b243e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1066f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2b243e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac19e6b0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1066f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55906/0 (socket says 172.21.15.76:55906) 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2abfff700 1 -- 172.21.15.76:0/3224212747 learned_addr learned my addr 172.21.15.76:0/3224212747 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.912+0000 7fa2abfff700 1 -- 172.21.15.76:0/3224212747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a0009550 con 0x7fa2ac1070b0 2024-03-23T15:33:51.962 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.913+0000 7fa2abfff700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1066f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fb0 tx=0x7fa2a0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.913+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a00038b0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.913+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac19e7f0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.913+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2a0003a10 con 0x7fa2ac1070b0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.913+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a00195e0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.913+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac19ebd0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.914+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 45928+0+0 (secure 0 0 0) 0x7fa2a00198c0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.914+0000 7fa2a97fa700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.914+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 --> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa29403bb20 con 0x7fa294038f50 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.914+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa2a004d1e0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.914+0000 7fa2ab7fe700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 msgr2=0x7fa29403b410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/3268079315 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:45.914+0000 7fa2ab7fe700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:46.115+0000 7fa2ab7fe700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 msgr2=0x7fa29403b410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/3268079315 2024-03-23T15:33:51.963 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:46.115+0000 7fa2ab7fe700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:46.516+0000 7fa2ab7fe700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 msgr2=0x7fa29403b410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/3268079315 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:46.516+0000 7fa2ab7fe700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:47.317+0000 7fa2ab7fe700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 msgr2=0x7fa29403b410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/3268079315 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:47.317+0000 7fa2ab7fe700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:48.918+0000 7fa2ab7fe700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 msgr2=0x7fa29403b410 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/3268079315 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:48.918+0000 7fa2ab7fe700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:50.342+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mgrmap(e 6) ==== 45695+0+0 (secure 0 0 0) 0x7fa2a002d9b0 con 0x7fa2ac1070b0 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:50.342+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 msgr2=0x7fa29403b410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:50.342+0000 7fa2a97fa700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.343+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 7) ==== 45822+0+0 (secure 0 0 0) 0x7fa2a004ea80 con 0x7fa2ac1070b0 2024-03-23T15:33:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.343+0000 7fa2a97fa700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fa29403c840 0x7fa29403ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.343+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa29403bb20 con 0x7fa29403c840 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.344+0000 7fa2ab7fe700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fa29403c840 0x7fa29403ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.344+0000 7fa2ab7fe700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fa29403c840 0x7fa29403ec30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa29c003a10 tx=0x7fa29c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.345+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa29403bb20 con 0x7fa29403c840 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.351+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa2ac10b000 con 0x7fa29403c840 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.351+0000 7fa2a97fa700 1 -- 172.21.15.76:0/3224212747 <== mgr.14118 v2:172.21.15.76:6800/36977200 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fa2ac10b000 con 0x7fa29403c840 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fa29403c840 msgr2=0x7fa29403ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fa29403c840 0x7fa29403ec30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa29c003a10 tx=0x7fa29c009290 comp rx=0 tx=0).stop 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 msgr2=0x7fa2ac1066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1066f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fb0 tx=0x7fa2a0011b20 comp rx=0 tx=0).stop 2024-03-23T15:33:51.965 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 shutdown_connections 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2ac1070b0 0x7fa2ac1066f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fa29403c840 0x7fa29403ec30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 --2- 172.21.15.76:0/3224212747 >> [v2:172.21.15.76:6800/3268079315,v1:172.21.15.76:6801/3268079315] conn(0x7fa294038f50 0x7fa29403b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 >> 172.21.15.76:0/3224212747 conn(0x7fa2ac1018b0 msgr2=0x7fa2ac102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 shutdown_connections 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:51.352+0000 7fa2b243e700 1 -- 172.21.15.76:0/3224212747 wait complete. 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:mgr epoch 5 is available 2024-03-23T15:33:51.966 INFO:teuthology.orchestra.run.smithi076.stdout:Setting orchestrator backend to cephadm... 2024-03-23T15:33:52.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:52 smithi076 ceph-mon[32176]: [23/Mar/2024:15:33:51] ENGINE Bus STARTING 2024-03-23T15:33:52.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:52 smithi076 ceph-mon[32176]: [23/Mar/2024:15:33:51] ENGINE Serving on http://172.21.15.76:8765 2024-03-23T15:33:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:52 smithi076 ceph-mon[32176]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T15:33:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:52 smithi076 ceph-mon[32176]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T15:33:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:52 smithi076 ceph-mon[32176]: [23/Mar/2024:15:33:51] ENGINE Serving on https://172.21.15.76:7150 2024-03-23T15:33:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:52 smithi076 ceph-mon[32176]: [23/Mar/2024:15:33:51] ENGINE Bus STARTED 2024-03-23T15:33:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:52 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:33:53.068 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.360+0000 7f08c75a1700 1 Processor -- start 2024-03-23T15:33:53.068 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.360+0000 7f08c75a1700 1 -- start start 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.360+0000 7f08c75a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c01080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.360+0000 7f08c75a1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01086b0 con 0x7f08c0107cc0 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08c533d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c01080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08c533d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c01080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55948/0 (socket says 172.21.15.76:55948) 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08c533d700 1 -- 172.21.15.76:0/488901210 learned_addr learned my addr 172.21.15.76:0/488901210 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08c533d700 1 -- 172.21.15.76:0/488901210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c0108ec0 con 0x7f08c0107cc0 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08c533d700 1 --2- 172.21.15.76:0/488901210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c01080e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0098a0 tx=0x7f08bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1590705b2568ac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08b7fff700 1 -- 172.21.15.76:0/488901210 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08bc004160 con 0x7f08c0107cc0 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08b7fff700 1 -- 172.21.15.76:0/488901210 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08bc0042c0 con 0x7f08c0107cc0 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.361+0000 7f08b7fff700 1 -- 172.21.15.76:0/488901210 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08bc011ac0 con 0x7f08c0107cc0 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 -- 172.21.15.76:0/488901210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 msgr2=0x7f08c01080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 --2- 172.21.15.76:0/488901210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c01080e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0098a0 tx=0x7f08bc009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:53.069 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 -- 172.21.15.76:0/488901210 shutdown_connections 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 --2- 172.21.15.76:0/488901210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c01080e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 -- 172.21.15.76:0/488901210 >> 172.21.15.76:0/488901210 conn(0x7f08c0103280 msgr2=0x7f08c01056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 -- 172.21.15.76:0/488901210 shutdown_connections 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 -- 172.21.15.76:0/488901210 wait complete. 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 Processor -- start 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.362+0000 7f08c75a1700 1 -- start start 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c75a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c75a1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08c019bf20 con 0x7f08c0107cc0 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c533d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c533d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c019b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55962/0 (socket says 172.21.15.76:55962) 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c533d700 1 -- 172.21.15.76:0/1176791587 learned_addr learned my addr 172.21.15.76:0/1176791587 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c533d700 1 -- 172.21.15.76:0/1176791587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08bc009550 con 0x7f08c0107cc0 2024-03-23T15:33:53.070 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c533d700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c019b9e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009fb0 tx=0x7f08bc011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08bc0038b0 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08bc003a10 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c019c120 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08bc0195e0 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.363+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c019c5c0 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.364+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 7) ==== 45822+0+0 (secure 0 0 0) 0x7f08bc01aa10 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.364+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c010c0f0 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.364+0000 7f08b67fc700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f08ac038e90 0x7f08ac03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.364+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f08bc04ceb0 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.365+0000 7f08c4b3c700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f08ac038e90 0x7f08ac03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.365+0000 7f08c4b3c700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f08ac038e90 0x7f08ac03b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006fd0 tx=0x7f08b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.370+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f08bc003b80 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.396+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f08bc023390 con 0x7f08c0107cc0 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.580+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f08c01043e0 con 0x7f08ac038e90 2024-03-23T15:33:53.071 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.588+0000 7f08b67fc700 1 -- 172.21.15.76:0/1176791587 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f08c01043e0 con 0x7f08ac038e90 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f08ac038e90 msgr2=0x7f08ac03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f08ac038e90 0x7f08ac03b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08b0006fd0 tx=0x7f08b0006e40 comp rx=0 tx=0).stop 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 msgr2=0x7f08c019b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c019b9e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009fb0 tx=0x7f08bc011b20 comp rx=0 tx=0).stop 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 shutdown_connections 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f08ac038e90 0x7f08ac03b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 --2- 172.21.15.76:0/1176791587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c0107cc0 0x7f08c019b9e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 >> 172.21.15.76:0/1176791587 conn(0x7f08c0103280 msgr2=0x7f08c0103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 shutdown_connections 2024-03-23T15:33:53.072 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:52.591+0000 7f08c75a1700 1 -- 172.21.15.76:0/1176791587 wait complete. 2024-03-23T15:33:53.479 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:53 smithi076 ceph-mon[32176]: mgrmap e8: x(active, since 2s) 2024-03-23T15:33:53.479 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:53 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:33:53.479 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:53 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:33:54.224 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-23T15:33:54.224 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.477+0000 7fc302b3e700 1 Processor -- start 2024-03-23T15:33:54.224 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.477+0000 7fc302b3e700 1 -- start start 2024-03-23T15:33:54.224 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.477+0000 7fc302b3e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc10ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:54.224 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.477+0000 7fc302b3e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc07be60 con 0x7fc2fc108770 2024-03-23T15:33:54.224 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.477+0000 7fc3008da700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc10ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:54.224 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.478+0000 7fc3008da700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc10ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55976/0 (socket says 172.21.15.76:55976) 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.478+0000 7fc3008da700 1 -- 172.21.15.76:0/2588321316 learned_addr learned my addr 172.21.15.76:0/2588321316 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.478+0000 7fc3008da700 1 -- 172.21.15.76:0/2588321316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc10b0a0 con 0x7fc2fc108770 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.478+0000 7fc3008da700 1 --2- 172.21.15.76:0/2588321316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc10ab60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098a0 tx=0x7fc2f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c444b19162bbe0b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.478+0000 7fc2fb7fe700 1 -- 172.21.15.76:0/2588321316 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f0004160 con 0x7fc2fc108770 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.478+0000 7fc2fb7fe700 1 -- 172.21.15.76:0/2588321316 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc2f00042c0 con 0x7fc2fc108770 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.478+0000 7fc2fb7fe700 1 -- 172.21.15.76:0/2588321316 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f0011ac0 con 0x7fc2fc108770 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.479+0000 7fc302b3e700 1 -- 172.21.15.76:0/2588321316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 msgr2=0x7fc2fc10ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.479+0000 7fc302b3e700 1 --2- 172.21.15.76:0/2588321316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc10ab60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098a0 tx=0x7fc2f0009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.479+0000 7fc302b3e700 1 -- 172.21.15.76:0/2588321316 shutdown_connections 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.479+0000 7fc302b3e700 1 --2- 172.21.15.76:0/2588321316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc10ab60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.479+0000 7fc302b3e700 1 -- 172.21.15.76:0/2588321316 >> 172.21.15.76:0/2588321316 conn(0x7fc2fc102160 msgr2=0x7fc2fc1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:54.225 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.479+0000 7fc302b3e700 1 -- 172.21.15.76:0/2588321316 shutdown_connections 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.479+0000 7fc302b3e700 1 -- 172.21.15.76:0/2588321316 wait complete. 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc302b3e700 1 Processor -- start 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc302b3e700 1 -- start start 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc302b3e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc302b3e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc19bf00 con 0x7fc2fc108770 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc3008da700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc3008da700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55988/0 (socket says 172.21.15.76:55988) 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc3008da700 1 -- 172.21.15.76:0/3454858528 learned_addr learned my addr 172.21.15.76:0/3454858528 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc3008da700 1 -- 172.21.15.76:0/3454858528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f0009550 con 0x7fc2fc108770 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc3008da700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc19b9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009fb0 tx=0x7fc2f0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:54.226 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc2f9ffb700 1 -- 172.21.15.76:0/3454858528 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f00038b0 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc19c100 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc2f9ffb700 1 -- 172.21.15.76:0/3454858528 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc2f0003a10 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.480+0000 7fc2f9ffb700 1 -- 172.21.15.76:0/3454858528 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f00195e0 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.481+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc19c5a0 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.481+0000 7fc2f9ffb700 1 -- 172.21.15.76:0/3454858528 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7fc2f0019870 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.481+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc10bea0 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.482+0000 7fc2f9ffb700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fc2ec038fa0 0x7fc2ec03b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.482+0000 7fc2f9ffb700 1 -- 172.21.15.76:0/3454858528 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc2f004d140 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.482+0000 7fc2fbfff700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fc2ec038fa0 0x7fc2ec03b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.482+0000 7fc2fbfff700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fc2ec038fa0 0x7fc2ec03b460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8006fd0 tx=0x7fc2e8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.487+0000 7fc2f9ffb700 1 -- 172.21.15.76:0/3454858528 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc2f0018dc0 con 0x7fc2fc108770 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.683+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fc2fc1032c0 con 0x7fc2ec038fa0 2024-03-23T15:33:54.227 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.684+0000 7fc2f9ffb700 1 -- 172.21.15.76:0/3454858528 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fc2fc1032c0 con 0x7fc2ec038fa0 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fc2ec038fa0 msgr2=0x7fc2ec03b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fc2ec038fa0 0x7fc2ec03b460 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8006fd0 tx=0x7fc2e8006e40 comp rx=0 tx=0).stop 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 msgr2=0x7fc2fc19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc19b9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009fb0 tx=0x7fc2f0011b20 comp rx=0 tx=0).stop 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 shutdown_connections 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7fc2ec038fa0 0x7fc2ec03b460 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 --2- 172.21.15.76:0/3454858528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc2fc108770 0x7fc2fc19b9c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 >> 172.21.15.76:0/3454858528 conn(0x7fc2fc102160 msgr2=0x7fc2fc102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 shutdown_connections 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:53.686+0000 7fc302b3e700 1 -- 172.21.15.76:0/3454858528 wait complete. 2024-03-23T15:33:54.228 INFO:teuthology.orchestra.run.smithi076.stdout:Generating ssh key... 2024-03-23T15:33:54.660 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-mon[32176]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: Generating public/private rsa key pair. 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: Your identification has been saved in /tmp/tmpu81h3akb/key. 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: Your public key has been saved in /tmp/tmpu81h3akb/key.pub. 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: The key fingerprint is: 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: SHA256:H0tZT0maVajks26rA0WE5Gn+lL9nP48hy4GoK6+pVM8 ceph-89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: The key's randomart image is: 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: +---[RSA 3072]----+ 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | ..o. oo.| 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | .... .=.. | 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | +. o+.o | 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | o .++o | 2024-03-23T15:33:55.289 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | . S.* o. | 2024-03-23T15:33:55.290 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | . o .* =. | 2024-03-23T15:33:55.290 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | . E ..=.+ . | 2024-03-23T15:33:55.290 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | . .. . ..o=o+ | 2024-03-23T15:33:55.290 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: | ..o++. .+=+..=| 2024-03-23T15:33:55.290 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:33:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: +----[SHA256]-----+ 2024-03-23T15:33:55.573 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:55 smithi076 ceph-mon[32176]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:33:55.573 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:55 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:33:55.573 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:55 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:33:55.581 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.688+0000 7faa1bb57700 1 Processor -- start 2024-03-23T15:33:55.581 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.688+0000 7faa1bb57700 1 -- start start 2024-03-23T15:33:55.581 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.688+0000 7faa1bb57700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa141080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:55.581 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.688+0000 7faa1bb57700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa14108690 con 0x7faa14107ca0 2024-03-23T15:33:55.581 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.688+0000 7faa198f3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa141080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.688+0000 7faa198f3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa141080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54740/0 (socket says 172.21.15.76:54740) 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.688+0000 7faa198f3700 1 -- 172.21.15.76:0/824467323 learned_addr learned my addr 172.21.15.76:0/824467323 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa198f3700 1 -- 172.21.15.76:0/824467323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa14108ea0 con 0x7faa14107ca0 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa198f3700 1 --2- 172.21.15.76:0/824467323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa141080c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa100098a0 tx=0x7faa10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914c36d500193927 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa188f1700 1 -- 172.21.15.76:0/824467323 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa10004160 con 0x7faa14107ca0 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa188f1700 1 -- 172.21.15.76:0/824467323 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa100042c0 con 0x7faa14107ca0 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa188f1700 1 -- 172.21.15.76:0/824467323 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa10011ac0 con 0x7faa14107ca0 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa1bb57700 1 -- 172.21.15.76:0/824467323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 msgr2=0x7faa141080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa1bb57700 1 --2- 172.21.15.76:0/824467323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa141080c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa100098a0 tx=0x7faa10009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa1bb57700 1 -- 172.21.15.76:0/824467323 shutdown_connections 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa1bb57700 1 --2- 172.21.15.76:0/824467323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa141080c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.689+0000 7faa1bb57700 1 -- 172.21.15.76:0/824467323 >> 172.21.15.76:0/824467323 conn(0x7faa14103280 msgr2=0x7faa141056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:55.582 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa1bb57700 1 -- 172.21.15.76:0/824467323 shutdown_connections 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa1bb57700 1 -- 172.21.15.76:0/824467323 wait complete. 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa1bb57700 1 Processor -- start 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa1bb57700 1 -- start start 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa1bb57700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa1419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa1bb57700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa1419bf40 con 0x7faa14107ca0 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa198f3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa1419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa198f3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa1419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54746/0 (socket says 172.21.15.76:54746) 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.690+0000 7faa198f3700 1 -- 172.21.15.76:0/1409215325 learned_addr learned my addr 172.21.15.76:0/1409215325 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.691+0000 7faa198f3700 1 -- 172.21.15.76:0/1409215325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa10009550 con 0x7faa14107ca0 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.691+0000 7faa198f3700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa1419ba00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa10009fb0 tx=0x7faa10011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.691+0000 7faa0affd700 1 -- 172.21.15.76:0/1409215325 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa100038b0 con 0x7faa14107ca0 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.691+0000 7faa0affd700 1 -- 172.21.15.76:0/1409215325 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa10003a10 con 0x7faa14107ca0 2024-03-23T15:33:55.583 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.691+0000 7faa0affd700 1 -- 172.21.15.76:0/1409215325 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa100195e0 con 0x7faa14107ca0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.691+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1419c140 con 0x7faa14107ca0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.691+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1419c5e0 con 0x7faa14107ca0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.692+0000 7faa0affd700 1 -- 172.21.15.76:0/1409215325 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7faa100198a0 con 0x7faa14107ca0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.692+0000 7faa0affd700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7faa00038f50 0x7faa0003b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.692+0000 7faa0affd700 1 -- 172.21.15.76:0/1409215325 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa1004d110 con 0x7faa14107ca0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.692+0000 7faa190f2700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7faa00038f50 0x7faa0003b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.692+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1410c0d0 con 0x7faa14107ca0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.693+0000 7faa190f2700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7faa00038f50 0x7faa0003b410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa04006fd0 tx=0x7faa04006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.697+0000 7faa0affd700 1 -- 172.21.15.76:0/1409215325 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7faa10019ba0 con 0x7faa14107ca0 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.892+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7faa141043b0 con 0x7faa00038f50 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.945+0000 7faa0affd700 1 -- 172.21.15.76:0/1409215325 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa141043b0 con 0x7faa00038f50 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7faa00038f50 msgr2=0x7faa0003b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7faa00038f50 0x7faa0003b410 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa04006fd0 tx=0x7faa04006e40 comp rx=0 tx=0).stop 2024-03-23T15:33:55.584 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 msgr2=0x7faa1419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:55.585 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa1419ba00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa10009fb0 tx=0x7faa10011b20 comp rx=0 tx=0).stop 2024-03-23T15:33:55.585 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 shutdown_connections 2024-03-23T15:33:55.585 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7faa00038f50 0x7faa0003b410 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:55.585 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 --2- 172.21.15.76:0/1409215325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa14107ca0 0x7faa1419ba00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:55.585 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 >> 172.21.15.76:0/1409215325 conn(0x7faa14103280 msgr2=0x7faa14103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:55.585 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 shutdown_connections 2024-03-23T15:33:55.585 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:54.947+0000 7faa1bb57700 1 -- 172.21.15.76:0/1409215325 wait complete. 2024-03-23T15:33:56.486 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:56 smithi076 ceph-mon[32176]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:33:56.487 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:56 smithi076 ceph-mon[32176]: Generating ssh key... 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdXRLCPnE171D1GmqOmiQBxQfkiuDKsm0oSu8Ukc0fucijTpu8uT8PNIv0N/sAr6xnv9NKgOgFqh1Not6t192+DRJRuqvn92SKVBxE9+U+NApVZmBHaXndI5XLPVupS5vWVZsI18pnhdmmLvc1+Lt19r2qdJ61JNOh+xVL5niCiYJv5mvw1bgpdcfLXAWvHNwUUS03Th5orViWGw62nCW4RHBHV05KLbd1mOfgUn3Yra8kIy3Mcl5DaXV0ChdYPBor0U+QLSNEOK7/BNPa9tVH8w9tCdxls6QKqsyfB5AC58gjXAxhEr7WZiF6PxRrsaHimd6q8vmWX4THoPOk+iLc19D5t6mUfCRg5BD9zNKc+2oxsstwMLRewueaG0jThxJn3G2x/fm0h303Mjrirgw7xxCaRsMJHYzlBQpZ934uc7edhOnnjEsDvmY5jK0lbBGv61QuDfBb12MjJoW04B6U2qQ4ws6XJxTlAK67T6pSxiOuuxOTkmmhTK5MDbloBX0= ceph-89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.984+0000 7f0368605700 1 Processor -- start 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.984+0000 7f0368605700 1 -- start start 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.984+0000 7f0368605700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f0360106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f0368605700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03601075b0 con 0x7f0360106bc0 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f03663a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f0360106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f03663a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f0360106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54760/0 (socket says 172.21.15.76:54760) 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f03663a1700 1 -- 172.21.15.76:0/351132555 learned_addr learned my addr 172.21.15.76:0/351132555 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:56.754 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f03663a1700 1 -- 172.21.15.76:0/351132555 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0360107dc0 con 0x7f0360106bc0 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f03663a1700 1 --2- 172.21.15.76:0/351132555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f0360106fe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03500098a0 tx=0x7f0350009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42fd40aef57a1c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f036539f700 1 -- 172.21.15.76:0/351132555 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0350004160 con 0x7f0360106bc0 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f036539f700 1 -- 172.21.15.76:0/351132555 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03500042c0 con 0x7f0360106bc0 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.985+0000 7f036539f700 1 -- 172.21.15.76:0/351132555 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0350011ac0 con 0x7f0360106bc0 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 -- 172.21.15.76:0/351132555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 msgr2=0x7f0360106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 --2- 172.21.15.76:0/351132555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f0360106fe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03500098a0 tx=0x7f0350009bb0 comp rx=0 tx=0).stop 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 -- 172.21.15.76:0/351132555 shutdown_connections 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 --2- 172.21.15.76:0/351132555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f0360106fe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 -- 172.21.15.76:0/351132555 >> 172.21.15.76:0/351132555 conn(0x7f0360102160 msgr2=0x7f03601045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 -- 172.21.15.76:0/351132555 shutdown_connections 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 -- 172.21.15.76:0/351132555 wait complete. 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 Processor -- start 2024-03-23T15:33:56.755 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.986+0000 7f0368605700 1 -- start start 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f0368605700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f036019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f0368605700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f036019ae30 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03663a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f036019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03663a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f036019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54766/0 (socket says 172.21.15.76:54766) 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03663a1700 1 -- 172.21.15.76:0/1713449306 learned_addr learned my addr 172.21.15.76:0/1713449306 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03663a1700 1 -- 172.21.15.76:0/1713449306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0350009550 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03663a1700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f036019a8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0350009fb0 tx=0x7f0350011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03577fe700 1 -- 172.21.15.76:0/1713449306 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03500038b0 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03577fe700 1 -- 172.21.15.76:0/1713449306 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0350003a10 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f03577fe700 1 -- 172.21.15.76:0/1713449306 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03500195e0 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036019b030 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.987+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036019b4d0 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.988+0000 7f03577fe700 1 -- 172.21.15.76:0/1713449306 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f035001a430 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.988+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036010aff0 con 0x7f0360106bc0 2024-03-23T15:33:56.756 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.988+0000 7f03577fe700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f034c038fa0 0x7f034c03b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.988+0000 7f03577fe700 1 -- 172.21.15.76:0/1713449306 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f035004d0d0 con 0x7f0360106bc0 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.989+0000 7f0365ba0700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f034c038fa0 0x7f034c03b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.989+0000 7f0365ba0700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f034c038fa0 0x7f034c03b460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f035c006fd0 tx=0x7f035c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:55.993+0000 7f03577fe700 1 -- 172.21.15.76:0/1713449306 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0350003b80 con 0x7f0360106bc0 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.190+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f0360103270 con 0x7f034c038fa0 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.190+0000 7f03577fe700 1 -- 172.21.15.76:0/1713449306 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f0360103270 con 0x7f034c038fa0 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f034c038fa0 msgr2=0x7f034c03b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f034c038fa0 0x7f034c03b460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f035c006fd0 tx=0x7f035c006e40 comp rx=0 tx=0).stop 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 msgr2=0x7f036019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f036019a8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0350009fb0 tx=0x7f0350011b20 comp rx=0 tx=0).stop 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 shutdown_connections 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f034c038fa0 0x7f034c03b460 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 --2- 172.21.15.76:0/1713449306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0360106bc0 0x7f036019a8f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:33:56.757 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.192+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 >> 172.21.15.76:0/1713449306 conn(0x7f0360102160 msgr2=0x7f0360102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:33:56.758 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.193+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 shutdown_connections 2024-03-23T15:33:56.758 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:56.193+0000 7f0368605700 1 -- 172.21.15.76:0/1713449306 wait complete. 2024-03-23T15:33:56.758 INFO:teuthology.orchestra.run.smithi076.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-23T15:33:56.758 INFO:teuthology.orchestra.run.smithi076.stdout:Adding key to root@localhost authorized_keys... 2024-03-23T15:33:56.758 INFO:teuthology.orchestra.run.smithi076.stdout:Adding host smithi076... 2024-03-23T15:33:57.687 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:57 smithi076 ceph-mon[32176]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:33:58.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:58 smithi076 ceph-mon[32176]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "addr": "172.21.15.76", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:33:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:33:59 smithi076 ceph-mon[32176]: Deploying cephadm binary to smithi076 2024-03-23T15:34:02.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Added host 'smithi076' with addr '172.21.15.76' 2024-03-23T15:34:02.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.145+0000 7f9dc5e16700 1 Processor -- start 2024-03-23T15:34:02.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.145+0000 7f9dc5e16700 1 -- start start 2024-03-23T15:34:02.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dc5e16700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:02.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dc5e16700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc007c8e0 con 0x7f9dc007deb0 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbf7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbf7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54778/0 (socket says 172.21.15.76:54778) 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbf7fe700 1 -- 172.21.15.76:0/1126629643 learned_addr learned my addr 172.21.15.76:0/1126629643 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbf7fe700 1 -- 172.21.15.76:0/1126629643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc007ca20 con 0x7f9dc007deb0 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbf7fe700 1 --2- 172.21.15.76:0/1126629643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc007c310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9db40098a0 tx=0x7f9db4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b98281576a5ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbe7fc700 1 -- 172.21.15.76:0/1126629643 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db4004160 con 0x7f9dc007deb0 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbe7fc700 1 -- 172.21.15.76:0/1126629643 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9db40042c0 con 0x7f9dc007deb0 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.146+0000 7f9dbe7fc700 1 -- 172.21.15.76:0/1126629643 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db4011ac0 con 0x7f9dc007deb0 2024-03-23T15:34:02.036 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.147+0000 7f9dc5e16700 1 -- 172.21.15.76:0/1126629643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 msgr2=0x7f9dc007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.147+0000 7f9dc5e16700 1 --2- 172.21.15.76:0/1126629643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc007c310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9db40098a0 tx=0x7f9db4009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.147+0000 7f9dc5e16700 1 -- 172.21.15.76:0/1126629643 shutdown_connections 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.147+0000 7f9dc5e16700 1 --2- 172.21.15.76:0/1126629643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc007c310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.147+0000 7f9dc5e16700 1 -- 172.21.15.76:0/1126629643 >> 172.21.15.76:0/1126629643 conn(0x7f9dc0103280 msgr2=0x7f9dc01056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.147+0000 7f9dc5e16700 1 -- 172.21.15.76:0/1126629643 shutdown_connections 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.147+0000 7f9dc5e16700 1 -- 172.21.15.76:0/1126629643 wait complete. 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dc5e16700 1 Processor -- start 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dc5e16700 1 -- start start 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dc5e16700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dc5e16700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc019e140 con 0x7f9dc007deb0 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dbf7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:02.037 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dbf7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54792/0 (socket says 172.21.15.76:54792) 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dbf7fe700 1 -- 172.21.15.76:0/2772755790 learned_addr learned my addr 172.21.15.76:0/2772755790 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dbf7fe700 1 -- 172.21.15.76:0/2772755790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db4009550 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.148+0000 7f9dbf7fe700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc019dc00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9db4011780 tx=0x7f9db4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.149+0000 7f9dbcff9700 1 -- 172.21.15.76:0/2772755790 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db4003af0 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.149+0000 7f9dbcff9700 1 -- 172.21.15.76:0/2772755790 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9db4003c50 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.149+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc019e340 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.149+0000 7f9dbcff9700 1 -- 172.21.15.76:0/2772755790 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9db40197e0 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.149+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc019e7e0 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.149+0000 7f9dbcff9700 1 -- 172.21.15.76:0/2772755790 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f9db4019a20 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.150+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc010e0b0 con 0x7f9dc007deb0 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.150+0000 7f9dbcff9700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f9da8038fa0 0x7f9da803b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.150+0000 7f9dbcff9700 1 -- 172.21.15.76:0/2772755790 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9db404d2d0 con 0x7f9dc007deb0 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.150+0000 7f9dbeffd700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f9da8038fa0 0x7f9da803b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.150+0000 7f9dbeffd700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f9da8038fa0 0x7f9da803b460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9db0006fd0 tx=0x7f9db0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.155+0000 7f9dbcff9700 1 -- 172.21.15.76:0/2772755790 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9db4018c20 con 0x7f9dc007deb0 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:33:57.353+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi076", "addr": "172.21.15.76", "target": ["mon-mgr", ""]}) -- 0x7f9dc0104430 con 0x7f9da8038fa0 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.500+0000 7f9dbcff9700 1 -- 172.21.15.76:0/2772755790 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9dc0104430 con 0x7f9da8038fa0 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f9da8038fa0 msgr2=0x7f9da803b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f9da8038fa0 0x7f9da803b460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9db0006fd0 tx=0x7f9db0006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 msgr2=0x7f9dc019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc019dc00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9db4011780 tx=0x7f9db4011b40 comp rx=0 tx=0).stop 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 shutdown_connections 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f9da8038fa0 0x7f9da803b460 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:02.039 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 --2- 172.21.15.76:0/2772755790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9dc007deb0 0x7f9dc019dc00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:02.040 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.502+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 >> 172.21.15.76:0/2772755790 conn(0x7f9dc0103280 msgr2=0x7f9dc0103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:02.040 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.503+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 shutdown_connections 2024-03-23T15:34:02.040 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:01.503+0000 7f9dc5e16700 1 -- 172.21.15.76:0/2772755790 wait complete. 2024-03-23T15:34:02.040 INFO:teuthology.orchestra.run.smithi076.stdout:Deploying unmanaged mon service... 2024-03-23T15:34:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:02 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:34:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:02 smithi076 ceph-mon[32176]: Added host smithi076 2024-03-23T15:34:02.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:02 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:03.411 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-23T15:34:03.411 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.489+0000 7efdf5764700 1 Processor -- start 2024-03-23T15:34:03.411 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.489+0000 7efdf5764700 1 -- start start 2024-03-23T15:34:03.411 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.489+0000 7efdf5764700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf0106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:03.411 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.489+0000 7efdf5764700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01075a0 con 0x7efdf0106bb0 2024-03-23T15:34:03.411 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdeeffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf0106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdeeffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf0106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54794/0 (socket says 172.21.15.76:54794) 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdeeffd700 1 -- 172.21.15.76:0/4112409806 learned_addr learned my addr 172.21.15.76:0/4112409806 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdeeffd700 1 -- 172.21.15.76:0/4112409806 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf0107db0 con 0x7efdf0106bb0 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdeeffd700 1 --2- 172.21.15.76:0/4112409806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf0106fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efde00098a0 tx=0x7efde0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc02a332f2863cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdee7fc700 1 -- 172.21.15.76:0/4112409806 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde0004160 con 0x7efdf0106bb0 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdee7fc700 1 -- 172.21.15.76:0/4112409806 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efde00042c0 con 0x7efdf0106bb0 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.490+0000 7efdee7fc700 1 -- 172.21.15.76:0/4112409806 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde0011ac0 con 0x7efdf0106bb0 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.491+0000 7efdf5764700 1 -- 172.21.15.76:0/4112409806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 msgr2=0x7efdf0106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.491+0000 7efdf5764700 1 --2- 172.21.15.76:0/4112409806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf0106fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efde00098a0 tx=0x7efde0009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.491+0000 7efdf5764700 1 -- 172.21.15.76:0/4112409806 shutdown_connections 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.491+0000 7efdf5764700 1 --2- 172.21.15.76:0/4112409806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf0106fd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.491+0000 7efdf5764700 1 -- 172.21.15.76:0/4112409806 >> 172.21.15.76:0/4112409806 conn(0x7efdf0102130 msgr2=0x7efdf0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.491+0000 7efdf5764700 1 -- 172.21.15.76:0/4112409806 shutdown_connections 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.491+0000 7efdf5764700 1 -- 172.21.15.76:0/4112409806 wait complete. 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdf5764700 1 Processor -- start 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdf5764700 1 -- start start 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdf5764700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf007ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdf5764700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efdf007bf80 con 0x7efdf0106bb0 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdeeffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf007ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdeeffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf007ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54804/0 (socket says 172.21.15.76:54804) 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdeeffd700 1 -- 172.21.15.76:0/2944111895 learned_addr learned my addr 172.21.15.76:0/2944111895 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdeeffd700 1 -- 172.21.15.76:0/2944111895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde0009550 con 0x7efdf0106bb0 2024-03-23T15:34:03.413 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.492+0000 7efdeeffd700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf007ba40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efde0011780 tx=0x7efde0011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.493+0000 7efdecff9700 1 -- 172.21.15.76:0/2944111895 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde0003af0 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.493+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf019e720 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.493+0000 7efdecff9700 1 -- 172.21.15.76:0/2944111895 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efde0003c50 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.493+0000 7efdecff9700 1 -- 172.21.15.76:0/2944111895 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde00197e0 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.493+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf019ebc0 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.493+0000 7efdecff9700 1 -- 172.21.15.76:0/2944111895 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7efde0019a20 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.493+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf010afe0 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.494+0000 7efdecff9700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7efddc038fa0 0x7efddc03b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.494+0000 7efdecff9700 1 -- 172.21.15.76:0/2944111895 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efde004d2f0 con 0x7efdf0106bb0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.494+0000 7efde67fc700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7efddc038fa0 0x7efddc03b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:03.414 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.494+0000 7efde67fc700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7efddc038fa0 0x7efddc03b460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efdd8006fd0 tx=0x7efdd8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.499+0000 7efdecff9700 1 -- 172.21.15.76:0/2944111895 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7efde000b900 con 0x7efdf0106bb0 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.716+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7efdf0103260 con 0x7efddc038fa0 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.721+0000 7efdecff9700 1 -- 172.21.15.76:0/2944111895 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efdf0103260 con 0x7efddc038fa0 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7efddc038fa0 msgr2=0x7efddc03b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7efddc038fa0 0x7efddc03b460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efdd8006fd0 tx=0x7efdd8006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 msgr2=0x7efdf007ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf007ba40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efde0011780 tx=0x7efde0011b40 comp rx=0 tx=0).stop 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 shutdown_connections 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7efddc038fa0 0x7efddc03b460 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 --2- 172.21.15.76:0/2944111895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdf0106bb0 0x7efdf007ba40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 >> 172.21.15.76:0/2944111895 conn(0x7efdf0102130 msgr2=0x7efdf0102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 shutdown_connections 2024-03-23T15:34:03.415 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:02.723+0000 7efdf5764700 1 -- 172.21.15.76:0/2944111895 wait complete. 2024-03-23T15:34:03.416 INFO:teuthology.orchestra.run.smithi076.stdout:Deploying unmanaged mgr service... 2024-03-23T15:34:03.981 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:03 smithi076 ceph-mon[32176]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:03.981 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:03 smithi076 ceph-mon[32176]: Saving service mon spec with placement count:5 2024-03-23T15:34:03.981 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:03 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:34:04.602 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-23T15:34:04.602 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.781+0000 7f697b485700 1 Processor -- start 2024-03-23T15:34:04.602 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.781+0000 7f697b485700 1 -- start start 2024-03-23T15:34:04.602 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f697b485700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f6974106990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:04.602 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f697b485700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f697407be40 con 0x7f6974106570 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f6979221700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f6974106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f6979221700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f6974106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54808/0 (socket says 172.21.15.76:54808) 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f6979221700 1 -- 172.21.15.76:0/2572869430 learned_addr learned my addr 172.21.15.76:0/2572869430 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f6979221700 1 -- 172.21.15.76:0/2572869430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6974106ed0 con 0x7f6974106570 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f6979221700 1 --2- 172.21.15.76:0/2572869430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f6974106990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69640098a0 tx=0x7f6964009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17761af6381ce75 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f696bfff700 1 -- 172.21.15.76:0/2572869430 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6964004160 con 0x7f6974106570 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f696bfff700 1 -- 172.21.15.76:0/2572869430 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f69640042c0 con 0x7f6974106570 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.782+0000 7f696bfff700 1 -- 172.21.15.76:0/2572869430 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6964011ac0 con 0x7f6974106570 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.783+0000 7f697b485700 1 -- 172.21.15.76:0/2572869430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 msgr2=0x7f6974106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.783+0000 7f697b485700 1 --2- 172.21.15.76:0/2572869430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f6974106990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69640098a0 tx=0x7f6964009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.783+0000 7f697b485700 1 -- 172.21.15.76:0/2572869430 shutdown_connections 2024-03-23T15:34:04.603 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.783+0000 7f697b485700 1 --2- 172.21.15.76:0/2572869430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f6974106990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.783+0000 7f697b485700 1 -- 172.21.15.76:0/2572869430 >> 172.21.15.76:0/2572869430 conn(0x7f6974102130 msgr2=0x7f6974104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.783+0000 7f697b485700 1 -- 172.21.15.76:0/2572869430 shutdown_connections 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.783+0000 7f697b485700 1 -- 172.21.15.76:0/2572869430 wait complete. 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f697b485700 1 Processor -- start 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f697b485700 1 -- start start 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f697b485700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f697419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f697b485700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f697419e0e0 con 0x7f6974106570 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f6979221700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f697419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f6979221700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f697419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54818/0 (socket says 172.21.15.76:54818) 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f6979221700 1 -- 172.21.15.76:0/664094175 learned_addr learned my addr 172.21.15.76:0/664094175 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:04.604 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f6979221700 1 -- 172.21.15.76:0/664094175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6964009550 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f6979221700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f697419dba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6964009fb0 tx=0x7f6964011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f696a7fc700 1 -- 172.21.15.76:0/664094175 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69640038b0 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f696a7fc700 1 -- 172.21.15.76:0/664094175 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6964003a10 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697419e2e0 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.784+0000 7f696a7fc700 1 -- 172.21.15.76:0/664094175 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69640195e0 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.785+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697419e780 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.785+0000 7f696a7fc700 1 -- 172.21.15.76:0/664094175 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f69640198e0 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.785+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69741977f0 con 0x7f6974106570 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.785+0000 7f696a7fc700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f6960038f50 0x7f696003b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:04.605 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.786+0000 7f696a7fc700 1 -- 172.21.15.76:0/664094175 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f696404d2d0 con 0x7f6974106570 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.786+0000 7f6978a20700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f6960038f50 0x7f696003b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.786+0000 7f6978a20700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f6960038f50 0x7f696003b410 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6970006fd0 tx=0x7f6970006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:03.790+0000 7f696a7fc700 1 -- 172.21.15.76:0/664094175 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f696402b340 con 0x7f6974106570 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.004+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f697402ddc0 con 0x7f6960038f50 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.008+0000 7f696a7fc700 1 -- 172.21.15.76:0/664094175 <== mgr.14118 v2:172.21.15.76:6800/36977200 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f697402ddc0 con 0x7f6960038f50 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f6960038f50 msgr2=0x7f696003b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f6960038f50 0x7f696003b410 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6970006fd0 tx=0x7f6970006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 msgr2=0x7f697419dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f697419dba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6964009fb0 tx=0x7f6964011b20 comp rx=0 tx=0).stop 2024-03-23T15:34:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 shutdown_connections 2024-03-23T15:34:04.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f6960038f50 0x7f696003b410 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:04.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 --2- 172.21.15.76:0/664094175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6974106570 0x7f697419dba0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:04.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 >> 172.21.15.76:0/664094175 conn(0x7f6974102130 msgr2=0x7f6974194930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:04.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 shutdown_connections 2024-03-23T15:34:04.607 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:04.010+0000 7f697b485700 1 -- 172.21.15.76:0/664094175 wait complete. 2024-03-23T15:34:05.172 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:05 smithi076 ceph-mon[32176]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:05.172 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:05 smithi076 ceph-mon[32176]: Saving service mgr spec with placement count:2 2024-03-23T15:34:05.173 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:05 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c9e12c700 1 Processor -- start 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c9e12c700 1 -- start start 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c9e12c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c9e12c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9807c8a0 con 0x7f8c9807de70 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c977fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c977fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47742/0 (socket says 172.21.15.76:47742) 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c977fe700 1 -- 172.21.15.76:0/1224162466 learned_addr learned my addr 172.21.15.76:0/1224162466 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:05.893 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c977fe700 1 -- 172.21.15.76:0/1224162466 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9807c9e0 con 0x7f8c9807de70 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.014+0000 7f8c977fe700 1 --2- 172.21.15.76:0/1224162466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9807c2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c880098a0 tx=0x7f8c88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6787d75070742136 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c967fc700 1 -- 172.21.15.76:0/1224162466 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c88004160 con 0x7f8c9807de70 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c967fc700 1 -- 172.21.15.76:0/1224162466 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c880042c0 con 0x7f8c9807de70 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c967fc700 1 -- 172.21.15.76:0/1224162466 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c88011ac0 con 0x7f8c9807de70 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c9e12c700 1 -- 172.21.15.76:0/1224162466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 msgr2=0x7f8c9807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c9e12c700 1 --2- 172.21.15.76:0/1224162466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9807c2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c880098a0 tx=0x7f8c88009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c9e12c700 1 -- 172.21.15.76:0/1224162466 shutdown_connections 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c9e12c700 1 --2- 172.21.15.76:0/1224162466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9807c2d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c9e12c700 1 -- 172.21.15.76:0/1224162466 >> 172.21.15.76:0/1224162466 conn(0x7f8c98103280 msgr2=0x7f8c981056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c9e12c700 1 -- 172.21.15.76:0/1224162466 shutdown_connections 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.015+0000 7f8c9e12c700 1 -- 172.21.15.76:0/1224162466 wait complete. 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.016+0000 7f8c9e12c700 1 Processor -- start 2024-03-23T15:34:05.894 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.016+0000 7f8c9e12c700 1 -- start start 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.016+0000 7f8c9e12c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.016+0000 7f8c9e12c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9819e070 con 0x7f8c9807de70 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.016+0000 7f8c977fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.016+0000 7f8c977fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9819db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47752/0 (socket says 172.21.15.76:47752) 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.016+0000 7f8c977fe700 1 -- 172.21.15.76:0/3737799462 learned_addr learned my addr 172.21.15.76:0/3737799462 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.017+0000 7f8c977fe700 1 -- 172.21.15.76:0/3737799462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c88009550 con 0x7f8c9807de70 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.017+0000 7f8c977fe700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9819db30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c88003f70 tx=0x7f8c88011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.017+0000 7f8c94ff9700 1 -- 172.21.15.76:0/3737799462 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c880039f0 con 0x7f8c9807de70 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.017+0000 7f8c94ff9700 1 -- 172.21.15.76:0/3737799462 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c88003b50 con 0x7f8c9807de70 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.017+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9819e270 con 0x7f8c9807de70 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.017+0000 7f8c94ff9700 1 -- 172.21.15.76:0/3737799462 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c8801a760 con 0x7f8c9807de70 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.017+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9819e710 con 0x7f8c9807de70 2024-03-23T15:34:05.895 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.018+0000 7f8c94ff9700 1 -- 172.21.15.76:0/3737799462 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f8c8801a900 con 0x7f8c9807de70 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.018+0000 7f8c94ff9700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f8c80038fa0 0x7f8c8003b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.018+0000 7f8c94ff9700 1 -- 172.21.15.76:0/3737799462 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8c8804d2e0 con 0x7f8c9807de70 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.018+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9810e0f0 con 0x7f8c9807de70 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.018+0000 7f8c96ffd700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f8c80038fa0 0x7f8c8003b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.018+0000 7f8c96ffd700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f8c80038fa0 0x7f8c8003b460 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c006fd0 tx=0x7f8c8c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.023+0000 7f8c94ff9700 1 -- 172.21.15.76:0/3737799462 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8c88018530 con 0x7f8c9807de70 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.227+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f8c9810e300 con 0x7f8c9807de70 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.230+0000 7f8c94ff9700 1 -- 172.21.15.76:0/3737799462 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f8c88018350 con 0x7f8c9807de70 2024-03-23T15:34:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f8c80038fa0 msgr2=0x7f8c8003b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f8c80038fa0 0x7f8c8003b460 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c006fd0 tx=0x7f8c8c006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 msgr2=0x7f8c9819db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9819db30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c88003f70 tx=0x7f8c88011b40 comp rx=0 tx=0).stop 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 shutdown_connections 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f8c80038fa0 0x7f8c8003b460 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 --2- 172.21.15.76:0/3737799462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9807de70 0x7f8c9819db30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 >> 172.21.15.76:0/3737799462 conn(0x7f8c98103280 msgr2=0x7f8c98103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 shutdown_connections 2024-03-23T15:34:05.897 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:05.232+0000 7f8c9e12c700 1 -- 172.21.15.76:0/3737799462 wait complete. 2024-03-23T15:34:06.255 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:06 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3737799462' entity='client.admin' 2024-03-23T15:34:07.284 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.276+0000 7f15df917700 1 Processor -- start 2024-03-23T15:34:07.285 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.276+0000 7f15df917700 1 -- start start 2024-03-23T15:34:07.285 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15df917700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d810ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:07.285 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15df917700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15d807be40 con 0x7f15d81087a0 2024-03-23T15:34:07.285 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15dd6b3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d810ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:07.285 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15dd6b3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d810ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47760/0 (socket says 172.21.15.76:47760) 2024-03-23T15:34:07.285 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15dd6b3700 1 -- 172.21.15.76:0/2165141554 learned_addr learned my addr 172.21.15.76:0/2165141554 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:07.285 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15dd6b3700 1 -- 172.21.15.76:0/2165141554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d810b0d0 con 0x7f15d81087a0 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15dd6b3700 1 --2- 172.21.15.76:0/2165141554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d810ab90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15d40098a0 tx=0x7f15d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22103650247ee08 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15cffff700 1 -- 172.21.15.76:0/2165141554 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15d4004160 con 0x7f15d81087a0 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15cffff700 1 -- 172.21.15.76:0/2165141554 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15d40042c0 con 0x7f15d81087a0 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.277+0000 7f15cffff700 1 -- 172.21.15.76:0/2165141554 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15d4011ac0 con 0x7f15d81087a0 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.278+0000 7f15df917700 1 -- 172.21.15.76:0/2165141554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 msgr2=0x7f15d810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.278+0000 7f15df917700 1 --2- 172.21.15.76:0/2165141554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d810ab90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15d40098a0 tx=0x7f15d4009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.278+0000 7f15df917700 1 -- 172.21.15.76:0/2165141554 shutdown_connections 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.278+0000 7f15df917700 1 --2- 172.21.15.76:0/2165141554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d810ab90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:07.286 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.278+0000 7f15df917700 1 -- 172.21.15.76:0/2165141554 >> 172.21.15.76:0/2165141554 conn(0x7f15d8102190 msgr2=0x7f15d81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.278+0000 7f15df917700 1 -- 172.21.15.76:0/2165141554 shutdown_connections 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.278+0000 7f15df917700 1 -- 172.21.15.76:0/2165141554 wait complete. 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15df917700 1 Processor -- start 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15df917700 1 -- start start 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15df917700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15df917700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15d819e100 con 0x7f15d81087a0 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15dd6b3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15dd6b3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47764/0 (socket says 172.21.15.76:47764) 2024-03-23T15:34:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15dd6b3700 1 -- 172.21.15.76:0/1996194337 learned_addr learned my addr 172.21.15.76:0/1996194337 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15dd6b3700 1 -- 172.21.15.76:0/1996194337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d4009550 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15dd6b3700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d819dbc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15d4011a70 tx=0x7f15d4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15ce7fc700 1 -- 172.21.15.76:0/1996194337 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15d40039c0 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15ce7fc700 1 -- 172.21.15.76:0/1996194337 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15d4003b20 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d819e300 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.279+0000 7f15ce7fc700 1 -- 172.21.15.76:0/1996194337 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15d4024610 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.280+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d819e7a0 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.280+0000 7f15ce7fc700 1 -- 172.21.15.76:0/1996194337 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7f15d402d3f0 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.280+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d810bed0 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.280+0000 7f15ce7fc700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f15c4038ff0 0x7f15c403b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.281+0000 7f15ce7fc700 1 -- 172.21.15.76:0/1996194337 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f15d404d040 con 0x7f15d81087a0 2024-03-23T15:34:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.281+0000 7f15dceb2700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f15c4038ff0 0x7f15c403b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.281+0000 7f15dceb2700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f15c4038ff0 0x7f15c403b4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f15c8006fd0 tx=0x7f15c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.285+0000 7f15ce7fc700 1 -- 172.21.15.76:0/1996194337 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f15d4081280 con 0x7f15d81087a0 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.484+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f15d810c0e0 con 0x7f15d81087a0 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.486+0000 7f15ce7fc700 1 -- 172.21.15.76:0/1996194337 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v7)=0 v7) ==== 130+0+0 (secure 0 0 0) 0x7f15d4010c70 con 0x7f15d81087a0 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f15c4038ff0 msgr2=0x7f15c403b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f15c4038ff0 0x7f15c403b4b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f15c8006fd0 tx=0x7f15c8006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 msgr2=0x7f15d819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d819dbc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15d4011a70 tx=0x7f15d4011b50 comp rx=0 tx=0).stop 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 shutdown_connections 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f15c4038ff0 0x7f15c403b4b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 --2- 172.21.15.76:0/1996194337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15d81087a0 0x7f15d819dbc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:07.289 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 >> 172.21.15.76:0/1996194337 conn(0x7f15d8102190 msgr2=0x7f15d8102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:07.290 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 shutdown_connections 2024-03-23T15:34:07.290 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:06.488+0000 7f15df917700 1 -- 172.21.15.76:0/1996194337 wait complete. 2024-03-23T15:34:07.290 INFO:teuthology.orchestra.run.smithi076.stdout:Enabling the dashboard module... 2024-03-23T15:34:07.563 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:07 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1996194337' entity='client.admin' 2024-03-23T15:34:07.563 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:07 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:34:07.563 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:07 smithi076 ceph-mon[32176]: from='mgr.14118 172.21.15.76:0/2012406986' entity='mgr.x' 2024-03-23T15:34:08.796 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:08 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2584881002' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-23T15:34:08.796 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:08 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: ignoring --setuser ceph since I am not root 2024-03-23T15:34:08.796 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:08 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: ignoring --setgroup ceph since I am not root 2024-03-23T15:34:09.111 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.663+0000 7ff6615c9700 1 Processor -- start 2024-03-23T15:34:09.111 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.663+0000 7ff6615c9700 1 -- start start 2024-03-23T15:34:09.111 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.663+0000 7ff6615c9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c1074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:09.111 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.663+0000 7ff6615c9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c107aa0 con 0x7ff65c1070b0 2024-03-23T15:34:09.111 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.663+0000 7ff65affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c1074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:09.111 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.663+0000 7ff65affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c1074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47776/0 (socket says 172.21.15.76:47776) 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.663+0000 7ff65affd700 1 -- 172.21.15.76:0/1804287752 learned_addr learned my addr 172.21.15.76:0/1804287752 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.664+0000 7ff65affd700 1 -- 172.21.15.76:0/1804287752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c107dd0 con 0x7ff65c1070b0 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.664+0000 7ff65affd700 1 --2- 172.21.15.76:0/1804287752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c1074d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6500098a0 tx=0x7ff650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb906c98ff6bc27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.664+0000 7ff659ffb700 1 -- 172.21.15.76:0/1804287752 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff650004160 con 0x7ff65c1070b0 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.664+0000 7ff659ffb700 1 -- 172.21.15.76:0/1804287752 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6500042c0 con 0x7ff65c1070b0 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.664+0000 7ff659ffb700 1 -- 172.21.15.76:0/1804287752 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff650011a80 con 0x7ff65c1070b0 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 -- 172.21.15.76:0/1804287752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 msgr2=0x7ff65c1074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 --2- 172.21.15.76:0/1804287752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c1074d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6500098a0 tx=0x7ff650009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 -- 172.21.15.76:0/1804287752 shutdown_connections 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 --2- 172.21.15.76:0/1804287752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c1074d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 -- 172.21.15.76:0/1804287752 >> 172.21.15.76:0/1804287752 conn(0x7ff65c101910 msgr2=0x7ff65c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 -- 172.21.15.76:0/1804287752 shutdown_connections 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 -- 172.21.15.76:0/1804287752 wait complete. 2024-03-23T15:34:09.112 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.665+0000 7ff6615c9700 1 Processor -- start 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff6615c9700 1 -- start start 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff6615c9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff6615c9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c19be70 con 0x7ff65c1070b0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff65affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff65affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47790/0 (socket says 172.21.15.76:47790) 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff65affd700 1 -- 172.21.15.76:0/2584881002 learned_addr learned my addr 172.21.15.76:0/2584881002 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff65affd700 1 -- 172.21.15.76:0/2584881002 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff650009550 con 0x7ff65c1070b0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff65affd700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c19b930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff650003f20 tx=0x7ff650011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6500039f0 con 0x7ff65c1070b0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff650003b50 con 0x7ff65c1070b0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c19c070 con 0x7ff65c1070b0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.666+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff65001a6d0 con 0x7ff65c1070b0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.667+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c19c510 con 0x7ff65c1070b0 2024-03-23T15:34:09.113 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.667+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 8) ==== 45928+0+0 (secure 0 0 0) 0x7ff65001a980 con 0x7ff65c1070b0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.667+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c195550 con 0x7ff65c1070b0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.667+0000 7ff643fff700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7ff644038fa0 0x7ff64403b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.668+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff65004e100 con 0x7ff65c1070b0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.668+0000 7ff65a7fc700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7ff644038fa0 0x7ff64403b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.668+0000 7ff65a7fc700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7ff644038fa0 0x7ff64403b460 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.672+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff65001ac30 con 0x7ff65c1070b0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:07.946+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff65c10b000 con 0x7ff65c1070b0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.489+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff650018be0 con 0x7ff65c1070b0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.491+0000 7ff643fff700 1 -- 172.21.15.76:0/2584881002 <== mon.0 v2:172.21.15.76:3300/0 8 ==== mgrmap(e 9) ==== 45941+0+0 (secure 0 0 0) 0x7ff65002d9b0 con 0x7ff65c1070b0 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7ff644038fa0 msgr2=0x7ff64403b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7ff644038fa0 0x7ff64403b460 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 msgr2=0x7ff65c19b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:09.114 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c19b930 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff650003f20 tx=0x7ff650011b00 comp rx=0 tx=0).stop 2024-03-23T15:34:09.115 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 shutdown_connections 2024-03-23T15:34:09.115 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7ff644038fa0 0x7ff64403b460 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:09.115 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 --2- 172.21.15.76:0/2584881002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff65c1070b0 0x7ff65c19b930 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:09.115 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.492+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 >> 172.21.15.76:0/2584881002 conn(0x7ff65c101910 msgr2=0x7ff65c192690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:09.115 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.493+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 shutdown_connections 2024-03-23T15:34:09.115 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:08.493+0000 7ff6615c9700 1 -- 172.21.15.76:0/2584881002 wait complete. 2024-03-23T15:34:09.445 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:09.084+0000 7f62de792200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:34:09.446 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:09.188+0000 7f62de792200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:34:09.753 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2584881002' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-23T15:34:09.753 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:09 smithi076 ceph-mon[32176]: mgrmap e9: x(active, since 18s) 2024-03-23T15:34:10.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:09.875+0000 7f62de792200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:34:10.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:09.968+0000 7f62de792200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:34:10.417 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:10.111+0000 7f62de792200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:34:10.418 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:10.207+0000 7f62de792200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:34:10.418 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:10.292+0000 7f62de792200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.544+0000 7f1b03579700 1 Processor -- start 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.544+0000 7f1b03579700 1 -- start start 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.544+0000 7f1b03579700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc1069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:10.427 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.544+0000 7f1b03579700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc07be20 con 0x7f1afc1065a0 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1b01315700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc1069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1b01315700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc1069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47804/0 (socket says 172.21.15.76:47804) 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1b01315700 1 -- 172.21.15.76:0/1269690525 learned_addr learned my addr 172.21.15.76:0/1269690525 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1b01315700 1 -- 172.21.15.76:0/1269690525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc106f00 con 0x7f1afc1065a0 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1b01315700 1 --2- 172.21.15.76:0/1269690525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc1069c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1af80098a0 tx=0x7f1af8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f2bfcf3c535911 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1af3fff700 1 -- 172.21.15.76:0/1269690525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af8004160 con 0x7f1afc1065a0 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1af3fff700 1 -- 172.21.15.76:0/1269690525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af80042c0 con 0x7f1afc1065a0 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.545+0000 7f1af3fff700 1 -- 172.21.15.76:0/1269690525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af8011a80 con 0x7f1afc1065a0 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.546+0000 7f1b03579700 1 -- 172.21.15.76:0/1269690525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 msgr2=0x7f1afc1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.546+0000 7f1b03579700 1 --2- 172.21.15.76:0/1269690525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc1069c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1af80098a0 tx=0x7f1af8009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.546+0000 7f1b03579700 1 -- 172.21.15.76:0/1269690525 shutdown_connections 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.546+0000 7f1b03579700 1 --2- 172.21.15.76:0/1269690525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc1069c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.546+0000 7f1b03579700 1 -- 172.21.15.76:0/1269690525 >> 172.21.15.76:0/1269690525 conn(0x7f1afc102160 msgr2=0x7f1afc1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.546+0000 7f1b03579700 1 -- 172.21.15.76:0/1269690525 shutdown_connections 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.546+0000 7f1b03579700 1 -- 172.21.15.76:0/1269690525 wait complete. 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.547+0000 7f1b03579700 1 Processor -- start 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.547+0000 7f1b03579700 1 -- start start 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.547+0000 7f1b03579700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.547+0000 7f1b03579700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc19e1a0 con 0x7f1afc1065a0 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.547+0000 7f1b01315700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.547+0000 7f1b01315700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc19dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47814/0 (socket says 172.21.15.76:47814) 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.547+0000 7f1b01315700 1 -- 172.21.15.76:0/3071240913 learned_addr learned my addr 172.21.15.76:0/3071240913 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.548+0000 7f1b01315700 1 -- 172.21.15.76:0/3071240913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af8009550 con 0x7f1afc1065a0 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.548+0000 7f1b01315700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc19dc60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1af8003f20 tx=0x7f1af8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.548+0000 7f1af27fc700 1 -- 172.21.15.76:0/3071240913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af80039f0 con 0x7f1afc1065a0 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.548+0000 7f1af27fc700 1 -- 172.21.15.76:0/3071240913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af8003b50 con 0x7f1afc1065a0 2024-03-23T15:34:10.429 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.548+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc19e3a0 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.549+0000 7f1af27fc700 1 -- 172.21.15.76:0/3071240913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1af801a6d0 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.549+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc19ef50 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.549+0000 7f1af27fc700 1 -- 172.21.15.76:0/3071240913 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 9) ==== 45941+0+0 (secure 0 0 0) 0x7f1af801aa30 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.549+0000 7f1af27fc700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 0x7f1ae803b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.549+0000 7f1b00b14700 1 -- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 msgr2=0x7f1ae803b460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/36977200 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.549+0000 7f1af27fc700 1 -- 172.21.15.76:0/3071240913 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1af8052580 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.549+0000 7f1b00b14700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 0x7f1ae803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.550+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc069300 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.555+0000 7f1af27fc700 1 -- 172.21.15.76:0/3071240913 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1af8053060 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.750+0000 7f1b00b14700 1 -- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 msgr2=0x7f1ae803b460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/36977200 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.750+0000 7f1b00b14700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 0x7f1ae803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.817+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1afc19f960 con 0x7f1afc1065a0 2024-03-23T15:34:10.430 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.817+0000 7f1af27fc700 1 -- 172.21.15.76:0/3071240913 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1af80103c0 con 0x7f1afc1065a0 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 msgr2=0x7f1ae803b460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 0x7f1ae803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 msgr2=0x7f1afc19dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc19dc60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1af8003f20 tx=0x7f1af8011b00 comp rx=0 tx=0).stop 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 shutdown_connections 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f1ae8038fa0 0x7f1ae803b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 --2- 172.21.15.76:0/3071240913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1afc1065a0 0x7f1afc19dc60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.819+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 >> 172.21.15.76:0/3071240913 conn(0x7f1afc102160 msgr2=0x7f1afc194960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.820+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 shutdown_connections 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:09.820+0000 7f1b03579700 1 -- 172.21.15.76:0/3071240913 wait complete. 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for the mgr to restart... 2024-03-23T15:34:10.431 INFO:teuthology.orchestra.run.smithi076.stdout:Waiting for mgr epoch 9... 2024-03-23T15:34:10.668 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3071240913' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T15:34:10.669 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:10.562+0000 7f62de792200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:34:11.042 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:10.675+0000 7f62de792200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:34:11.042 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:10.763+0000 7f62de792200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:34:11.042 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:10.861+0000 7f62de792200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:34:11.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:11 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:11.039+0000 7f62de792200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:34:12.296 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:11 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:11.992+0000 7f62de792200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:34:12.296 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:12.122+0000 7f62de792200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:34:12.549 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:12.457+0000 7f62de792200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:34:12.550 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:12.547+0000 7f62de792200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:34:12.825 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:12.741+0000 7f62de792200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:34:13.213 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:12.925+0000 7f62de792200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:34:13.469 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:13.210+0000 7f62de792200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:34:13.469 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:13.466+0000 7f62de792200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:34:13.825 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:13.702+0000 7f62de792200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:34:14.825 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:14.505+0000 7f62de792200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:34:15.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:15.069+0000 7f62de792200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:34:15.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:15.165+0000 7f62de792200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:34:15.825 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:34:15.381+0000 7f62de792200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: Active manager daemon x restarted 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: Activating manager daemon x 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: osdmap e3: 0 total, 0 up, 0 in 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: mgrmap e10: x(active, starting, since 0.00323461s) 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T15:34:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: Manager daemon x is now available 2024-03-23T15:34:15.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:15.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:16.658 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T15:34:16.658 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T15:34:16.658 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:16 smithi076 ceph-mon[32176]: mgrmap e11: x(active, since 1.0056s) 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout { 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout } 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.885+0000 7f1004331700 1 Processor -- start 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.885+0000 7f1004331700 1 -- start start 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f1004331700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f1004331700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1075b0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10020cd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10020cd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47826/0 (socket says 172.21.15.76:47826) 2024-03-23T15:34:17.320 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10020cd700 1 -- 172.21.15.76:0/1732976177 learned_addr learned my addr 172.21.15.76:0/1732976177 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10020cd700 1 -- 172.21.15.76:0/1732976177 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc107dc0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10020cd700 1 --2- 172.21.15.76:0/1732976177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc106fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0098a0 tx=0x7f0fec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8642abe2ed3d2de1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10010cb700 1 -- 172.21.15.76:0/1732976177 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fec004160 con 0x7f0ffc106bc0 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10010cb700 1 -- 172.21.15.76:0/1732976177 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fec0042c0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.886+0000 7f10010cb700 1 -- 172.21.15.76:0/1732976177 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fec011a80 con 0x7f0ffc106bc0 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.887+0000 7f1004331700 1 -- 172.21.15.76:0/1732976177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 msgr2=0x7f0ffc106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.887+0000 7f1004331700 1 --2- 172.21.15.76:0/1732976177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc106fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0098a0 tx=0x7f0fec009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.887+0000 7f1004331700 1 -- 172.21.15.76:0/1732976177 shutdown_connections 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.887+0000 7f1004331700 1 --2- 172.21.15.76:0/1732976177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc106fe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.887+0000 7f1004331700 1 -- 172.21.15.76:0/1732976177 >> 172.21.15.76:0/1732976177 conn(0x7f0ffc102160 msgr2=0x7f0ffc1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.887+0000 7f1004331700 1 -- 172.21.15.76:0/1732976177 shutdown_connections 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.887+0000 7f1004331700 1 -- 172.21.15.76:0/1732976177 wait complete. 2024-03-23T15:34:17.321 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f1004331700 1 Processor -- start 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f1004331700 1 -- start start 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f1004331700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc07ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f1004331700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc07bf80 con 0x7f0ffc106bc0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f10020cd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc07ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f10020cd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc07ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47836/0 (socket says 172.21.15.76:47836) 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f10020cd700 1 -- 172.21.15.76:0/3818127221 learned_addr learned my addr 172.21.15.76:0/3818127221 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.888+0000 7f10020cd700 1 -- 172.21.15.76:0/3818127221 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fec009550 con 0x7f0ffc106bc0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.889+0000 7f10020cd700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc07ba40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fec003f20 tx=0x7f0fec011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.889+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fec0038b0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.889+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fec003a10 con 0x7f0ffc106bc0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.889+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc19e700 con 0x7f0ffc106bc0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.889+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fec01a5c0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.889+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc19eba0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.322 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.890+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 9) ==== 45941+0+0 (secure 0 0 0) 0x7f0fec01a830 con 0x7f0ffc106bc0 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.890+0000 7f0ff37fe700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.890+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 --> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0ffc10aff0 con 0x7f0fe8038fa0 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.890+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0fec04d1b0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.890+0000 7f10018cc700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 msgr2=0x7f0fe803b460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/36977200 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:10.890+0000 7f10018cc700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:11.091+0000 7f10018cc700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 msgr2=0x7f0fe803b460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/36977200 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:11.091+0000 7f10018cc700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:11.492+0000 7f10018cc700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 msgr2=0x7f0fe803b460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/36977200 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:11.492+0000 7f10018cc700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:12.293+0000 7f10018cc700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 msgr2=0x7f0fe803b460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/36977200 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:12.293+0000 7f10018cc700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:13.894+0000 7f10018cc700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 msgr2=0x7f0fe803b460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/36977200 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:13.894+0000 7f10018cc700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T15:34:17.323 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:15.396+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mgrmap(e 10) ==== 45708+0+0 (secure 0 0 0) 0x7f0fec02d960 con 0x7f0ffc106bc0 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:15.396+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 msgr2=0x7f0fe803b460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:15.396+0000 7f0ff37fe700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.397+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 11) ==== 45835+0+0 (secure 0 0 0) 0x7f0fec0168b0 con 0x7f0ffc106bc0 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.397+0000 7f0ff37fe700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fe803c870 0x7f0fe803ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.397+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0ffc10aff0 con 0x7f0fe803c870 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.398+0000 7f10018cc700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fe803c870 0x7f0fe803ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.398+0000 7f10018cc700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fe803c870 0x7f0fe803ec60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8003a10 tx=0x7f0ff80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.399+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0ffc10aff0 con 0x7f0fe803c870 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.405+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0ffc10b160 con 0x7f0fe803c870 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.405+0000 7f0ff37fe700 1 -- 172.21.15.76:0/3818127221 <== mgr.14150 v2:172.21.15.76:6800/2479514030 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f0ffc10b160 con 0x7f0fe803c870 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fe803c870 msgr2=0x7f0fe803ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:17.324 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fe803c870 0x7f0fe803ec60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8003a10 tx=0x7f0ff80092b0 comp rx=0 tx=0).stop 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 msgr2=0x7f0ffc07ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc07ba40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fec003f20 tx=0x7f0fec011b00 comp rx=0 tx=0).stop 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 shutdown_connections 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fe803c870 0x7f0fe803ec60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ffc106bc0 0x7f0ffc07ba40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 --2- 172.21.15.76:0/3818127221 >> [v2:172.21.15.76:6800/36977200,v1:172.21.15.76:6801/36977200] conn(0x7f0fe8038fa0 0x7f0fe803b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 >> 172.21.15.76:0/3818127221 conn(0x7f0ffc102160 msgr2=0x7f0ffc102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 shutdown_connections 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:16.406+0000 7f1004331700 1 -- 172.21.15.76:0/3818127221 wait complete. 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:mgr epoch 9 is available 2024-03-23T15:34:17.325 INFO:teuthology.orchestra.run.smithi076.stdout:Generating a dashboard self-signed certificate... 2024-03-23T15:34:17.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:17 smithi076 ceph-mon[32176]: [23/Mar/2024:15:34:16] ENGINE Bus STARTING 2024-03-23T15:34:17.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:17 smithi076 ceph-mon[32176]: [23/Mar/2024:15:34:16] ENGINE Serving on https://172.21.15.76:7150 2024-03-23T15:34:17.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:17 smithi076 ceph-mon[32176]: [23/Mar/2024:15:34:16] ENGINE Serving on http://172.21.15.76:8765 2024-03-23T15:34:17.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:17 smithi076 ceph-mon[32176]: [23/Mar/2024:15:34:16] ENGINE Bus STARTED 2024-03-23T15:34:17.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:17 smithi076 ceph-mon[32176]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T15:34:17.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:17 smithi076 ceph-mon[32176]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T15:34:17.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:18.674 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-23T15:34:18.674 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.764+0000 7f223c42f700 1 Processor -- start 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.764+0000 7f223c42f700 1 -- start start 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.765+0000 7f223c42f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f2234106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.765+0000 7f223c42f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22341075b0 con 0x7f2234106bc0 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.765+0000 7f223a1cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f2234106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.765+0000 7f223a1cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f2234106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46578/0 (socket says 172.21.15.76:46578) 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.765+0000 7f223a1cb700 1 -- 172.21.15.76:0/382579967 learned_addr learned my addr 172.21.15.76:0/382579967 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.765+0000 7f223a1cb700 1 -- 172.21.15.76:0/382579967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2234107dc0 con 0x7f2234106bc0 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.765+0000 7f223a1cb700 1 --2- 172.21.15.76:0/382579967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f2234106fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22300098a0 tx=0x7f2230009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273b40a32934c3a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f22391c9700 1 -- 172.21.15.76:0/382579967 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2230004160 con 0x7f2234106bc0 2024-03-23T15:34:18.675 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f22391c9700 1 -- 172.21.15.76:0/382579967 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22300042c0 con 0x7f2234106bc0 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f22391c9700 1 -- 172.21.15.76:0/382579967 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2230011a80 con 0x7f2234106bc0 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f223c42f700 1 -- 172.21.15.76:0/382579967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 msgr2=0x7f2234106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f223c42f700 1 --2- 172.21.15.76:0/382579967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f2234106fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22300098a0 tx=0x7f2230009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f223c42f700 1 -- 172.21.15.76:0/382579967 shutdown_connections 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f223c42f700 1 --2- 172.21.15.76:0/382579967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f2234106fe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f223c42f700 1 -- 172.21.15.76:0/382579967 >> 172.21.15.76:0/382579967 conn(0x7f2234102140 msgr2=0x7f22341045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f223c42f700 1 -- 172.21.15.76:0/382579967 shutdown_connections 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.766+0000 7f223c42f700 1 -- 172.21.15.76:0/382579967 wait complete. 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.767+0000 7f223c42f700 1 Processor -- start 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.767+0000 7f223c42f700 1 -- start start 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.767+0000 7f223c42f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f223419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.767+0000 7f223c42f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f223419bf70 con 0x7f2234106bc0 2024-03-23T15:34:18.676 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.767+0000 7f223a1cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f223419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f223a1cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f223419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46588/0 (socket says 172.21.15.76:46588) 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f223a1cb700 1 -- 172.21.15.76:0/953467023 learned_addr learned my addr 172.21.15.76:0/953467023 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f223a1cb700 1 -- 172.21.15.76:0/953467023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2230009550 con 0x7f2234106bc0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f223a1cb700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f223419ba30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2230003f20 tx=0x7f2230011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22300038b0 con 0x7f2234106bc0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2230003a10 con 0x7f2234106bc0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223419c170 con 0x7f2234106bc0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f223001a5c0 con 0x7f2234106bc0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.768+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223419c610 con 0x7f2234106bc0 2024-03-23T15:34:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.769+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 11) ==== 45835+0+0 (secure 0 0 0) 0x7f223001a850 con 0x7f2234106bc0 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.769+0000 7f222b7fe700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2220038e90 0x7f222003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.769+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f223004d290 con 0x7f2234106bc0 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.769+0000 7f22399ca700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2220038e90 0x7f222003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.769+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2234195c60 con 0x7f2234106bc0 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.770+0000 7f22399ca700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2220038e90 0x7f222003b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2224006fd0 tx=0x7f2224006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.775+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f223001ab50 con 0x7f2234106bc0 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:17.981+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f223402db90 con 0x7f2220038e90 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.014+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 12) ==== 45941+0+0 (secure 0 0 0) 0x7f223001ad30 con 0x7f2234106bc0 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.111+0000 7f222b7fe700 1 -- 172.21.15.76:0/953467023 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f223402db90 con 0x7f2220038e90 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2220038e90 msgr2=0x7f222003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2220038e90 0x7f222003b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2224006fd0 tx=0x7f2224006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 msgr2=0x7f223419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f223419ba30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2230003f20 tx=0x7f2230011b00 comp rx=0 tx=0).stop 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 shutdown_connections 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2220038e90 0x7f222003b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 --2- 172.21.15.76:0/953467023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2234106bc0 0x7f223419ba30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 >> 172.21.15.76:0/953467023 conn(0x7f2234102140 msgr2=0x7f2234102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 shutdown_connections 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:18.113+0000 7f223c42f700 1 -- 172.21.15.76:0/953467023 wait complete. 2024-03-23T15:34:18.679 INFO:teuthology.orchestra.run.smithi076.stdout:Creating initial admin user... 2024-03-23T15:34:19.061 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:19 smithi076 ceph-mon[32176]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:19.061 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:19 smithi076 ceph-mon[32176]: mgrmap e12: x(active, since 2s) 2024-03-23T15:34:19.061 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:19.062 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:19.062 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:20.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:20 smithi076 ceph-mon[32176]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:20.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$w6dFkcViPnaS3gG2mHe13Om/gpvNZtgdqlCZ7IQE3YPG5x2Ob1b2.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711208059, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.091+0000 7fd05fc7f700 1 Processor -- start 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.091+0000 7fd05fc7f700 1 -- start start 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.091+0000 7fd05fc7f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd0581080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.091+0000 7fd05fc7f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd058108690 con 0x7fd058107ca0 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.091+0000 7fd05da1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd0581080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.091+0000 7fd05da1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd0581080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46594/0 (socket says 172.21.15.76:46594) 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.091+0000 7fd05da1b700 1 -- 172.21.15.76:0/1448847939 learned_addr learned my addr 172.21.15.76:0/1448847939 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.092+0000 7fd05da1b700 1 -- 172.21.15.76:0/1448847939 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058108ea0 con 0x7fd058107ca0 2024-03-23T15:34:20.446 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.092+0000 7fd05da1b700 1 --2- 172.21.15.76:0/1448847939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd0581080c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0540098a0 tx=0x7fd054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e462fee8af7efb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.092+0000 7fd05ca19700 1 -- 172.21.15.76:0/1448847939 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd054004160 con 0x7fd058107ca0 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.092+0000 7fd05ca19700 1 -- 172.21.15.76:0/1448847939 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0540042c0 con 0x7fd058107ca0 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.092+0000 7fd05ca19700 1 -- 172.21.15.76:0/1448847939 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd054011a80 con 0x7fd058107ca0 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 -- 172.21.15.76:0/1448847939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 msgr2=0x7fd0581080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 --2- 172.21.15.76:0/1448847939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd0581080c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0540098a0 tx=0x7fd054009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 -- 172.21.15.76:0/1448847939 shutdown_connections 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 --2- 172.21.15.76:0/1448847939 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd0581080c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 -- 172.21.15.76:0/1448847939 >> 172.21.15.76:0/1448847939 conn(0x7fd058103280 msgr2=0x7fd0581056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 -- 172.21.15.76:0/1448847939 shutdown_connections 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 -- 172.21.15.76:0/1448847939 wait complete. 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 Processor -- start 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 -- start start 2024-03-23T15:34:20.447 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd05807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.093+0000 7fd05fc7f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd05807e0a0 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd05da1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd05807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd05da1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd05807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46602/0 (socket says 172.21.15.76:46602) 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd05da1b700 1 -- 172.21.15.76:0/485487893 learned_addr learned my addr 172.21.15.76:0/485487893 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd05da1b700 1 -- 172.21.15.76:0/485487893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd054009550 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd05da1b700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd05807db60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd054000c00 tx=0x7fd054011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd04effd700 1 -- 172.21.15.76:0/485487893 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0540039e0 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd04effd700 1 -- 172.21.15.76:0/485487893 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd054003b40 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05807c290 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd04effd700 1 -- 172.21.15.76:0/485487893 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd054024820 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.094+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05807c730 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.095+0000 7fd04effd700 1 -- 172.21.15.76:0/485487893 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 12) ==== 45941+0+0 (secure 0 0 0) 0x7fd054024980 con 0x7fd058107ca0 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.095+0000 7fd04effd700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd044038ff0 0x7fd04403b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:20.448 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.095+0000 7fd04effd700 1 -- 172.21.15.76:0/485487893 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd05404d250 con 0x7fd058107ca0 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.096+0000 7fd05d21a700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd044038ff0 0x7fd04403b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.096+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058069300 con 0x7fd058107ca0 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.096+0000 7fd05d21a700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd044038ff0 0x7fd04403b4b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd048006fd0 tx=0x7fd048006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.101+0000 7fd04effd700 1 -- 172.21.15.76:0/485487893 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd054024e60 con 0x7fd058107ca0 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.331+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fd058063650 con 0x7fd044038ff0 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.555+0000 7fd04effd700 1 -- 172.21.15.76:0/485487893 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fd058063650 con 0x7fd044038ff0 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd044038ff0 msgr2=0x7fd04403b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd044038ff0 0x7fd04403b4b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd048006fd0 tx=0x7fd048006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 msgr2=0x7fd05807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd05807db60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd054000c00 tx=0x7fd054011b00 comp rx=0 tx=0).stop 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 shutdown_connections 2024-03-23T15:34:20.449 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd044038ff0 0x7fd04403b4b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 --2- 172.21.15.76:0/485487893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd058107ca0 0x7fd05807db60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 >> 172.21.15.76:0/485487893 conn(0x7fd058103280 msgr2=0x7fd058103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.557+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 shutdown_connections 2024-03-23T15:34:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:19.558+0000 7fd05fc7f700 1 -- 172.21.15.76:0/485487893 wait complete. 2024-03-23T15:34:20.450 INFO:teuthology.orchestra.run.smithi076.stdout:Fetching dashboard port number... 2024-03-23T15:34:21.456 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:21 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2954985259' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-23T15:34:21.797 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stdout 8443 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.878+0000 7ff4840a5700 1 Processor -- start 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.878+0000 7ff4840a5700 1 -- start start 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.878+0000 7ff4840a5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff4840a5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c07c8c0 con 0x7ff47c07de90 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff481e41700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff481e41700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46604/0 (socket says 172.21.15.76:46604) 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff481e41700 1 -- 172.21.15.76:0/342030419 learned_addr learned my addr 172.21.15.76:0/342030419 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff481e41700 1 -- 172.21.15.76:0/342030419 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c07ca00 con 0x7ff47c07de90 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff481e41700 1 --2- 172.21.15.76:0/342030419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c07c2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff474009e80 tx=0x7ff47400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f2313f293630c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:21.798 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff480e3f700 1 -- 172.21.15.76:0/342030419 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47400e070 con 0x7ff47c07de90 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff480e3f700 1 -- 172.21.15.76:0/342030419 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff47400b8c0 con 0x7ff47c07de90 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.879+0000 7ff480e3f700 1 -- 172.21.15.76:0/342030419 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4740044a0 con 0x7ff47c07de90 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.880+0000 7ff4840a5700 1 -- 172.21.15.76:0/342030419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 msgr2=0x7ff47c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.880+0000 7ff4840a5700 1 --2- 172.21.15.76:0/342030419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c07c2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff474009e80 tx=0x7ff47400b330 comp rx=0 tx=0).stop 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.880+0000 7ff4840a5700 1 -- 172.21.15.76:0/342030419 shutdown_connections 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.880+0000 7ff4840a5700 1 --2- 172.21.15.76:0/342030419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c07c2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.880+0000 7ff4840a5700 1 -- 172.21.15.76:0/342030419 >> 172.21.15.76:0/342030419 conn(0x7ff47c103280 msgr2=0x7ff47c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.880+0000 7ff4840a5700 1 -- 172.21.15.76:0/342030419 shutdown_connections 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.880+0000 7ff4840a5700 1 -- 172.21.15.76:0/342030419 wait complete. 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff4840a5700 1 Processor -- start 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff4840a5700 1 -- start start 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff4840a5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:21.799 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff4840a5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c19e050 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff481e41700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff481e41700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c19db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46606/0 (socket says 172.21.15.76:46606) 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff481e41700 1 -- 172.21.15.76:0/2954985259 learned_addr learned my addr 172.21.15.76:0/2954985259 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff481e41700 1 -- 172.21.15.76:0/2954985259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff474009b30 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.881+0000 7ff481e41700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c19db10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff474009fd0 tx=0x7ff47400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.882+0000 7ff47affd700 1 -- 172.21.15.76:0/2954985259 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47400e040 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.882+0000 7ff47affd700 1 -- 172.21.15.76:0/2954985259 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff474003e20 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.882+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c19e250 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.882+0000 7ff47affd700 1 -- 172.21.15.76:0/2954985259 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47401c6f0 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.882+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c19e750 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.883+0000 7ff47affd700 1 -- 172.21.15.76:0/2954985259 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 12) ==== 45941+0+0 (secure 0 0 0) 0x7ff474012a30 con 0x7ff47c07de90 2024-03-23T15:34:21.800 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.883+0000 7ff47affd700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff464038ff0 0x7ff46403b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.883+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47c10e140 con 0x7ff47c07de90 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.883+0000 7ff47affd700 1 -- 172.21.15.76:0/2954985259 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff47404d020 con 0x7ff47c07de90 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.883+0000 7ff481640700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff464038ff0 0x7ff46403b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.883+0000 7ff481640700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff464038ff0 0x7ff46403b4b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff470006fd0 tx=0x7ff470006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:20.888+0000 7ff47affd700 1 -- 172.21.15.76:0/2954985259 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff474081280 con 0x7ff47c07de90 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.083+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff47c10e380 con 0x7ff47c07de90 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.084+0000 7ff47affd700 1 -- 172.21.15.76:0/2954985259 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff4740170b0 con 0x7ff47c07de90 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff464038ff0 msgr2=0x7ff46403b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff464038ff0 0x7ff46403b4b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff470006fd0 tx=0x7ff470006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 msgr2=0x7ff47c19db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c19db10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff474009fd0 tx=0x7ff47400bcf0 comp rx=0 tx=0).stop 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 shutdown_connections 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff464038ff0 0x7ff46403b4b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:21.801 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 --2- 172.21.15.76:0/2954985259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff47c07de90 0x7ff47c19db10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 >> 172.21.15.76:0/2954985259 conn(0x7ff47c103280 msgr2=0x7ff47c103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 shutdown_connections 2024-03-23T15:34:21.802 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:21.086+0000 7ff4840a5700 1 -- 172.21.15.76:0/2954985259 wait complete. 2024-03-23T15:34:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T15:34:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout disabled 2024-03-23T15:34:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T15:34:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:systemctl: stdout inactive 2024-03-23T15:34:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:firewalld.service is not enabled 2024-03-23T15:34:21.813 INFO:teuthology.orchestra.run.smithi076.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-23T15:34:21.814 INFO:teuthology.orchestra.run.smithi076.stdout:Ceph Dashboard is now available at: 2024-03-23T15:34:21.814 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:21.814 INFO:teuthology.orchestra.run.smithi076.stdout: URL: https://smithi076.front.sepia.ceph.com:8443/ 2024-03-23T15:34:21.814 INFO:teuthology.orchestra.run.smithi076.stdout: User: admin 2024-03-23T15:34:21.814 INFO:teuthology.orchestra.run.smithi076.stdout: Password: 2rc90guurb 2024-03-23T15:34:21.814 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:21.815 INFO:teuthology.orchestra.run.smithi076.stdout:Saving cluster configuration to /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config directory 2024-03-23T15:34:21.815 INFO:teuthology.orchestra.run.smithi076.stdout:Enabling autotune for osd_memory_target 2024-03-23T15:34:23.031 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1ae15f700 1 Processor -- start 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1ae15f700 1 -- start start 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1ae15f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a81069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1ae15f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a807be40 con 0x7fb1a8106580 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a81069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a81069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46610/0 (socket says 172.21.15.76:46610) 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1a77fe700 1 -- 172.21.15.76:0/1521002954 learned_addr learned my addr 172.21.15.76:0/1521002954 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.240+0000 7fb1a77fe700 1 -- 172.21.15.76:0/1521002954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a8106ee0 con 0x7fb1a8106580 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1a77fe700 1 --2- 172.21.15.76:0/1521002954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a81069a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb1980098a0 tx=0x7fb198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef33765a7aa068b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1a67fc700 1 -- 172.21.15.76:0/1521002954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb198004160 con 0x7fb1a8106580 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1a67fc700 1 -- 172.21.15.76:0/1521002954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1980042c0 con 0x7fb1a8106580 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1a67fc700 1 -- 172.21.15.76:0/1521002954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb198011a80 con 0x7fb1a8106580 2024-03-23T15:34:23.032 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1521002954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 msgr2=0x7fb1a81069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1ae15f700 1 --2- 172.21.15.76:0/1521002954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a81069a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb1980098a0 tx=0x7fb198009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1521002954 shutdown_connections 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1ae15f700 1 --2- 172.21.15.76:0/1521002954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a81069a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.241+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1521002954 >> 172.21.15.76:0/1521002954 conn(0x7fb1a8102160 msgr2=0x7fb1a81045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.242+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1521002954 shutdown_connections 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.242+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1521002954 wait complete. 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.242+0000 7fb1ae15f700 1 Processor -- start 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.242+0000 7fb1ae15f700 1 -- start start 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1ae15f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1ae15f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a819e180 con 0x7fb1a8106580 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:23.033 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a819dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46614/0 (socket says 172.21.15.76:46614) 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a77fe700 1 -- 172.21.15.76:0/1788411995 learned_addr learned my addr 172.21.15.76:0/1788411995 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a77fe700 1 -- 172.21.15.76:0/1788411995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb198009550 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a77fe700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a819dc40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb198003f20 tx=0x7fb198011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a4ff9700 1 -- 172.21.15.76:0/1788411995 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1980039f0 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a819e380 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a4ff9700 1 -- 172.21.15.76:0/1788411995 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb198003b50 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1a4ff9700 1 -- 172.21.15.76:0/1788411995 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb19801a6d0 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.243+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a819ef30 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.244+0000 7fb1a4ff9700 1 -- 172.21.15.76:0/1788411995 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fb19801a990 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.244+0000 7fb1a4ff9700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb190039040 0x7fb19003b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.244+0000 7fb1a4ff9700 1 -- 172.21.15.76:0/1788411995 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb19804d4c0 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.244+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a8063830 con 0x7fb1a8106580 2024-03-23T15:34:23.034 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.244+0000 7fb1a6ffd700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb190039040 0x7fb19003b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.245+0000 7fb1a6ffd700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb190039040 0x7fb19003b500 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb19c006fd0 tx=0x7fb19c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.249+0000 7fb1a4ff9700 1 -- 172.21.15.76:0/1788411995 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb198022330 con 0x7fb1a8106580 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.441+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fb1a819fa00 con 0x7fb1a8106580 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.441+0000 7fb1a4ff9700 1 -- 172.21.15.76:0/1788411995 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fb198018be0 con 0x7fb1a8106580 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb190039040 msgr2=0x7fb19003b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb190039040 0x7fb19003b500 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb19c006fd0 tx=0x7fb19c006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 msgr2=0x7fb1a819dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a819dc40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb198003f20 tx=0x7fb198011b00 comp rx=0 tx=0).stop 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 shutdown_connections 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb190039040 0x7fb19003b500 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 --2- 172.21.15.76:0/1788411995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a8106580 0x7fb1a819dc40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.443+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 >> 172.21.15.76:0/1788411995 conn(0x7fb1a8102160 msgr2=0x7fb1a8194940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.444+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 shutdown_connections 2024-03-23T15:34:23.035 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:22.444+0000 7fb1ae15f700 1 -- 172.21.15.76:0/1788411995 wait complete. 2024-03-23T15:34:23.317 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:23 smithi076 ceph-mon[32176]: mgrmap e13: x(active, since 6s) 2024-03-23T15:34:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.416+0000 7fdd58ca9700 1 Processor -- start 2024-03-23T15:34:24.372 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.416+0000 7fdd58ca9700 1 -- start start 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.416+0000 7fdd58ca9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd541085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.416+0000 7fdd58ca9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54108b90 con 0x7fdd541081a0 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.416+0000 7fdd5259c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd541085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.416+0000 7fdd5259c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd541085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46620/0 (socket says 172.21.15.76:46620) 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.416+0000 7fdd5259c700 1 -- 172.21.15.76:0/244928770 learned_addr learned my addr 172.21.15.76:0/244928770 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.417+0000 7fdd5259c700 1 -- 172.21.15.76:0/244928770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd54106db0 con 0x7fdd541081a0 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.417+0000 7fdd5259c700 1 --2- 172.21.15.76:0/244928770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd541085c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd440098a0 tx=0x7fdd44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cba708d757db2fe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.417+0000 7fdd5159a700 1 -- 172.21.15.76:0/244928770 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd44004160 con 0x7fdd541081a0 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.417+0000 7fdd5159a700 1 -- 172.21.15.76:0/244928770 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd440042c0 con 0x7fdd541081a0 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.417+0000 7fdd5159a700 1 -- 172.21.15.76:0/244928770 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd44011a80 con 0x7fdd541081a0 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 -- 172.21.15.76:0/244928770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 msgr2=0x7fdd541085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 --2- 172.21.15.76:0/244928770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd541085c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd440098a0 tx=0x7fdd44009bb0 comp rx=0 tx=0).stop 2024-03-23T15:34:24.373 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 -- 172.21.15.76:0/244928770 shutdown_connections 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 --2- 172.21.15.76:0/244928770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd541085c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 -- 172.21.15.76:0/244928770 >> 172.21.15.76:0/244928770 conn(0x7fdd54101920 msgr2=0x7fdd54103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 -- 172.21.15.76:0/244928770 shutdown_connections 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 -- 172.21.15.76:0/244928770 wait complete. 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 Processor -- start 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.418+0000 7fdd58ca9700 1 -- start start 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd58ca9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd5419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd58ca9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5419bf70 con 0x7fdd541081a0 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd5259c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd5419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd5259c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd5419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46624/0 (socket says 172.21.15.76:46624) 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd5259c700 1 -- 172.21.15.76:0/1368057166 learned_addr learned my addr 172.21.15.76:0/1368057166 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd5259c700 1 -- 172.21.15.76:0/1368057166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd44009550 con 0x7fdd541081a0 2024-03-23T15:34:24.374 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd5259c700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd5419ba30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd44003f20 tx=0x7fdd44011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd437fe700 1 -- 172.21.15.76:0/1368057166 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd440038b0 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd437fe700 1 -- 172.21.15.76:0/1368057166 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd44003a10 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5419c170 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd437fe700 1 -- 172.21.15.76:0/1368057166 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd4401a5c0 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.419+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5419c610 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.420+0000 7fdd437fe700 1 -- 172.21.15.76:0/1368057166 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fdd4401a7a0 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.420+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd541955f0 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.420+0000 7fdd437fe700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdd3c0390a0 0x7fdd3c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.420+0000 7fdd437fe700 1 -- 172.21.15.76:0/1368057166 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdd4404d380 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.421+0000 7fdd51d9b700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdd3c0390a0 0x7fdd3c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.421+0000 7fdd51d9b700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdd3c0390a0 0x7fdd3c03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd48006fd0 tx=0x7fdd48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.425+0000 7fdd437fe700 1 -- 172.21.15.76:0/1368057166 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fdd44022330 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.726+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fdd5410aff0 con 0x7fdd541081a0 2024-03-23T15:34:24.375 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.728+0000 7fdd437fe700 1 -- 172.21.15.76:0/1368057166 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7fdd44018be0 con 0x7fdd541081a0 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.730+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdd3c0390a0 msgr2=0x7fdd3c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.730+0000 7fdd58ca9700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdd3c0390a0 0x7fdd3c03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd48006fd0 tx=0x7fdd48006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.730+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 msgr2=0x7fdd5419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.730+0000 7fdd58ca9700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd5419ba30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd44003f20 tx=0x7fdd44011b00 comp rx=0 tx=0).stop 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.730+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 shutdown_connections 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.730+0000 7fdd58ca9700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdd3c0390a0 0x7fdd3c03b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.731+0000 7fdd58ca9700 1 --2- 172.21.15.76:0/1368057166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd541081a0 0x7fdd5419ba30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.731+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 >> 172.21.15.76:0/1368057166 conn(0x7fdd54101920 msgr2=0x7fdd54192730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.731+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 shutdown_connections 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr 2024-03-23T15:34:23.731+0000 7fdd58ca9700 1 -- 172.21.15.76:0/1368057166 wait complete. 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.376 INFO:teuthology.orchestra.run.smithi076.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:Or, if you are only running a single cluster on this host: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: ceph telemetry on 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:For more information see: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:24.377 INFO:teuthology.orchestra.run.smithi076.stdout:Bootstrap complete. 2024-03-23T15:34:24.411 INFO:tasks.cephadm:Fetching config... 2024-03-23T15:34:24.412 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:34:24.412 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-23T15:34:24.427 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-23T15:34:24.427 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:34:24.427 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-23T15:34:24.482 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-23T15:34:24.482 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:34:24.482 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-23T15:34:24.550 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-23T15:34:24.551 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:34:24.551 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-23T15:34:24.605 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-23T15:34:24.605 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdXRLCPnE171D1GmqOmiQBxQfkiuDKsm0oSu8Ukc0fucijTpu8uT8PNIv0N/sAr6xnv9NKgOgFqh1Not6t192+DRJRuqvn92SKVBxE9+U+NApVZmBHaXndI5XLPVupS5vWVZsI18pnhdmmLvc1+Lt19r2qdJ61JNOh+xVL5niCiYJv5mvw1bgpdcfLXAWvHNwUUS03Th5orViWGw62nCW4RHBHV05KLbd1mOfgUn3Yra8kIy3Mcl5DaXV0ChdYPBor0U+QLSNEOK7/BNPa9tVH8w9tCdxls6QKqsyfB5AC58gjXAxhEr7WZiF6PxRrsaHimd6q8vmWX4THoPOk+iLc19D5t6mUfCRg5BD9zNKc+2oxsstwMLRewueaG0jThxJn3G2x/fm0h303Mjrirgw7xxCaRsMJHYzlBQpZ934uc7edhOnnjEsDvmY5jK0lbBGv61QuDfBb12MjJoW04B6U2qQ4ws6XJxTlAK67T6pSxiOuuxOTkmmhTK5MDbloBX0= ceph-89c3e582-e92a-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T15:34:24.684 INFO:teuthology.orchestra.run.smithi076.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdXRLCPnE171D1GmqOmiQBxQfkiuDKsm0oSu8Ukc0fucijTpu8uT8PNIv0N/sAr6xnv9NKgOgFqh1Not6t192+DRJRuqvn92SKVBxE9+U+NApVZmBHaXndI5XLPVupS5vWVZsI18pnhdmmLvc1+Lt19r2qdJ61JNOh+xVL5niCiYJv5mvw1bgpdcfLXAWvHNwUUS03Th5orViWGw62nCW4RHBHV05KLbd1mOfgUn3Yra8kIy3Mcl5DaXV0ChdYPBor0U+QLSNEOK7/BNPa9tVH8w9tCdxls6QKqsyfB5AC58gjXAxhEr7WZiF6PxRrsaHimd6q8vmWX4THoPOk+iLc19D5t6mUfCRg5BD9zNKc+2oxsstwMLRewueaG0jThxJn3G2x/fm0h303Mjrirgw7xxCaRsMJHYzlBQpZ934uc7edhOnnjEsDvmY5jK0lbBGv61QuDfBb12MjJoW04B6U2qQ4ws6XJxTlAK67T6pSxiOuuxOTkmmhTK5MDbloBX0= ceph-89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:34:24.699 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdXRLCPnE171D1GmqOmiQBxQfkiuDKsm0oSu8Ukc0fucijTpu8uT8PNIv0N/sAr6xnv9NKgOgFqh1Not6t192+DRJRuqvn92SKVBxE9+U+NApVZmBHaXndI5XLPVupS5vWVZsI18pnhdmmLvc1+Lt19r2qdJ61JNOh+xVL5niCiYJv5mvw1bgpdcfLXAWvHNwUUS03Th5orViWGw62nCW4RHBHV05KLbd1mOfgUn3Yra8kIy3Mcl5DaXV0ChdYPBor0U+QLSNEOK7/BNPa9tVH8w9tCdxls6QKqsyfB5AC58gjXAxhEr7WZiF6PxRrsaHimd6q8vmWX4THoPOk+iLc19D5t6mUfCRg5BD9zNKc+2oxsstwMLRewueaG0jThxJn3G2x/fm0h303Mjrirgw7xxCaRsMJHYzlBQpZ934uc7edhOnnjEsDvmY5jK0lbBGv61QuDfBb12MjJoW04B6U2qQ4ws6XJxTlAK67T6pSxiOuuxOTkmmhTK5MDbloBX0= ceph-89c3e582-e92a-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T15:34:24.744 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdXRLCPnE171D1GmqOmiQBxQfkiuDKsm0oSu8Ukc0fucijTpu8uT8PNIv0N/sAr6xnv9NKgOgFqh1Not6t192+DRJRuqvn92SKVBxE9+U+NApVZmBHaXndI5XLPVupS5vWVZsI18pnhdmmLvc1+Lt19r2qdJ61JNOh+xVL5niCiYJv5mvw1bgpdcfLXAWvHNwUUS03Th5orViWGw62nCW4RHBHV05KLbd1mOfgUn3Yra8kIy3Mcl5DaXV0ChdYPBor0U+QLSNEOK7/BNPa9tVH8w9tCdxls6QKqsyfB5AC58gjXAxhEr7WZiF6PxRrsaHimd6q8vmWX4THoPOk+iLc19D5t6mUfCRg5BD9zNKc+2oxsstwMLRewueaG0jThxJn3G2x/fm0h303Mjrirgw7xxCaRsMJHYzlBQpZ934uc7edhOnnjEsDvmY5jK0lbBGv61QuDfBb12MjJoW04B6U2qQ4ws6XJxTlAK67T6pSxiOuuxOTkmmhTK5MDbloBX0= ceph-89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:34:24.759 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdXRLCPnE171D1GmqOmiQBxQfkiuDKsm0oSu8Ukc0fucijTpu8uT8PNIv0N/sAr6xnv9NKgOgFqh1Not6t192+DRJRuqvn92SKVBxE9+U+NApVZmBHaXndI5XLPVupS5vWVZsI18pnhdmmLvc1+Lt19r2qdJ61JNOh+xVL5niCiYJv5mvw1bgpdcfLXAWvHNwUUS03Th5orViWGw62nCW4RHBHV05KLbd1mOfgUn3Yra8kIy3Mcl5DaXV0ChdYPBor0U+QLSNEOK7/BNPa9tVH8w9tCdxls6QKqsyfB5AC58gjXAxhEr7WZiF6PxRrsaHimd6q8vmWX4THoPOk+iLc19D5t6mUfCRg5BD9zNKc+2oxsstwMLRewueaG0jThxJn3G2x/fm0h303Mjrirgw7xxCaRsMJHYzlBQpZ934uc7edhOnnjEsDvmY5jK0lbBGv61QuDfBb12MjJoW04B6U2qQ4ws6XJxTlAK67T6pSxiOuuxOTkmmhTK5MDbloBX0= ceph-89c3e582-e92a-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T15:34:24.804 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdXRLCPnE171D1GmqOmiQBxQfkiuDKsm0oSu8Ukc0fucijTpu8uT8PNIv0N/sAr6xnv9NKgOgFqh1Not6t192+DRJRuqvn92SKVBxE9+U+NApVZmBHaXndI5XLPVupS5vWVZsI18pnhdmmLvc1+Lt19r2qdJ61JNOh+xVL5niCiYJv5mvw1bgpdcfLXAWvHNwUUS03Th5orViWGw62nCW4RHBHV05KLbd1mOfgUn3Yra8kIy3Mcl5DaXV0ChdYPBor0U+QLSNEOK7/BNPa9tVH8w9tCdxls6QKqsyfB5AC58gjXAxhEr7WZiF6PxRrsaHimd6q8vmWX4THoPOk+iLc19D5t6mUfCRg5BD9zNKc+2oxsstwMLRewueaG0jThxJn3G2x/fm0h303Mjrirgw7xxCaRsMJHYzlBQpZ934uc7edhOnnjEsDvmY5jK0lbBGv61QuDfBb12MjJoW04B6U2qQ4ws6XJxTlAK67T6pSxiOuuxOTkmmhTK5MDbloBX0= ceph-89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:34:24.819 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-23T15:34:24.926 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:24 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1368057166' entity='client.admin' 2024-03-23T15:34:25.060 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:34:27.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.252+0000 7f0573145700 1 -- 172.21.15.76:0/2154984799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 msgr2=0x7f056c103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:27.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.252+0000 7f0573145700 1 --2- 172.21.15.76:0/2154984799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c103660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0560009910 tx=0x7f0560009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:27.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.252+0000 7f0573145700 1 -- 172.21.15.76:0/2154984799 shutdown_connections 2024-03-23T15:34:27.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.252+0000 7f0573145700 1 --2- 172.21.15.76:0/2154984799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c103660 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.252+0000 7f0573145700 1 -- 172.21.15.76:0/2154984799 >> 172.21.15.76:0/2154984799 conn(0x7f056c0fb880 msgr2=0x7f056c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0573145700 1 -- 172.21.15.76:0/2154984799 shutdown_connections 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0573145700 1 -- 172.21.15.76:0/2154984799 wait complete. 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0573145700 1 Processor -- start 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0573145700 1 -- start start 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0573145700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0573145700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f056c197680 con 0x7f056c103240 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0570ee1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0570ee1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43400/0 (socket says 172.21.15.76:43400) 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.253+0000 7f0570ee1700 1 -- 172.21.15.76:0/3855419071 learned_addr learned my addr 172.21.15.76:0/3855419071 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:27.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.254+0000 7f0570ee1700 1 -- 172.21.15.76:0/3855419071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600095f0 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.254+0000 7f0570ee1700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c197140 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0560005850 tx=0x7f0560004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.254+0000 7f0569ffb700 1 -- 172.21.15.76:0/3855419071 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f056001d070 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.254+0000 7f0569ffb700 1 -- 172.21.15.76:0/3855419071 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056000be40 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.254+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c197880 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.254+0000 7f0569ffb700 1 -- 172.21.15.76:0/3855419071 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0560022ea0 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.254+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c197d20 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.255+0000 7f0569ffb700 1 -- 172.21.15.76:0/3855419071 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7f056000f4b0 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.255+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056c190ac0 con 0x7f056c103240 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.255+0000 7f0569ffb700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f055c038fe0 0x7f055c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:27.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.255+0000 7f0569ffb700 1 -- 172.21.15.76:0/3855419071 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f056004d110 con 0x7f056c103240 2024-03-23T15:34:27.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.255+0000 7f056bfff700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f055c038fe0 0x7f055c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:27.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.256+0000 7f056bfff700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f055c038fe0 0x7f055c03b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0558006fd0 tx=0x7f0558006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:27.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.260+0000 7f0569ffb700 1 -- 172.21.15.76:0/3855419071 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0560017440 con 0x7f056c103240 2024-03-23T15:34:27.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.458+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f056c063830 con 0x7f056c103240 2024-03-23T15:34:27.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:27.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:27.460 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:34:27.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:27.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:27.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:27.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.463+0000 7f0569ffb700 1 -- 172.21.15.76:0/3855419071 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0560027030 con 0x7f056c103240 2024-03-23T15:34:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f055c038fe0 msgr2=0x7f055c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f055c038fe0 0x7f055c03b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0558006fd0 tx=0x7f0558006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 msgr2=0x7f056c197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c197140 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0560005850 tx=0x7f0560004ff0 comp rx=0 tx=0).stop 2024-03-23T15:34:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 shutdown_connections 2024-03-23T15:34:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f055c038fe0 0x7f055c03b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 --2- 172.21.15.76:0/3855419071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056c103240 0x7f056c197140 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 >> 172.21.15.76:0/3855419071 conn(0x7f056c0fb880 msgr2=0x7f056c0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 shutdown_connections 2024-03-23T15:34:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:27.465+0000 7f0573145700 1 -- 172.21.15.76:0/3855419071 wait complete. 2024-03-23T15:34:28.068 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-23T15:34:28.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-23T15:34:28.304 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:34:28.717 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:28 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3855419071' entity='client.admin' 2024-03-23T15:34:28.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:28.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.816+0000 7f6904325700 1 -- 172.21.15.76:0/347292181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 msgr2=0x7f68fc104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.816+0000 7f6904325700 1 --2- 172.21.15.76:0/347292181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc104b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009910 tx=0x7f68f8009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.816+0000 7f6904325700 1 -- 172.21.15.76:0/347292181 shutdown_connections 2024-03-23T15:34:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.816+0000 7f6904325700 1 --2- 172.21.15.76:0/347292181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc104b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.816+0000 7f6904325700 1 -- 172.21.15.76:0/347292181 >> 172.21.15.76:0/347292181 conn(0x7f68fc0fc110 msgr2=0x7f68fc0fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.816+0000 7f6904325700 1 -- 172.21.15.76:0/347292181 shutdown_connections 2024-03-23T15:34:29.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.816+0000 7f6904325700 1 -- 172.21.15.76:0/347292181 wait complete. 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.817+0000 7f6904325700 1 Processor -- start 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.817+0000 7f6904325700 1 -- start start 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.817+0000 7f6904325700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.817+0000 7f6904325700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc1976e0 con 0x7f68fc102760 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.817+0000 7f69020c1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.817+0000 7f69020c1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43428/0 (socket says 172.21.15.76:43428) 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.817+0000 7f69020c1700 1 -- 172.21.15.76:0/3215752857 learned_addr learned my addr 172.21.15.76:0/3215752857 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.818+0000 7f69020c1700 1 -- 172.21.15.76:0/3215752857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f80095f0 con 0x7f68fc102760 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.818+0000 7f69020c1700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc1971a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68f8005b40 tx=0x7f68f8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.818+0000 7f68f37fe700 1 -- 172.21.15.76:0/3215752857 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68f801d070 con 0x7f68fc102760 2024-03-23T15:34:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.818+0000 7f68f37fe700 1 -- 172.21.15.76:0/3215752857 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f800be40 con 0x7f68fc102760 2024-03-23T15:34:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.818+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc1978e0 con 0x7f68fc102760 2024-03-23T15:34:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.818+0000 7f68f37fe700 1 -- 172.21.15.76:0/3215752857 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68f8022dc0 con 0x7f68fc102760 2024-03-23T15:34:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.818+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc197d80 con 0x7f68fc102760 2024-03-23T15:34:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.819+0000 7f68f37fe700 1 -- 172.21.15.76:0/3215752857 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7f68f800f460 con 0x7f68fc102760 2024-03-23T15:34:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.819+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc190ab0 con 0x7f68fc102760 2024-03-23T15:34:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.819+0000 7f68f37fe700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f68e8038fe0 0x7f68e803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.819+0000 7f68f37fe700 1 -- 172.21.15.76:0/3215752857 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f68f804d230 con 0x7f68fc102760 2024-03-23T15:34:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.819+0000 7f69018c0700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f68e8038fe0 0x7f68e803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:29.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.820+0000 7f69018c0700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f68e8038fe0 0x7f68e803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:29.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:29.824+0000 7f68f37fe700 1 -- 172.21.15.76:0/3215752857 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f68f8017440 con 0x7f68fc102760 2024-03-23T15:34:30.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.022+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f68fc02db60 con 0x7f68e8038fe0 2024-03-23T15:34:30.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.025+0000 7f68f37fe700 1 -- 172.21.15.76:0/3215752857 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68fc02db60 con 0x7f68e8038fe0 2024-03-23T15:34:30.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f68e8038fe0 msgr2=0x7f68e803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:30.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f68e8038fe0 0x7f68e803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:30.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 msgr2=0x7f68fc1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc1971a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68f8005b40 tx=0x7f68f8004ff0 comp rx=0 tx=0).stop 2024-03-23T15:34:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 shutdown_connections 2024-03-23T15:34:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f68e8038fe0 0x7f68e803b4a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 --2- 172.21.15.76:0/3215752857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68fc102760 0x7f68fc1971a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.027+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 >> 172.21.15.76:0/3215752857 conn(0x7f68fc0fc110 msgr2=0x7f68fc0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.028+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 shutdown_connections 2024-03-23T15:34:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:30.028+0000 7f6904325700 1 -- 172.21.15.76:0/3215752857 wait complete. 2024-03-23T15:34:30.753 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-03-23T15:34:30.753 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:34:30.753 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-03-23T15:34:30.772 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:34:30.772 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:34:30.830 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-03-23T15:34:30.830 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch host add smithi183 2024-03-23T15:34:31.083 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:34:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:34:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:34:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: Updating smithi076:/etc/ceph/ceph.conf 2024-03-23T15:34:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:34:31.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:34:31.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:31.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:31.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:32.282 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:32 smithi076 ceph-mon[32176]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:34:32.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.513+0000 7f76193d6700 1 -- 172.21.15.76:0/3364519292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 msgr2=0x7f7614102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:32.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.513+0000 7f76193d6700 1 --2- 172.21.15.76:0/3364519292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614102210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7604009910 tx=0x7f7604009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:32.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.513+0000 7f76193d6700 1 -- 172.21.15.76:0/3364519292 shutdown_connections 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.513+0000 7f76193d6700 1 --2- 172.21.15.76:0/3364519292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614102210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.513+0000 7f76193d6700 1 -- 172.21.15.76:0/3364519292 >> 172.21.15.76:0/3364519292 conn(0x7f76140fd6c0 msgr2=0x7f76140ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.513+0000 7f76193d6700 1 -- 172.21.15.76:0/3364519292 shutdown_connections 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.513+0000 7f76193d6700 1 -- 172.21.15.76:0/3364519292 wait complete. 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f76193d6700 1 Processor -- start 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f76193d6700 1 -- start start 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f76193d6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f76193d6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7614199c70 con 0x7f7614101e30 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f7612ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:32.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f7612ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43440/0 (socket says 172.21.15.76:43440) 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f7612ffd700 1 -- 172.21.15.76:0/3065058192 learned_addr learned my addr 172.21.15.76:0/3065058192 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f7612ffd700 1 -- 172.21.15.76:0/3065058192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76040095f0 con 0x7f7614101e30 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f7612ffd700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614199730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7604004f40 tx=0x7f7604005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.514+0000 7f7610ff9700 1 -- 172.21.15.76:0/3065058192 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f760401d070 con 0x7f7614101e30 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.515+0000 7f7610ff9700 1 -- 172.21.15.76:0/3065058192 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f760400bc30 con 0x7f7614101e30 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.515+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7614199e70 con 0x7f7614101e30 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.515+0000 7f7610ff9700 1 -- 172.21.15.76:0/3065058192 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7604003970 con 0x7f7614101e30 2024-03-23T15:34:32.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.515+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761419a290 con 0x7f7614101e30 2024-03-23T15:34:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.515+0000 7f7610ff9700 1 -- 172.21.15.76:0/3065058192 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7f7604003c30 con 0x7f7614101e30 2024-03-23T15:34:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.515+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7614063830 con 0x7f7614101e30 2024-03-23T15:34:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.516+0000 7f7610ff9700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7600038f80 0x7f760003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.516+0000 7f7610ff9700 1 -- 172.21.15.76:0/3065058192 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f760404ced0 con 0x7f7614101e30 2024-03-23T15:34:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.516+0000 7f760bfff700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7600038f80 0x7f760003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:32.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.516+0000 7f760bfff700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7600038f80 0x7f760003b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f75fc006fd0 tx=0x7f75fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:32.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.520+0000 7f7610ff9700 1 -- 172.21.15.76:0/3065058192 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7604029930 con 0x7f7614101e30 2024-03-23T15:34:32.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:32.720+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7f761410c100 con 0x7f7600038f80 2024-03-23T15:34:34.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:33 smithi076 ceph-mon[32176]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:34.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:33 smithi076 ceph-mon[32176]: Deploying cephadm binary to smithi183 2024-03-23T15:34:36.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:36 smithi076 ceph-mon[32176]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:36.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.876+0000 7f7610ff9700 1 -- 172.21.15.76:0/3065058192 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f761410c100 con 0x7f7600038f80 2024-03-23T15:34:36.879 INFO:teuthology.orchestra.run.smithi076.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-03-23T15:34:36.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7600038f80 msgr2=0x7f760003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:36.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7600038f80 0x7f760003b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f75fc006fd0 tx=0x7f75fc006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:36.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 msgr2=0x7f7614199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:36.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614199730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7604004f40 tx=0x7f7604005020 comp rx=0 tx=0).stop 2024-03-23T15:34:36.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 shutdown_connections 2024-03-23T15:34:36.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7600038f80 0x7f760003b440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:36.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 --2- 172.21.15.76:0/3065058192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7614101e30 0x7f7614199730 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:36.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 >> 172.21.15.76:0/3065058192 conn(0x7f76140fd6c0 msgr2=0x7f7614074430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:36.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 shutdown_connections 2024-03-23T15:34:36.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:36.879+0000 7f76193d6700 1 -- 172.21.15.76:0/3065058192 wait complete. 2024-03-23T15:34:37.341 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T15:34:37.576 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:34:38.019 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:38.019 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:37 smithi076 ceph-mon[32176]: Added host smithi183 2024-03-23T15:34:38.019 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:38.019 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:38.019 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:37 smithi076 ceph-mon[32176]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:38.019 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:39.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.159+0000 7f06d2e01700 1 -- 172.21.15.76:0/1611378477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 msgr2=0x7f06cc106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:39.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.159+0000 7f06d2e01700 1 --2- 172.21.15.76:0/1611378477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc106ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009910 tx=0x7f06c0009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:39.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.159+0000 7f06d2e01700 1 -- 172.21.15.76:0/1611378477 shutdown_connections 2024-03-23T15:34:39.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.159+0000 7f06d2e01700 1 --2- 172.21.15.76:0/1611378477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc106ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:39.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.159+0000 7f06d2e01700 1 -- 172.21.15.76:0/1611378477 >> 172.21.15.76:0/1611378477 conn(0x7f06cc0fc590 msgr2=0x7f06cc0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.159+0000 7f06d2e01700 1 -- 172.21.15.76:0/1611378477 shutdown_connections 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.159+0000 7f06d2e01700 1 -- 172.21.15.76:0/1611378477 wait complete. 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.160+0000 7f06d2e01700 1 Processor -- start 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.160+0000 7f06d2e01700 1 -- start start 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.160+0000 7f06d2e01700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.160+0000 7f06d2e01700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc196930 con 0x7f06cc106ac0 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.160+0000 7f06d0b9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.160+0000 7f06d0b9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc1963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33898/0 (socket says 172.21.15.76:33898) 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06d0b9d700 1 -- 172.21.15.76:0/2104680857 learned_addr learned my addr 172.21.15.76:0/2104680857 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06d0b9d700 1 -- 172.21.15.76:0/2104680857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c00095f0 con 0x7f06cc106ac0 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06d0b9d700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc1963f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06c0004f40 tx=0x7f06c0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06c9ffb700 1 -- 172.21.15.76:0/2104680857 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c001d070 con 0x7f06cc106ac0 2024-03-23T15:34:39.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc196b90 con 0x7f06cc106ac0 2024-03-23T15:34:39.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06c9ffb700 1 -- 172.21.15.76:0/2104680857 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c000bbd0 con 0x7f06cc106ac0 2024-03-23T15:34:39.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06c9ffb700 1 -- 172.21.15.76:0/2104680857 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c0003900 con 0x7f06cc106ac0 2024-03-23T15:34:39.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.161+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc197010 con 0x7f06cc106ac0 2024-03-23T15:34:39.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.162+0000 7f06c9ffb700 1 -- 172.21.15.76:0/2104680857 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7f06c0003bc0 con 0x7f06cc106ac0 2024-03-23T15:34:39.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.162+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc063830 con 0x7f06cc106ac0 2024-03-23T15:34:39.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.162+0000 7f06c9ffb700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f06bc038f80 0x7f06bc03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:39.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.162+0000 7f06c9ffb700 1 -- 172.21.15.76:0/2104680857 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f06c004cfa0 con 0x7f06cc106ac0 2024-03-23T15:34:39.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.163+0000 7f06cbfff700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f06bc038f80 0x7f06bc03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:39.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.163+0000 7f06cbfff700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f06bc038f80 0x7f06bc03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:39.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.167+0000 7f06c9ffb700 1 -- 172.21.15.76:0/2104680857 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f06c0081280 con 0x7f06cc106ac0 2024-03-23T15:34:39.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.366+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06cc10b0b0 con 0x7f06bc038f80 2024-03-23T15:34:39.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.367+0000 7f06c9ffb700 1 -- 172.21.15.76:0/2104680857 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f06cc10b0b0 con 0x7f06bc038f80 2024-03-23T15:34:39.369 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:39.369 INFO:teuthology.orchestra.run.smithi076.stdout:[{"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f06bc038f80 msgr2=0x7f06bc03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f06bc038f80 0x7f06bc03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 msgr2=0x7f06cc1963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc1963f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06c0004f40 tx=0x7f06c0005020 comp rx=0 tx=0).stop 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 shutdown_connections 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f06bc038f80 0x7f06bc03b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 --2- 172.21.15.76:0/2104680857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06cc106ac0 0x7f06cc1963f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.369+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 >> 172.21.15.76:0/2104680857 conn(0x7f06cc0fc590 msgr2=0x7f06cc1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.370+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 shutdown_connections 2024-03-23T15:34:39.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:39.370+0000 7f06d2e01700 1 -- 172.21.15.76:0/2104680857 wait complete. 2024-03-23T15:34:39.931 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:39.931 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:39 smithi076 ceph-mon[32176]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:34:39.931 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:39 smithi076 ceph-mon[32176]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:39.959 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-03-23T15:34:39.959 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:34:39.959 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-03-23T15:34:39.979 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:34:39.979 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:34:40.036 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-03-23T15:34:40.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch host add smithi192 2024-03-23T15:34:40.280 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:34:41.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 -- 172.21.15.76:0/4235689757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 msgr2=0x7f4d18106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:41.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 --2- 172.21.15.76:0/4235689757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18106f50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009910 tx=0x7f4d08009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:41.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 -- 172.21.15.76:0/4235689757 shutdown_connections 2024-03-23T15:34:41.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 --2- 172.21.15.76:0/4235689757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18106f50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:41.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 -- 172.21.15.76:0/4235689757 >> 172.21.15.76:0/4235689757 conn(0x7f4d180fc5f0 msgr2=0x7f4d180fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:41.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 -- 172.21.15.76:0/4235689757 shutdown_connections 2024-03-23T15:34:41.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 -- 172.21.15.76:0/4235689757 wait complete. 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 Processor -- start 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.674+0000 7f4d1e050700 1 -- start start 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d1e050700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d1e050700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d18197a60 con 0x7f4d18106b70 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d177fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d177fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33918/0 (socket says 172.21.15.76:33918) 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d177fe700 1 -- 172.21.15.76:0/476971511 learned_addr learned my addr 172.21.15.76:0/476971511 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d177fe700 1 -- 172.21.15.76:0/476971511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d080095f0 con 0x7f4d18106b70 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d177fe700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18197520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d08004f40 tx=0x7f4d08005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d14ff9700 1 -- 172.21.15.76:0/476971511 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d0801d070 con 0x7f4d18106b70 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d14ff9700 1 -- 172.21.15.76:0/476971511 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0800bbd0 con 0x7f4d18106b70 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d18197cc0 con 0x7f4d18106b70 2024-03-23T15:34:41.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d14ff9700 1 -- 172.21.15.76:0/476971511 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d08003900 con 0x7f4d18106b70 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.675+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d18198140 con 0x7f4d18106b70 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.676+0000 7f4d14ff9700 1 -- 172.21.15.76:0/476971511 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7f4d08003bc0 con 0x7f4d18106b70 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.676+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d18063830 con 0x7f4d18106b70 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.676+0000 7f4d14ff9700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d00038f80 0x7f4d0003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.676+0000 7f4d14ff9700 1 -- 172.21.15.76:0/476971511 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4d0804cf10 con 0x7f4d18106b70 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.676+0000 7f4d16ffd700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d00038f80 0x7f4d0003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.677+0000 7f4d16ffd700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d00038f80 0x7f4d0003b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:41.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.681+0000 7f4d14ff9700 1 -- 172.21.15.76:0/476971511 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4d08029540 con 0x7f4d18106b70 2024-03-23T15:34:41.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:41.881+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) -- 0x7f4d1810b0d0 con 0x7f4d00038f80 2024-03-23T15:34:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:34:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:34:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:34:43.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-23T15:34:43.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:34:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:34:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: Deploying cephadm binary to smithi192 2024-03-23T15:34:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:34:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:44 smithi076 ceph-mon[32176]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:46.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.996+0000 7f4d14ff9700 1 -- 172.21.15.76:0/476971511 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4d1810b0d0 con 0x7f4d00038f80 2024-03-23T15:34:46.000 INFO:teuthology.orchestra.run.smithi076.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-03-23T15:34:46.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d00038f80 msgr2=0x7f4d0003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:46.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d00038f80 0x7f4d0003b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:46.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 msgr2=0x7f4d18197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:46.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18197520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d08004f40 tx=0x7f4d08005020 comp rx=0 tx=0).stop 2024-03-23T15:34:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 shutdown_connections 2024-03-23T15:34:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d00038f80 0x7f4d0003b440 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 --2- 172.21.15.76:0/476971511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d18106b70 0x7f4d18197520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 >> 172.21.15.76:0/476971511 conn(0x7f4d180fc5f0 msgr2=0x7f4d18104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 shutdown_connections 2024-03-23T15:34:46.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:45.999+0000 7f4d1e050700 1 -- 172.21.15.76:0/476971511 wait complete. 2024-03-23T15:34:46.546 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:46 smithi076 ceph-mon[32176]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:46.546 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:46 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:46.547 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:46 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:46.573 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T15:34:46.805 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:34:47.812 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:47 smithi076 ceph-mon[32176]: Added host smithi192 2024-03-23T15:34:47.812 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:47.812 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.297+0000 7faa16ec8700 1 -- 172.21.15.76:0/4292339955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 msgr2=0x7faa10102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.297+0000 7faa16ec8700 1 --2- 172.21.15.76:0/4292339955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10102220 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa04009910 tx=0x7faa04009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 -- 172.21.15.76:0/4292339955 shutdown_connections 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 --2- 172.21.15.76:0/4292339955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10102220 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 -- 172.21.15.76:0/4292339955 >> 172.21.15.76:0/4292339955 conn(0x7faa100fd710 msgr2=0x7faa100ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 -- 172.21.15.76:0/4292339955 shutdown_connections 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 -- 172.21.15.76:0/4292339955 wait complete. 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 Processor -- start 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 -- start start 2024-03-23T15:34:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa16ec8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa100763f0 con 0x7faa10101e40 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa14c64700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa14c64700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57624/0 (socket says 172.21.15.76:57624) 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.298+0000 7faa14c64700 1 -- 172.21.15.76:0/3438390954 learned_addr learned my addr 172.21.15.76:0/3438390954 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.299+0000 7faa14c64700 1 -- 172.21.15.76:0/3438390954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa040095f0 con 0x7faa10101e40 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.299+0000 7faa14c64700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10074500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa04004f40 tx=0x7faa04005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.299+0000 7faa0dffb700 1 -- 172.21.15.76:0/3438390954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa0401d070 con 0x7faa10101e40 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.299+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa10074a40 con 0x7faa10101e40 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.299+0000 7faa0dffb700 1 -- 172.21.15.76:0/3438390954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0400bc30 con 0x7faa10101e40 2024-03-23T15:34:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.299+0000 7faa0dffb700 1 -- 172.21.15.76:0/3438390954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa04003970 con 0x7faa10101e40 2024-03-23T15:34:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.299+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa10074e60 con 0x7faa10101e40 2024-03-23T15:34:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.300+0000 7faa0dffb700 1 -- 172.21.15.76:0/3438390954 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7faa04003c30 con 0x7faa10101e40 2024-03-23T15:34:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.300+0000 7faa0dffb700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faa00038f80 0x7faa0003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.300+0000 7faa0dffb700 1 -- 172.21.15.76:0/3438390954 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa0404cd60 con 0x7faa10101e40 2024-03-23T15:34:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.300+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa10103320 con 0x7faa10101e40 2024-03-23T15:34:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.300+0000 7faa0ffff700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faa00038f80 0x7faa0003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.301+0000 7faa0ffff700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faa00038f80 0x7faa0003b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.305+0000 7faa0dffb700 1 -- 172.21.15.76:0/3438390954 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7faa04026080 con 0x7faa10101e40 2024-03-23T15:34:48.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.508+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa1010c070 con 0x7faa00038f80 2024-03-23T15:34:48.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.510+0000 7faa0dffb700 1 -- 172.21.15.76:0/3438390954 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7faa1010c070 con 0x7faa00038f80 2024-03-23T15:34:48.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:34:48.513 INFO:teuthology.orchestra.run.smithi076.stdout:[{"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faa00038f80 msgr2=0x7faa0003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faa00038f80 0x7faa0003b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 msgr2=0x7faa10074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10074500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa04004f40 tx=0x7faa04005020 comp rx=0 tx=0).stop 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 shutdown_connections 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faa00038f80 0x7faa0003b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 --2- 172.21.15.76:0/3438390954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa10101e40 0x7faa10074500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 >> 172.21.15.76:0/3438390954 conn(0x7faa100fd710 msgr2=0x7faa10103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 shutdown_connections 2024-03-23T15:34:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:48.512+0000 7faa16ec8700 1 -- 172.21.15.76:0/3438390954 wait complete. 2024-03-23T15:34:48.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:48 smithi076 ceph-mon[32176]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:49.129 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-23T15:34:49.130 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-23T15:34:49.364 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:34:49.736 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:49 smithi076 ceph-mon[32176]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:34:49.737 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:49 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:50.751 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:50 smithi076 ceph-mon[32176]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.940+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1485873292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 msgr2=0x7fc3ec102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.940+0000 7fc3f12ef700 1 --2- 172.21.15.76:0/1485873292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec102220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009910 tx=0x7fc3dc009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.940+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1485873292 shutdown_connections 2024-03-23T15:34:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.940+0000 7fc3f12ef700 1 --2- 172.21.15.76:0/1485873292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec102220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.940+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1485873292 >> 172.21.15.76:0/1485873292 conn(0x7fc3ec0fd710 msgr2=0x7fc3ec0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.940+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1485873292 shutdown_connections 2024-03-23T15:34:50.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.940+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1485873292 wait complete. 2024-03-23T15:34:50.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.941+0000 7fc3f12ef700 1 Processor -- start 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.941+0000 7fc3f12ef700 1 -- start start 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.941+0000 7fc3f12ef700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.941+0000 7fc3f12ef700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec197ac0 con 0x7fc3ec101e40 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.941+0000 7fc3eaffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3eaffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57640/0 (socket says 172.21.15.76:57640) 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3eaffd700 1 -- 172.21.15.76:0/1690175651 learned_addr learned my addr 172.21.15.76:0/1690175651 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3eaffd700 1 -- 172.21.15.76:0/1690175651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc0095f0 con 0x7fc3ec101e40 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3eaffd700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec197580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc004f40 tx=0x7fc3dc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3d3fff700 1 -- 172.21.15.76:0/1690175651 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3dc01d070 con 0x7fc3ec101e40 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3d3fff700 1 -- 172.21.15.76:0/1690175651 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3dc00bc30 con 0x7fc3ec101e40 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec197cc0 con 0x7fc3ec101e40 2024-03-23T15:34:50.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3d3fff700 1 -- 172.21.15.76:0/1690175651 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3dc003970 con 0x7fc3ec101e40 2024-03-23T15:34:50.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.942+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1980e0 con 0x7fc3ec101e40 2024-03-23T15:34:50.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.943+0000 7fc3d3fff700 1 -- 172.21.15.76:0/1690175651 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fc3dc003c30 con 0x7fc3ec101e40 2024-03-23T15:34:50.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.943+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec103320 con 0x7fc3ec101e40 2024-03-23T15:34:50.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.943+0000 7fc3d3fff700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc3d4038f80 0x7fc3d403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:50.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.943+0000 7fc3d3fff700 1 -- 172.21.15.76:0/1690175651 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc3dc04cee0 con 0x7fc3ec101e40 2024-03-23T15:34:50.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.943+0000 7fc3ea7fc700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc3d4038f80 0x7fc3d403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:50.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.944+0000 7fc3ea7fc700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc3d4038f80 0x7fc3d403b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0006fd0 tx=0x7fc3e0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:50.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:50.948+0000 7fc3d3fff700 1 -- 172.21.15.76:0/1690175651 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc3dc029950 con 0x7fc3ec101e40 2024-03-23T15:34:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.147+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc3ec063830 con 0x7fc3ec101e40 2024-03-23T15:34:51.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.530+0000 7fc3d3fff700 1 -- 172.21.15.76:0/1690175651 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc3dc026020 con 0x7fc3ec101e40 2024-03-23T15:34:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc3d4038f80 msgr2=0x7fc3d403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc3d4038f80 0x7fc3d403b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0006fd0 tx=0x7fc3e0006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 msgr2=0x7fc3ec197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec197580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc004f40 tx=0x7fc3dc005020 comp rx=0 tx=0).stop 2024-03-23T15:34:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 shutdown_connections 2024-03-23T15:34:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc3d4038f80 0x7fc3d403b440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 --2- 172.21.15.76:0/1690175651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3ec101e40 0x7fc3ec197580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 >> 172.21.15.76:0/1690175651 conn(0x7fc3ec0fd710 msgr2=0x7fc3ec103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 shutdown_connections 2024-03-23T15:34:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:34:51.533+0000 7fc3f12ef700 1 -- 172.21.15.76:0/1690175651 wait complete. 2024-03-23T15:34:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:adjusted tunables profile to default 2024-03-23T15:34:51.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:51 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1690175651' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T15:34:52.179 INFO:tasks.cephadm:Adding mon.a on smithi076 2024-03-23T15:34:52.179 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-03-23T15:34:52.179 INFO:tasks.cephadm:Adding mon.c on smithi192 2024-03-23T15:34:52.179 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c' 2024-03-23T15:34:52.410 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:34:52.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:52.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1690175651' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:52.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:53.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:53 smithi076 ceph-mon[32176]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-23T15:34:53.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:53 smithi076 ceph-mon[32176]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:34:53.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:53 smithi076 ceph-mon[32176]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:34:53.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:53 smithi076 ceph-mon[32176]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:34:53.906 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.903+0000 7fcff96cb700 1 -- 172.21.15.192:0/2682262816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 msgr2=0x7fcff4103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.903+0000 7fcff96cb700 1 --2- 172.21.15.192:0/2682262816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff4103210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009910 tx=0x7fcfe4009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.903+0000 7fcff96cb700 1 -- 172.21.15.192:0/2682262816 shutdown_connections 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.903+0000 7fcff96cb700 1 --2- 172.21.15.192:0/2682262816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff4103210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.903+0000 7fcff96cb700 1 -- 172.21.15.192:0/2682262816 >> 172.21.15.192:0/2682262816 conn(0x7fcff4077740 msgr2=0x7fcff40fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.903+0000 7fcff96cb700 1 -- 172.21.15.192:0/2682262816 shutdown_connections 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.903+0000 7fcff96cb700 1 -- 172.21.15.192:0/2682262816 wait complete. 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.904+0000 7fcff96cb700 1 Processor -- start 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.904+0000 7fcff96cb700 1 -- start start 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.904+0000 7fcff96cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff41975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.904+0000 7fcff96cb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4197b30 con 0x7fcff4102e30 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.904+0000 7fcff2ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff41975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.904+0000 7fcff2ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff41975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.192:49210/0 (socket says 172.21.15.192:49210) 2024-03-23T15:34:53.907 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.904+0000 7fcff2ffd700 1 -- 172.21.15.192:0/2208947132 learned_addr learned my addr 172.21.15.192:0/2208947132 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:34:53.908 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.905+0000 7fcff2ffd700 1 -- 172.21.15.192:0/2208947132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe40095f0 con 0x7fcff4102e30 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.905+0000 7fcff2ffd700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff41975f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006010 tx=0x7fcfe4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.905+0000 7fcfdbfff700 1 -- 172.21.15.192:0/2208947132 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfe401d070 con 0x7fcff4102e30 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.905+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff4197d30 con 0x7fcff4102e30 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.905+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff4198150 con 0x7fcff4102e30 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.905+0000 7fcfdbfff700 1 -- 172.21.15.192:0/2208947132 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe400bc30 con 0x7fcff4102e30 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.905+0000 7fcfdbfff700 1 -- 172.21.15.192:0/2208947132 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfe40038a0 con 0x7fcff4102e30 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.906+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe0005330 con 0x7fcff4102e30 2024-03-23T15:34:53.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.906+0000 7fcfdbfff700 1 -- 172.21.15.192:0/2208947132 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fcfe4004390 con 0x7fcff4102e30 2024-03-23T15:34:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.906+0000 7fcfdbfff700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcfdc038f80 0x7fcfdc03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.906+0000 7fcfdbfff700 1 -- 172.21.15.192:0/2208947132 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcfe404cda0 con 0x7fcff4102e30 2024-03-23T15:34:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.906+0000 7fcff27fc700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcfdc038f80 0x7fcfdc03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:53.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.907+0000 7fcff27fc700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcfdc038f80 0x7fcfdc03b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8006fd0 tx=0x7fcfe8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:53.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:53.911+0000 7fcfdbfff700 1 -- 172.21.15.192:0/2208947132 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fcfe400b750 con 0x7fcff4102e30 2024-03-23T15:34:54.118 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.116+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}) -- 0x7fcfe0000c80 con 0x7fcfdc038f80 2024-03-23T15:34:54.125 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.123+0000 7fcfdbfff700 1 -- 172.21.15.192:0/2208947132 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcfe0000c80 con 0x7fcfdc038f80 2024-03-23T15:34:54.125 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-03-23T15:34:54.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcfdc038f80 msgr2=0x7fcfdc03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:54.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcfdc038f80 0x7fcfdc03b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8006fd0 tx=0x7fcfe8006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:54.127 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 msgr2=0x7fcff41975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:54.128 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff41975f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4006010 tx=0x7fcfe4004ff0 comp rx=0 tx=0).stop 2024-03-23T15:34:54.128 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 shutdown_connections 2024-03-23T15:34:54.128 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcfdc038f80 0x7fcfdc03b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:54.128 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 --2- 172.21.15.192:0/2208947132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcff4102e30 0x7fcff41975f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:54.128 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 >> 172.21.15.192:0/2208947132 conn(0x7fcff4077740 msgr2=0x7fcff4103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:54.128 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 shutdown_connections 2024-03-23T15:34:54.128 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:54.125+0000 7fcff96cb700 1 -- 172.21.15.192:0/2208947132 wait complete. 2024-03-23T15:34:54.721 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.b.service 2024-03-23T15:34:54.724 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.c.service 2024-03-23T15:34:54.726 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T15:34:54.727 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T15:34:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:34:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:34:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:34:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:34:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:34:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:34:55.441 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:34:55.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:55 smithi076 ceph-mon[32176]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:34:55.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:55 smithi076 ceph-mon[32176]: Saving service mon spec with placement smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c;count:3 2024-03-23T15:34:55.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:55 smithi076 ceph-mon[32176]: Deploying daemon mon.c on smithi192 2024-03-23T15:34:56.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:56 smithi076 ceph-mon[32176]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:58.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 -- 172.21.15.192:0/900105517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 msgr2=0x7fe590106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:58.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 --2- 172.21.15.192:0/900105517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590106f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe580009910 tx=0x7fe580009c20 comp rx=0 tx=0).stop 2024-03-23T15:34:58.367 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 -- 172.21.15.192:0/900105517 shutdown_connections 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 --2- 172.21.15.192:0/900105517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590106f40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 -- 172.21.15.192:0/900105517 >> 172.21.15.192:0/900105517 conn(0x7fe590077740 msgr2=0x7fe5900fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 -- 172.21.15.192:0/900105517 shutdown_connections 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 -- 172.21.15.192:0/900105517 wait complete. 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.364+0000 7fe595b8d700 1 Processor -- start 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.365+0000 7fe595b8d700 1 -- start start 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.365+0000 7fe595b8d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.365+0000 7fe595b8d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe59006a5b0 con 0x7fe590106b60 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.365+0000 7fe58f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.365+0000 7fe58f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.192:49230/0 (socket says 172.21.15.192:49230) 2024-03-23T15:34:58.368 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.365+0000 7fe58f7fe700 1 -- 172.21.15.192:0/1344880409 learned_addr learned my addr 172.21.15.192:0/1344880409 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.365+0000 7fe58f7fe700 1 -- 172.21.15.192:0/1344880409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5800095f0 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.366+0000 7fe58f7fe700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590198370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe580006010 tx=0x7fe580004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.366+0000 7fe58cff9700 1 -- 172.21.15.192:0/1344880409 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe58001d070 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.366+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe590198970 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.366+0000 7fe58cff9700 1 -- 172.21.15.192:0/1344880409 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58000bc30 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.366+0000 7fe58cff9700 1 -- 172.21.15.192:0/1344880409 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe580003930 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.366+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe590198df0 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.367+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe590063830 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.367+0000 7fe58cff9700 1 -- 172.21.15.192:0/1344880409 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fe580003b60 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.367+0000 7fe58cff9700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe57c038fe0 0x7fe57c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.367+0000 7fe58cff9700 1 -- 172.21.15.192:0/1344880409 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe58004cd70 con 0x7fe590106b60 2024-03-23T15:34:58.370 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.367+0000 7fe58effd700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe57c038fe0 0x7fe57c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:34:58.371 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.368+0000 7fe58effd700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe57c038fe0 0x7fe57c03b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe578006fd0 tx=0x7fe578006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:34:58.378 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.372+0000 7fe58cff9700 1 -- 172.21.15.192:0/1344880409 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe58000ba70 con 0x7fe590106b60 2024-03-23T15:34:58.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.671+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe590199070 con 0x7fe590106b60 2024-03-23T15:34:58.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.672+0000 7fe58cff9700 1 -- 172.21.15.192:0/1344880409 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe580026070 con 0x7fe590106b60 2024-03-23T15:34:58.674 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:34:58.674 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","modified":"2024-03-23T15:33:15.904675Z","created":"2024-03-23T15:33:15.904675Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe57c038fe0 msgr2=0x7fe57c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe57c038fe0 0x7fe57c03b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe578006fd0 tx=0x7fe578006e40 comp rx=0 tx=0).stop 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 msgr2=0x7fe590198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590198370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe580006010 tx=0x7fe580004ff0 comp rx=0 tx=0).stop 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 shutdown_connections 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe57c038fe0 0x7fe57c03b4a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 --2- 172.21.15.192:0/1344880409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe590106b60 0x7fe590198370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 >> 172.21.15.192:0/1344880409 conn(0x7fe590077740 msgr2=0x7fe590108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 shutdown_connections 2024-03-23T15:34:58.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:34:58.676+0000 7fe595b8d700 1 -- 172.21.15.192:0/1344880409 wait complete. 2024-03-23T15:34:58.680 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-03-23T15:34:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:58 smithi076 ceph-mon[32176]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:34:58.995 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:58 smithi192 systemd[1]: Starting Ceph mon.c for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:34:59.772 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 podman[36634]: 2024-03-23 15:34:59.519625021 +0000 UTC m=+0.110044135 container create a2d3f72e129101dd95796bc9dbf9258aa483919e2c239e888eca591ff339ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 podman[36634]: 2024-03-23 15:34:59.428784588 +0000 UTC m=+0.019203737 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 podman[36634]: 2024-03-23 15:34:59.738987727 +0000 UTC m=+0.329406868 container init a2d3f72e129101dd95796bc9dbf9258aa483919e2c239e888eca591ff339ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 podman[36634]: 2024-03-23 15:34:59.741856954 +0000 UTC m=+0.332276086 container start a2d3f72e129101dd95796bc9dbf9258aa483919e2c239e888eca591ff339ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev), process ceph-mon, pid 2 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pidfile_write: ignore empty --pid-file 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: load: jerasure load: lrc 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: RocksDB version: 7.9.2 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Git sha 0 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Compile date 2024-03-23 03:28:38 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: DB SUMMARY 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: DB Session ID: 1AL8G3DCGGNZJHFHZL69 2024-03-23T15:34:59.773 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: CURRENT file: CURRENT 2024-03-23T15:34:59.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T15:34:59.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-23T15:34:59.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-23T15:34:59.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-23T15:34:59.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.error_if_exists: 0 2024-03-23T15:34:59.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.create_if_missing: 0 2024-03-23T15:34:59.774 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.paranoid_checks: 1 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.env: 0x559d644d2c20 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.info_log: 0x559d658c7ac0 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.statistics: (nil) 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.use_fsync: 0 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_log_file_size: 0 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T15:34:59.775 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.allow_fallocate: 1 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.use_direct_reads: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.db_log_dir: 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.wal_dir: 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T15:34:59.776 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.write_buffer_manager: 0x559d6589a1e0 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.unordered_write: 0 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T15:34:59.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T15:34:59.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.row_cache: None 2024-03-23T15:34:59.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.wal_filter: None 2024-03-23T15:34:59.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T15:34:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:34:59 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/1344880409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T15:35:00.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.two_write_queues: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.wal_compression: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.atomic_flush: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.log_readahead_size: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T15:35:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_background_jobs: 2 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_background_compactions: -1 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_subcompactions: 1 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T15:35:00.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_open_files: -1 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_background_flushes: -1 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Compression algorithms supported: 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kZSTD supported: 0 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kXpressCompression supported: 0 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kZlibCompression supported: 1 2024-03-23T15:35:00.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kSnappyCompression supported: 1 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kLZ4Compression supported: 1 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: kBZip2Compression supported: 0 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.merge_operator: 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_filter: None 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T15:35:00.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559d658c7bc0) 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks: 1 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: pin_top_level_index_and_filter: 1 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_type: 0 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_index_type: 0 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_shortening: 1 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: checksum: 4 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: no_block_cache: 0 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache: 0x559d658cb090 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_name: BinnedLRUCache 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_options: 2024-03-23T15:35:00.074 INFO:journalctl@ceph.mon.c.smithi192.stdout: capacity : 536870912 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_shard_bits : 4 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: strict_capacity_limit : 0 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: high_pri_pool_ratio: 0.000 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_cache_compressed: (nil) 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: persistent_cache: (nil) 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size: 4096 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_size_deviation: 10 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_restart_interval: 16 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: index_block_restart_interval: 1 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: metadata_block_size: 4096 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: partition_filters: 0 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: use_delta_encoding: 1 2024-03-23T15:35:00.075 INFO:journalctl@ceph.mon.c.smithi192.stdout: filter_policy: bloomfilter 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: whole_key_filtering: 1 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: verify_compression: 0 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: read_amp_bytes_per_bit: 0 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: format_version: 5 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_index_compression: 1 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: block_align: 0 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: max_auto_readahead_size: 262144 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: prepopulate_block_cache: 0 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: initial_auto_readahead_size: 8192 2024-03-23T15:35:00.076 INFO:journalctl@ceph.mon.c.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T15:35:00.078 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T15:35:00.078 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T15:35:00.078 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression: NoCompression 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.num_levels: 7 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T15:35:00.079 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T15:35:00.080 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T15:35:00.081 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T15:35:00.082 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T15:35:00.083 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.table_properties_collectors: 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.inplace_update_support: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.bloom_locality: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.max_successive_merges: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.ttl: 2592000 2024-03-23T15:35:00.084 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.enable_blob_files: false 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.min_blob_size: 0 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T15:35:00.085 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2b74b293-3bb2-4537-8e18-6c9ba7adf7fd 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208099771519, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208099771978, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711208099, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2b74b293-3bb2-4537-8e18-6c9ba7adf7fd", "db_session_id": "1AL8G3DCGGNZJHFHZL69", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208099772057, "job": 1, "event": "recovery_finished"} 2024-03-23T15:35:00.086 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559d659bc000 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: DB pointer 0x559d659a6000 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** DB Stats ** 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T15:35:00.087 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:00.088 INFO:journalctl@ceph.mon.c.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:00.089 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-03-23T15:35:00.089 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** Compaction Stats [default] ** 2024-03-23T15:35:00.089 INFO:journalctl@ceph.mon.c.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T15:35:00.089 INFO:journalctl@ceph.mon.c.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T15:35:00.089 INFO:journalctl@ceph.mon.c.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:00.089 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-03-23T15:35:00.089 INFO:journalctl@ceph.mon.c.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache BinnedLRUCache@0x559d658cb090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-03-23T15:35:00.090 INFO:journalctl@ceph.mon.c.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: using public_addr v2:172.21.15.192:0/0 -> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: starting mon.c rank -1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(???) e0 preinit fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).mds e1 new map 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout: e1 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout: legacy client fscid: -1 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout: 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout: No filesystems configured 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-23T15:35:00.091 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: Added host smithi192 2024-03-23T15:35:00.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1690175651' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1690175651' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.093 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.094 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: Saving service mon spec with placement smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c;count:3 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: Deploying daemon mon.c on smithi192 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/1344880409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 ceph-mon[36653]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 bash[36634]: a2d3f72e129101dd95796bc9dbf9258aa483919e2c239e888eca591ff339ae87 2024-03-23T15:35:00.095 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:34:59 smithi192 systemd[1]: Started Ceph mon.c for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:35:00.296 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T15:35:00.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T15:35:00.533 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:35:05.202 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.199+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/1686614748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 msgr2=0x7fe78c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:05.202 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.199+0000 7fe7a9bf6700 1 --2- 172.21.15.192:0/1686614748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 0x7fe78c005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe7a41012a0 tx=0x7fe79400feb0 comp rx=0 tx=0).stop 2024-03-23T15:35:05.202 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.199+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/1686614748 shutdown_connections 2024-03-23T15:35:05.202 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.199+0000 7fe7a9bf6700 1 --2- 172.21.15.192:0/1686614748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 0x7fe78c005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:05.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.199+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/1686614748 >> 172.21.15.192:0/1686614748 conn(0x7fe7a40772f0 msgr2=0x7fe7a4077700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:05.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.199+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/1686614748 shutdown_connections 2024-03-23T15:35:05.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.199+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/1686614748 wait complete. 2024-03-23T15:35:05.203 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:35:05.203 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.203 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: mon.a calling monitor election 2024-03-23T15:35:05.203 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.203 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:05.203 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: mon.c calling monitor election 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: monmap epoch 2 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: last_changed 2024-03-23T15:34:59.794880+0000 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: min_mon_release 19 (squid) 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: election_strategy: 1 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:05.204 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: fsmap 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: mgrmap e13: x(active, since 49s) 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: overall HEALTH_OK 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:05.205 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:05.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.200+0000 7fe7a9bf6700 1 Processor -- start 2024-03-23T15:35:05.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.200+0000 7fe7a9bf6700 1 -- start start 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a9bf6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 0x7fe7a41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a9bf6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 0x7fe7a419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a9bf6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a4198180 con 0x7fe78c0035c0 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a9bf6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a41982f0 con 0x7fe7a4197bf0 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 0x7fe7a419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 0x7fe7a419bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44404/0 (socket says 172.21.15.192:44404) 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a2ffd700 1 -- 172.21.15.192:0/3323881292 learned_addr learned my addr 172.21.15.192:0/3323881292 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a37fe700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 0x7fe7a41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a2ffd700 1 -- 172.21.15.192:0/3323881292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 msgr2=0x7fe7a419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a2ffd700 1 -- 172.21.15.192:0/3323881292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 msgr2=0x7fe7a419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a2ffd700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 0x7fe7a419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T15:35:05.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.201+0000 7fe7a2ffd700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 0x7fe7a419bf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a37fe700 1 -- 172.21.15.192:0/3323881292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 msgr2=0x7fe7a419bf80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a37fe700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 0x7fe7a419bf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a37fe700 1 -- 172.21.15.192:0/3323881292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79400f900 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a37fe700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 0x7fe7a41976b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe794010010 tx=0x7fe7940056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a0ff9700 1 -- 172.21.15.192:0/3323881292 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe794004aa0 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a419c520 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a0ff9700 1 -- 172.21.15.192:0/3323881292 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe794013e40 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a0ff9700 1 -- 172.21.15.192:0/3323881292 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe794011a80 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.202+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a419ca70 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.203+0000 7fe7a0ff9700 1 -- 172.21.15.192:0/3323881292 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fe794011cf0 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.203+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a4191360 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.204+0000 7fe7a0ff9700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe790038d20 0x7fe79003b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.204+0000 7fe7a0ff9700 1 -- 172.21.15.192:0/3323881292 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe79404e600 con 0x7fe78c0035c0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.204+0000 7fe7a2ffd700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe790038d20 0x7fe79003b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:05.209 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.205+0000 7fe7a2ffd700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe790038d20 0x7fe79003b1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe798009180 tx=0x7fe798008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:05.211 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.209+0000 7fe7a0ff9700 1 -- 172.21.15.192:0/3323881292 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe7940109c0 con 0x7fe78c0035c0 2024-03-23T15:35:05.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:35:05.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: mon.a calling monitor election 2024-03-23T15:35:05.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: mon.c calling monitor election 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: monmap epoch 2 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: last_changed 2024-03-23T15:34:59.794880+0000 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: min_mon_release 19 (squid) 2024-03-23T15:35:05.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: election_strategy: 1 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: fsmap 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: mgrmap e13: x(active, since 49s) 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: overall HEALTH_OK 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:05.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:05.486 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.483+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe7a4109b80 con 0x7fe78c0035c0 2024-03-23T15:35:05.486 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.484+0000 7fe7a0ff9700 1 -- 172.21.15.192:0/3323881292 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe7940103d0 con 0x7fe78c0035c0 2024-03-23T15:35:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:35:05.487 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","modified":"2024-03-23T15:34:59.794880Z","created":"2024-03-23T15:33:15.904675Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe790038d20 msgr2=0x7fe79003b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe790038d20 0x7fe79003b1e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe798009180 tx=0x7fe798008040 comp rx=0 tx=0).stop 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 msgr2=0x7fe7a41976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 0x7fe7a41976b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe794010010 tx=0x7fe7940056d0 comp rx=0 tx=0).stop 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 shutdown_connections 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe790038d20 0x7fe79003b1e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe78c0035c0 0x7fe7a41976b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 --2- 172.21.15.192:0/3323881292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4197bf0 0x7fe7a419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 >> 172.21.15.192:0/3323881292 conn(0x7fe7a40772f0 msgr2=0x7fe7a4107120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 shutdown_connections 2024-03-23T15:35:05.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:05.486+0000 7fe7a9bf6700 1 -- 172.21.15.192:0/3323881292 wait complete. 2024-03-23T15:35:05.490 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-03-23T15:35:06.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:05 smithi192 ceph-mon[36653]: Deploying daemon mon.b on smithi183 2024-03-23T15:35:06.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:05 smithi192 ceph-mon[36653]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:06.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:05 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/3323881292' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T15:35:06.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:05 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:06.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:05 smithi076 ceph-mon[32176]: Deploying daemon mon.b on smithi183 2024-03-23T15:35:06.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:05 smithi076 ceph-mon[32176]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:06.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:05 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/3323881292' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T15:35:06.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:05 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:07.108 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-23T15:35:07.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T15:35:07.345 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:35:08.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 podman[36277]: 2024-03-23 15:35:08.470338645 +0000 UTC m=+0.018029438 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:35:08.698 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:08 smithi192 ceph-mon[36653]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:08.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:08 smithi076 ceph-mon[32176]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 podman[36277]: 2024-03-23 15:35:08.71970215 +0000 UTC m=+0.267392925 container init c1f8e62a41f7bbacb84cc672989830bbb84f9f77548926a07a1dfaa85cddbeb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-23T15:35:09.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 podman[36277]: 2024-03-23 15:35:08.722589782 +0000 UTC m=+0.270280561 container start c1f8e62a41f7bbacb84cc672989830bbb84f9f77548926a07a1dfaa85cddbeb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev), process ceph-mon, pid 2 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pidfile_write: ignore empty --pid-file 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: load: jerasure load: lrc 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: RocksDB version: 7.9.2 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Git sha 0 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Compile date 2024-03-23 03:28:38 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: DB SUMMARY 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: DB Session ID: TR776QOAACQZO2ZR00LR 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: CURRENT file: CURRENT 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-23T15:35:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.error_if_exists: 0 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.create_if_missing: 0 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.paranoid_checks: 1 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.env: 0x555b8b1dcc20 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.info_log: 0x555b8c62fac0 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.statistics: (nil) 2024-03-23T15:35:09.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.use_fsync: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_log_file_size: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.allow_fallocate: 1 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.use_direct_reads: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.db_log_dir: 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.wal_dir: 2024-03-23T15:35:09.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.write_buffer_manager: 0x555b8c6021e0 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T15:35:09.137 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.unordered_write: 0 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.row_cache: None 2024-03-23T15:35:09.138 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.wal_filter: None 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.two_write_queues: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.wal_compression: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.atomic_flush: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T15:35:09.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.log_readahead_size: 0 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_background_jobs: 2 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_background_compactions: -1 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_subcompactions: 1 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T15:35:09.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_open_files: -1 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T15:35:09.142 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_background_flushes: -1 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Compression algorithms supported: 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kZSTD supported: 0 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kXpressCompression supported: 0 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kZlibCompression supported: 1 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kSnappyCompression supported: 1 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kLZ4Compression supported: 1 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: kBZip2Compression supported: 0 2024-03-23T15:35:09.143 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.merge_operator: 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_filter: None 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555b8c62fbc0) 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T15:35:09.144 INFO:journalctl@ceph.mon.b.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_type: 0 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: data_block_index_type: 0 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_shortening: 1 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: checksum: 4 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: no_block_cache: 0 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache: 0x555b8c633090 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_options: 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: capacity : 536870912 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: num_shard_bits : 4 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: strict_capacity_limit : 0 2024-03-23T15:35:09.145 INFO:journalctl@ceph.mon.b.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_compressed: (nil) 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: persistent_cache: (nil) 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_size: 4096 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_size_deviation: 10 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_restart_interval: 16 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_block_restart_interval: 1 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: metadata_block_size: 4096 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: partition_filters: 0 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: use_delta_encoding: 1 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: filter_policy: bloomfilter 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: whole_key_filtering: 1 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: verify_compression: 0 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-03-23T15:35:09.146 INFO:journalctl@ceph.mon.b.smithi183.stdout: format_version: 5 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout: enable_index_compression: 1 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_align: 0 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout: max_auto_readahead_size: 262144 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout: prepopulate_block_cache: 0 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout: initial_auto_readahead_size: 8192 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression: NoCompression 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T15:35:09.147 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.num_levels: 7 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T15:35:09.148 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T15:35:09.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T15:35:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.table_properties_collectors: 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.inplace_update_support: 0 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T15:35:09.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.bloom_locality: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.max_successive_merges: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.ttl: 2592000 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.enable_blob_files: false 2024-03-23T15:35:09.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.min_blob_size: 0 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f634c54a-9686-4299-ba51-03d5ee06ca9c 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208108752226, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-23T15:35:09.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208108752668, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711208108, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f634c54a-9686-4299-ba51-03d5ee06ca9c", "db_session_id": "TR776QOAACQZO2ZR00LR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711208108752749, "job": 1, "event": "recovery_finished"} 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555b8c724000 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: DB pointer 0x555b8c70e000 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** DB Stats ** 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T15:35:09.154 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** Compaction Stats [default] ** 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** Compaction Stats [default] ** 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T15:35:09.155 INFO:journalctl@ceph.mon.b.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Block cache BinnedLRUCache@0x555b8c633090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-23T15:35:09.156 INFO:journalctl@ceph.mon.b.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: starting mon.b rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(???) e0 preinit fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).mds e1 new map 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: e1 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: legacy client fscid: -1 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-03-23T15:35:09.157 INFO:journalctl@ceph.mon.b.smithi183.stdout: No filesystems configured 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Added host smithi192 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1690175651' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1690175651' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Saving service mon spec with placement smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c;count:3 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Deploying daemon mon.c on smithi192 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/1344880409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.a calling monitor election 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.c calling monitor election 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: monmap epoch 2 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: last_changed 2024-03-23T15:34:59.794880+0000 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:09.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: min_mon_release 19 (squid) 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: election_strategy: 1 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: fsmap 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mgrmap e13: x(active, since 49s) 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: overall HEALTH_OK 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: Deploying daemon mon.b on smithi183 2024-03-23T15:35:09.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/3323881292' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T15:35:09.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:09.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:09.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 ceph-mon[36296]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-23T15:35:09.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 bash[36277]: c1f8e62a41f7bbacb84cc672989830bbb84f9f77548926a07a1dfaa85cddbeb5 2024-03-23T15:35:09.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:08 smithi183 systemd[1]: Started Ceph mon.b for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:35:14.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: mon.a calling monitor election 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: mon.c calling monitor election 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: monmap epoch 3 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: last_changed 2024-03-23T15:35:08.774813+0000 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: min_mon_release 19 (squid) 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: election_strategy: 1 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: fsmap 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: mgrmap e13: x(active, since 58s) 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: overall HEALTH_OK 2024-03-23T15:35:14.321 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:14.322 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: mon.a calling monitor election 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: mon.c calling monitor election 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: monmap epoch 3 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: last_changed 2024-03-23T15:35:08.774813+0000 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: min_mon_release 19 (squid) 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: election_strategy: 1 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: fsmap 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:14.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: mgrmap e13: x(active, since 58s) 2024-03-23T15:35:14.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: overall HEALTH_OK 2024-03-23T15:35:14.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:14.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:14.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.903+0000 7fafabf0c700 1 -- 172.21.15.192:0/1713270074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf90006240 msgr2=0x7faf900086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:14.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.903+0000 7fafabf0c700 1 --2- 172.21.15.192:0/1713270074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf90006240 0x7faf900086b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fafa4101cc0 tx=0x7fafa000fe90 comp rx=0 tx=0).stop 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.904+0000 7fafabf0c700 1 -- 172.21.15.192:0/1713270074 shutdown_connections 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.904+0000 7fafabf0c700 1 --2- 172.21.15.192:0/1713270074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf90006240 0x7faf900086b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.904+0000 7fafabf0c700 1 -- 172.21.15.192:0/1713270074 >> 172.21.15.192:0/1713270074 conn(0x7fafa40fc710 msgr2=0x7fafa40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.904+0000 7fafabf0c700 1 -- 172.21.15.192:0/1713270074 shutdown_connections 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.904+0000 7fafabf0c700 1 -- 172.21.15.192:0/1713270074 wait complete. 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.904+0000 7fafabf0c700 1 Processor -- start 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafabf0c700 1 -- start start 2024-03-23T15:35:14.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafabf0c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf90006240 0x7fafa4102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafabf0c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 0x7fafa41032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafabf0c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa41a3ae0 0x7fafa41a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafabf0c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41a64d0 con 0x7fafa41a3ae0 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafabf0c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41a6640 con 0x7fafa4102e40 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafabf0c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41a6800 con 0x7faf90006240 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf90006240 0x7fafa4102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf90006240 0x7fafa4102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44638/0 (socket says 172.21.15.192:44638) 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 -- 172.21.15.192:0/2296345731 learned_addr learned my addr 172.21.15.192:0/2296345731 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa94a7700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 0x7fafa41032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa94a7700 1 -- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 msgr2=0x7fafa41032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa94a7700 1 -- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 msgr2=0x7fafa41032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T15:35:14.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa94a7700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 0x7fafa41032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa94a7700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 0x7fafa41032c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafaa4a9700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa41a3ae0 0x7fafa41a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 -- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 msgr2=0x7fafa41032c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 0x7fafa41032c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 -- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa41a3ae0 msgr2=0x7fafa41a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa41a3ae0 0x7fafa41a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.905+0000 7fafa9ca8700 1 -- 172.21.15.192:0/2296345731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa000f8e0 con 0x7faf90006240 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.906+0000 7fafa9ca8700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf90006240 0x7fafa4102900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafa0015040 tx=0x7fafa0005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.906+0000 7faf9affd700 1 -- 172.21.15.192:0/2296345731 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa001da90 con 0x7faf90006240 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.906+0000 7faf9affd700 1 -- 172.21.15.192:0/2296345731 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa001dbf0 con 0x7faf90006240 2024-03-23T15:35:14.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.906+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa41a6a80 con 0x7faf90006240 2024-03-23T15:35:14.913 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.906+0000 7faf9affd700 1 -- 172.21.15.192:0/2296345731 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00116a0 con 0x7faf90006240 2024-03-23T15:35:14.913 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.906+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa41a6f70 con 0x7faf90006240 2024-03-23T15:35:14.913 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.907+0000 7faf9affd700 1 -- 172.21.15.192:0/2296345731 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fafa0011800 con 0x7faf90006240 2024-03-23T15:35:14.913 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.907+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa4191b70 con 0x7faf90006240 2024-03-23T15:35:14.913 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.907+0000 7faf9affd700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf88039410 0x7faf8803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:14.913 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.907+0000 7faf9affd700 1 -- 172.21.15.192:0/2296345731 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fafa0000e40 con 0x7faf90006240 2024-03-23T15:35:14.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.907+0000 7fafa94a7700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf88039410 0x7faf8803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:14.914 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.908+0000 7fafa94a7700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf88039410 0x7faf8803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf94009850 tx=0x7faf94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:14.915 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:14.912+0000 7faf9affd700 1 -- 172.21.15.192:0/2296345731 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fafa002f440 con 0x7faf90006240 2024-03-23T15:35:15.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.195+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fafa4108aa0 con 0x7faf90006240 2024-03-23T15:35:15.198 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:15.198 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:15.198 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:15.198 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:15.199 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.196+0000 7faf9affd700 1 -- 172.21.15.192:0/2296345731 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fafa0025c10 con 0x7faf90006240 2024-03-23T15:35:15.199 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-03-23T15:35:15.199 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","modified":"2024-03-23T15:35:08.774813Z","created":"2024-03-23T15:33:15.904675Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T15:35:15.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf88039410 msgr2=0x7faf8803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:15.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf88039410 0x7faf8803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faf94009850 tx=0x7faf94008040 comp rx=0 tx=0).stop 2024-03-23T15:35:15.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf90006240 msgr2=0x7fafa4102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:15.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf90006240 0x7fafa4102900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafa0015040 tx=0x7fafa0005790 comp rx=0 tx=0).stop 2024-03-23T15:35:15.200 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 shutdown_connections 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf88039410 0x7faf8803b8d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa41a3ae0 0x7fafa41a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf90006240 0x7fafa4102900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 --2- 172.21.15.192:0/2296345731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafa4102e40 0x7fafa41032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 >> 172.21.15.192:0/2296345731 conn(0x7fafa40fc710 msgr2=0x7fafa418ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 shutdown_connections 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:15.198+0000 7fafabf0c700 1 -- 172.21.15.192:0/2296345731 wait complete. 2024-03-23T15:35:15.201 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2024-03-23T15:35:15.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:15.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:15.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:15.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:15.959 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-23T15:35:15.959 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-23T15:35:16.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:35:16.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: mon.a calling monitor election 2024-03-23T15:35:16.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: mon.c calling monitor election 2024-03-23T15:35:16.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:16.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:35:16.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: monmap epoch 3 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: last_changed 2024-03-23T15:35:08.774813+0000 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:16.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: min_mon_release 19 (squid) 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: election_strategy: 1 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: fsmap 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: mgrmap e13: x(active, since 58s) 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: overall HEALTH_OK 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:16.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:16.201 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:35:16.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: mon.b calling monitor election 2024-03-23T15:35:16.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: mon.b calling monitor election 2024-03-23T15:35:16.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: mon.c calling monitor election 2024-03-23T15:35:16.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: mon.a calling monitor election 2024-03-23T15:35:16.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T15:35:16.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: monmap epoch 3 2024-03-23T15:35:16.578 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: last_changed 2024-03-23T15:35:08.774813+0000 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: min_mon_release 19 (squid) 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: election_strategy: 1 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: fsmap 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: mgrmap e13: x(active, since 60s) 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: overall HEALTH_OK 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:16.580 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: mon.b calling monitor election 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: mon.b calling monitor election 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: mon.c calling monitor election 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: mon.a calling monitor election 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: monmap epoch 3 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: last_changed 2024-03-23T15:35:08.774813+0000 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: min_mon_release 19 (squid) 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: election_strategy: 1 2024-03-23T15:35:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: fsmap 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: mgrmap e13: x(active, since 60s) 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: overall HEALTH_OK 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:16.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:16.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: mon.b calling monitor election 2024-03-23T15:35:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: mon.b calling monitor election 2024-03-23T15:35:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: mon.c calling monitor election 2024-03-23T15:35:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: mon.a calling monitor election 2024-03-23T15:35:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-23T15:35:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: monmap epoch 3 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: last_changed 2024-03-23T15:35:08.774813+0000 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: created 2024-03-23T15:33:15.904675+0000 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: min_mon_release 19 (squid) 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: election_strategy: 1 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: 0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: fsmap 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: mgrmap e13: x(active, since 60s) 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: overall HEALTH_OK 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:16.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:17.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.595+0000 7fa63ceba700 1 -- 172.21.15.76:0/3692649289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638100390 msgr2=0x7fa638100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.595+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3692649289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638100390 0x7fa638100770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa628009910 tx=0x7fa628009c20 comp rx=0 tx=0).stop 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.595+0000 7fa63ceba700 1 -- 172.21.15.76:0/3692649289 shutdown_connections 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.595+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3692649289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638100390 0x7fa638100770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.595+0000 7fa63ceba700 1 -- 172.21.15.76:0/3692649289 >> 172.21.15.76:0/3692649289 conn(0x7fa6380fbd50 msgr2=0x7fa6380fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.595+0000 7fa63ceba700 1 -- 172.21.15.76:0/3692649289 shutdown_connections 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.595+0000 7fa63ceba700 1 -- 172.21.15.76:0/3692649289 wait complete. 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 Processor -- start 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 -- start start 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638100390 0x7fa63819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa63819d970 0x7fa6381975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638197b30 0x7fa638197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:17.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa63819e030 con 0x7fa638197b30 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381a4300 con 0x7fa63819d970 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa63ceba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381a4440 con 0x7fa638100390 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa636d9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638197b30 0x7fa638197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa636d9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638197b30 0x7fa638197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39616/0 (socket says 172.21.15.76:39616) 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa636d9d700 1 -- 172.21.15.76:0/3175845090 learned_addr learned my addr 172.21.15.76:0/3175845090 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa636d9d700 1 -- 172.21.15.76:0/3175845090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638100390 msgr2=0x7fa63819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa636d9d700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638100390 0x7fa63819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa636d9d700 1 -- 172.21.15.76:0/3175845090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa63819d970 msgr2=0x7fa6381975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.596+0000 7fa62dd9b700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa63819d970 0x7fa6381975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa636d9d700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa63819d970 0x7fa6381975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa636d9d700 1 -- 172.21.15.76:0/3175845090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6280095f0 con 0x7fa638197b30 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa62dd9b700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa63819d970 0x7fa6381975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:35:17.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa636d9d700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638197b30 0x7fa638197fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa63000ba50 tx=0x7fa63000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa62ffff700 1 -- 172.21.15.76:0/3175845090 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63000cab0 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa62ffff700 1 -- 172.21.15.76:0/3175845090 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63000cc10 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6381a4720 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa62ffff700 1 -- 172.21.15.76:0/3175845090 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6300075d0 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.597+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381a4c20 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.598+0000 7fa62ffff700 1 -- 172.21.15.76:0/3175845090 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fa6300077b0 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.598+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa638063830 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.598+0000 7fa62ffff700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa624039410 0x7fa62403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.598+0000 7fa62ffff700 1 -- 172.21.15.76:0/3175845090 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa63004b460 con 0x7fa638197b30 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.598+0000 7fa63659c700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa624039410 0x7fa62403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.598+0000 7fa63659c700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa624039410 0x7fa62403b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098e0 tx=0x7fa628005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:17.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.603+0000 7fa62ffff700 1 -- 172.21.15.76:0/3175845090 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa63001b8b0 con 0x7fa638197b30 2024-03-23T15:35:17.774 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:17 smithi076 ceph-mon[32176]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T15:35:17.774 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:17 smithi076 ceph-mon[32176]: Reconfiguring daemon mon.a on smithi076 2024-03-23T15:35:17.774 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:17.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.801+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa63810c090 con 0x7fa638197b30 2024-03-23T15:35:17.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.802+0000 7fa62ffff700 1 -- 172.21.15.76:0/3175845090 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fa63001b2c0 con 0x7fa638197b30 2024-03-23T15:35:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:# minimal ceph.conf for 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:17.807 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-03-23T15:35:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:35:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: mon_host = [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa624039410 msgr2=0x7fa62403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa624039410 0x7fa62403b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098e0 tx=0x7fa628005fb0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638197b30 msgr2=0x7fa638197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638197b30 0x7fa638197fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa63000ba50 tx=0x7fa63000be10 comp rx=0 tx=0).stop 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 shutdown_connections 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa624039410 0x7fa62403b8d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638197b30 0x7fa638197fe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa638100390 0x7fa63819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 --2- 172.21.15.76:0/3175845090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa63819d970 0x7fa6381975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 >> 172.21.15.76:0/3175845090 conn(0x7fa6380fbd50 msgr2=0x7fa638103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 shutdown_connections 2024-03-23T15:35:17.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:17.804+0000 7fa63ceba700 1 -- 172.21.15.76:0/3175845090 wait complete. 2024-03-23T15:35:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:17 smithi192 ceph-mon[36653]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T15:35:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:17 smithi192 ceph-mon[36653]: Reconfiguring daemon mon.a on smithi076 2024-03-23T15:35:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:17 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:17 smithi183 ceph-mon[36296]: Reconfiguring mon.a (unknown last config time)... 2024-03-23T15:35:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:17 smithi183 ceph-mon[36296]: Reconfiguring daemon mon.a on smithi076 2024-03-23T15:35:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:17 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:35:18.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:35:17 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:35:17.774+0000 7f629493c700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-23T15:35:18.448 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-23T15:35:18.448 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:35:18.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T15:35:18.478 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:35:18.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:35:18.546 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:35:18.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T15:35:18.581 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:35:18.581 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:35:18.648 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:18 smithi076 ceph-mon[32176]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:18.648 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:18 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3175845090' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:18.651 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:35:18.651 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T15:35:18.687 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:35:18.687 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:35:18.755 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:18 smithi192 ceph-mon[36653]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:18.755 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:18 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3175845090' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:18.759 INFO:tasks.cephadm:Adding mgr.x on smithi076 2024-03-23T15:35:18.759 INFO:tasks.cephadm:Adding mgr.y on smithi183 2024-03-23T15:35:18.759 INFO:tasks.cephadm:Adding mgr.z on smithi192 2024-03-23T15:35:18.759 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi076=x;smithi183=y;smithi192=z' 2024-03-23T15:35:18.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:18 smithi183 ceph-mon[36296]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:18.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:18 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3175845090' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:19.033 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:35:20.303 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: Reconfiguring mon.b (monmap changed)... 2024-03-23T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: Reconfiguring daemon mon.b on smithi183 2024-03-23T15:35:20.304 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:20 smithi192 ceph-mon[36653]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:20.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.506+0000 7f91e713b700 1 -- 172.21.15.192:0/3770756764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0102870 msgr2=0x7f91e0104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:20.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.506+0000 7f91e713b700 1 --2- 172.21.15.192:0/3770756764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0102870 0x7f91e0104c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009910 tx=0x7f91d4009c20 comp rx=0 tx=0).stop 2024-03-23T15:35:20.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.506+0000 7f91e713b700 1 -- 172.21.15.192:0/3770756764 shutdown_connections 2024-03-23T15:35:20.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.506+0000 7f91e713b700 1 --2- 172.21.15.192:0/3770756764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0102870 0x7f91e0104c60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.506+0000 7f91e713b700 1 -- 172.21.15.192:0/3770756764 >> 172.21.15.192:0/3770756764 conn(0x7f91e00fc1e0 msgr2=0x7f91e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:20.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.506+0000 7f91e713b700 1 -- 172.21.15.192:0/3770756764 shutdown_connections 2024-03-23T15:35:20.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.506+0000 7f91e713b700 1 -- 172.21.15.192:0/3770756764 wait complete. 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 Processor -- start 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 -- start start 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91e0102870 0x7f91e0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91e0197880 0x7f91e019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0197d00 0x7f91e019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91e019f620 con 0x7f91e0197d00 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91e019f790 con 0x7f91e0197880 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e713b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f91e019f950 con 0x7f91e0102870 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91e0102870 0x7f91e0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91e0102870 0x7f91e0197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44654/0 (socket says 172.21.15.192:44654) 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 -- 172.21.15.192:0/1200227714 learned_addr learned my addr 172.21.15.192:0/1200227714 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e56d8700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0197d00 0x7f91e019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91dffff700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91e0197880 0x7f91e019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:20.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 -- 172.21.15.192:0/1200227714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91e0197880 msgr2=0x7f91e019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91e0197880 0x7f91e019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 -- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0197d00 msgr2=0x7f91e019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0197d00 0x7f91e019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 -- 172.21.15.192:0/1200227714 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d40095f0 con 0x7f91e0102870 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91dffff700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91e0197880 0x7f91e019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.507+0000 7f91e4ed7700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91e0102870 0x7f91e0197340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91d4005f50 tx=0x7f91d400bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.508+0000 7f91ddffb700 1 -- 172.21.15.192:0/1200227714 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d401d070 con 0x7f91e0102870 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.508+0000 7f91ddffb700 1 -- 172.21.15.192:0/1200227714 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d4004ec0 con 0x7f91e0102870 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.508+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e019fbd0 con 0x7f91e0102870 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.508+0000 7f91ddffb700 1 -- 172.21.15.192:0/1200227714 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4022dd0 con 0x7f91e0102870 2024-03-23T15:35:20.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.508+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01a00c0 con 0x7f91e0102870 2024-03-23T15:35:20.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.509+0000 7f91ddffb700 1 -- 172.21.15.192:0/1200227714 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7f91d400f460 con 0x7f91e0102870 2024-03-23T15:35:20.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.509+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e0190bc0 con 0x7f91e0102870 2024-03-23T15:35:20.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.509+0000 7f91ddffb700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f91d0039410 0x7f91d003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:20.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.509+0000 7f91ddffb700 1 -- 172.21.15.192:0/1200227714 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f91d404d300 con 0x7f91e0102870 2024-03-23T15:35:20.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.509+0000 7f91dffff700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f91d0039410 0x7f91d003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:20.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.509+0000 7f91dffff700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f91d0039410 0x7f91d003b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009910 tx=0x7f91cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:20.516 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.514+0000 7f91ddffb700 1 -- 172.21.15.192:0/1200227714 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f91d4020940 con 0x7f91e0102870 2024-03-23T15:35:20.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:20.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:20.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: Reconfiguring mon.b (monmap changed)... 2024-03-23T15:35:20.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:20.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:20.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:20.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: Reconfiguring daemon mon.b on smithi183 2024-03-23T15:35:20.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:20 smithi076 ceph-mon[32176]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: Reconfiguring mon.b (monmap changed)... 2024-03-23T15:35:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: Reconfiguring daemon mon.b on smithi183 2024-03-23T15:35:20.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:20 smithi183 ceph-mon[36296]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:20.722 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.720+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi076=x;smithi183=y;smithi192=z", "target": ["mon-mgr", ""]}) -- 0x7f91e002dc00 con 0x7f91d0039410 2024-03-23T15:35:20.729 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.726+0000 7f91ddffb700 1 -- 172.21.15.192:0/1200227714 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f91e002dc00 con 0x7f91d0039410 2024-03-23T15:35:20.729 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2024-03-23T15:35:20.731 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.728+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f91d0039410 msgr2=0x7f91d003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:20.731 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.728+0000 7f91e713b700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f91d0039410 0x7f91d003b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009910 tx=0x7f91cc008040 comp rx=0 tx=0).stop 2024-03-23T15:35:20.731 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.728+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91e0102870 msgr2=0x7f91e0197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:20.731 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.728+0000 7f91e713b700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91e0102870 0x7f91e0197340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91d4005f50 tx=0x7f91d400bcf0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.731 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 shutdown_connections 2024-03-23T15:35:20.731 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f91d0039410 0x7f91d003b8d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.731 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91e0197d00 0x7f91e019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91e0102870 0x7f91e0197340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 --2- 172.21.15.192:0/1200227714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91e0197880 0x7f91e019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 >> 172.21.15.192:0/1200227714 conn(0x7f91e00fc1e0 msgr2=0x7f91e00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 shutdown_connections 2024-03-23T15:35:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:35:20.729+0000 7f91e713b700 1 -- 172.21.15.192:0/1200227714 wait complete. 2024-03-23T15:35:21.356 DEBUG:teuthology.orchestra.run.smithi183:mgr.y> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.y.service 2024-03-23T15:35:21.359 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.z.service 2024-03-23T15:35:21.361 INFO:tasks.cephadm:Deploying OSDs... 2024-03-23T15:35:21.361 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:35:21.361 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T15:35:21.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T15:35:21.380 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-03-23T15:35:21.437 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T15:35:21.437 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:21.437 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-23T15:35:21.437 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-23 15:34:25.356463187 +0000 2024-03-23T15:35:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-23 15:29:30.172078703 +0000 2024-03-23T15:35:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-23 15:29:30.172078703 +0000 2024-03-23T15:35:21.438 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-23 15:29:30.172078703 +0000 2024-03-23T15:35:21.438 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T15:35:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-23T15:35:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-23T15:35:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000227064 s, 2.3 MB/s 2024-03-23T15:35:21.510 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T15:35:21.567 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-23 15:34:25.402461751 +0000 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-23 15:29:30.518068002 +0000 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-23 15:29:30.518068002 +0000 2024-03-23T15:35:21.624 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-23 15:29:30.518068002 +0000 2024-03-23T15:35:21.625 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T15:35:21.691 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-23T15:35:21.691 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-23T15:35:21.691 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000223617 s, 2.3 MB/s 2024-03-23T15:35:21.692 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T15:35:21.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi076=x;smithi183=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:21.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: Saving service mgr spec with placement smithi076=x;smithi183=y;smithi192=z;count:3 2024-03-23T15:35:21.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:21.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:21.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:21.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: Reconfiguring mon.c (monmap changed)... 2024-03-23T15:35:21.749 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-03-23T15:35:21.805 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T15:35:21.805 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-23T15:35:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-23 15:34:25.435460722 +0000 2024-03-23T15:35:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-23 15:29:30.849057764 +0000 2024-03-23T15:35:21.806 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-23 15:29:30.849057764 +0000 2024-03-23T15:35:21.806 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-23 15:29:30.849057764 +0000 2024-03-23T15:35:21.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T15:35:21.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi076=x;smithi183=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: Saving service mgr spec with placement smithi076=x;smithi183=y;smithi192=z;count:3 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: Reconfiguring mon.c (monmap changed)... 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: Reconfiguring daemon mon.c on smithi192 2024-03-23T15:35:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:21 smithi076 ceph-mon[32176]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:21.852 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-23T15:35:21.852 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-23T15:35:21.852 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000166749 s, 3.1 MB/s 2024-03-23T15:35:21.853 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T15:35:21.909 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-23 15:34:25.473459536 +0000 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-23 15:29:31.188047280 +0000 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-23 15:29:31.188047280 +0000 2024-03-23T15:35:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-23 15:29:31.188047280 +0000 2024-03-23T15:35:21.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T15:35:22.030 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-03-23T15:35:22.030 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-03-23T15:35:22.030 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000239081 s, 2.1 MB/s 2024-03-23T15:35:22.031 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T15:35:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: Reconfiguring daemon mon.c on smithi192 2024-03-23T15:35:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:21 smithi192 ceph-mon[36653]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:22.087 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-23T15:35:22.087 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T15:35:22.106 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T15:35:22.106 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-03-23T15:35:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi076=x;smithi183=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: Saving service mgr spec with placement smithi076=x;smithi183=y;smithi192=z;count:3 2024-03-23T15:35:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: Reconfiguring mon.c (monmap changed)... 2024-03-23T15:35:22.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T15:35:22.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T15:35:22.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:22.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: Reconfiguring daemon mon.c on smithi192 2024-03-23T15:35:22.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:21 smithi183 ceph-mon[36296]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T15:35:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:22.151 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-23T15:35:22.151 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:22.151 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:22.152 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-23 15:35:08.271241483 +0000 2024-03-23T15:35:22.152 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-23 15:29:54.766750533 +0000 2024-03-23T15:35:22.152 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-23 15:29:54.766750533 +0000 2024-03-23T15:35:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-23 15:29:54.766750533 +0000 2024-03-23T15:35:22.152 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T15:35:22.221 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-23T15:35:22.221 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-23T15:35:22.221 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000242314 s, 2.1 MB/s 2024-03-23T15:35:22.222 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T15:35:22.280 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-23 15:35:08.271241483 +0000 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-23 15:29:55.111741008 +0000 2024-03-23T15:35:22.336 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-23 15:29:55.111741008 +0000 2024-03-23T15:35:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-23 15:29:55.111741008 +0000 2024-03-23T15:35:22.337 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T15:35:22.403 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-23T15:35:22.403 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-23T15:35:22.403 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000261027 s, 2.0 MB/s 2024-03-23T15:35:22.404 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T15:35:22.462 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-23 15:35:08.271241483 +0000 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-23 15:29:55.590727782 +0000 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-23 15:29:55.590727782 +0000 2024-03-23T15:35:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-23 15:29:55.590727782 +0000 2024-03-23T15:35:22.519 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T15:35:22.585 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-23T15:35:22.585 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-23T15:35:22.585 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000228509 s, 2.2 MB/s 2024-03-23T15:35:22.586 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T15:35:22.644 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-03-23T15:35:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-23 15:35:08.271241483 +0000 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-23 15:29:55.930718394 +0000 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-23 15:29:55.930718394 +0000 2024-03-23T15:35:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-23 15:29:55.930718394 +0000 2024-03-23T15:35:22.700 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T15:35:22.766 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-23T15:35:22.766 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-23T15:35:22.766 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000173016 s, 3.0 MB/s 2024-03-23T15:35:22.767 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T15:35:22.822 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-03-23T15:35:22.823 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T15:35:22.838 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T15:35:22.839 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-03-23T15:35:22.894 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T15:35:22.894 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:22.894 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-23T15:35:22.894 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:22.894 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:22.895 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-23 15:34:58.801562147 +0000 2024-03-23T15:35:22.895 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-23 15:29:57.703348067 +0000 2024-03-23T15:35:22.895 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-23 15:29:57.703348067 +0000 2024-03-23T15:35:22.895 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-23 15:29:57.703348067 +0000 2024-03-23T15:35:22.895 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T15:35:22.960 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-23T15:35:22.961 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-23T15:35:22.961 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000271709 s, 1.9 MB/s 2024-03-23T15:35:22.962 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T15:35:23.017 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-23 15:34:58.802562127 +0000 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-23 15:29:58.039340936 +0000 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-23 15:29:58.039340936 +0000 2024-03-23T15:35:23.071 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-23 15:29:58.039340936 +0000 2024-03-23T15:35:23.072 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T15:35:23.137 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-23T15:35:23.137 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-23T15:35:23.138 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000188414 s, 2.7 MB/s 2024-03-23T15:35:23.139 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T15:35:23.194 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-03-23T15:35:23.250 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T15:35:23.250 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:23.250 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-23T15:35:23.251 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:23.251 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:23.251 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-23 15:34:58.802562127 +0000 2024-03-23T15:35:23.251 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-23 15:29:58.397333339 +0000 2024-03-23T15:35:23.251 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-23 15:29:58.397333339 +0000 2024-03-23T15:35:23.251 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-23 15:29:58.397333339 +0000 2024-03-23T15:35:23.251 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T15:35:23.316 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-23T15:35:23.317 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-23T15:35:23.317 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000176831 s, 2.9 MB/s 2024-03-23T15:35:23.318 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T15:35:23.374 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-03-23T15:35:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T15:35:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T15:35:23.430 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-23T15:35:23.430 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:35:23.430 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T15:35:23.430 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-03-23 15:34:58.802562127 +0000 2024-03-23T15:35:23.430 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-03-23 15:29:58.738326102 +0000 2024-03-23T15:35:23.430 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-03-23 15:29:58.738326102 +0000 2024-03-23T15:35:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-03-23 15:29:58.738326102 +0000 2024-03-23T15:35:23.430 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T15:35:23.497 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-03-23T15:35:23.497 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-03-23T15:35:23.497 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000186737 s, 2.7 MB/s 2024-03-23T15:35:23.498 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T15:35:23.556 INFO:tasks.cephadm:Deploying osd.0 on smithi076 with /dev/vg_nvme/lv_4... 2024-03-23T15:35:23.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T15:35:23.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:23.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:23.787 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:23.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T15:35:23.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:23.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:24.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:24 smithi076 ceph-mon[32176]: Deploying daemon mgr.z on smithi192 2024-03-23T15:35:24.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:24 smithi076 ceph-mon[32176]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:24.681 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:24 smithi192 ceph-mon[36653]: Deploying daemon mgr.z on smithi192 2024-03-23T15:35:24.681 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:24 smithi192 ceph-mon[36653]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:24 smithi183 ceph-mon[36296]: Deploying daemon mgr.z on smithi192 2024-03-23T15:35:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:24 smithi183 ceph-mon[36296]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:25.048 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 systemd[1]: Starting Ceph mgr.z for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:35:25.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 podman[38490]: 2024-03-23 15:35:25.164480646 +0000 UTC m=+0.017829519 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:35:25.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 podman[38490]: 2024-03-23 15:35:25.268222747 +0000 UTC m=+0.121571648 container create 5d993f699ea106b60f07b59ffeda4ba17734c817f87d9fdab101b6d63e22e6a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-23T15:35:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T15:35:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T15:35:25.573 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-03-23T15:35:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-03-23T15:35:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113417 s, 925 MB/s 2024-03-23T15:35:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-03-23T15:35:25.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 podman[38490]: 2024-03-23 15:35:25.487538463 +0000 UTC m=+0.340887323 container init 5d993f699ea106b60f07b59ffeda4ba17734c817f87d9fdab101b6d63e22e6a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True) 2024-03-23T15:35:25.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 podman[38490]: 2024-03-23 15:35:25.490268294 +0000 UTC m=+0.343617154 container start 5d993f699ea106b60f07b59ffeda4ba17734c817f87d9fdab101b6d63e22e6a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-23T15:35:25.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 ceph-mgr[38510]: -- 172.21.15.192:0/138334021 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5556c3a6a1a0 con 0x5556c3a40800 2024-03-23T15:35:25.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 bash[38490]: 5d993f699ea106b60f07b59ffeda4ba17734c817f87d9fdab101b6d63e22e6a4 2024-03-23T15:35:25.820 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:25 smithi192 systemd[1]: Started Ceph mgr.z for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:35:26.129 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:26.033+0000 7f567731f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:35:26.153 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_4 2024-03-23T15:35:26.389 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:35:26.428 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:26.126+0000 7f567731f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:26.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:26.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:26.747 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.747 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.747 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.747 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:26.747 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T15:35:26.747 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:26.747 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:26.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:26.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:26.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:26.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T15:35:26.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:26.770 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:27.069 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:26.765+0000 7f567731f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:35:27.069 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:26.857+0000 7f567731f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:35:27.069 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:26.998+0000 7f567731f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:35:27.432 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:27.092+0000 7f567731f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:35:27.432 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:27.175+0000 7f567731f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:35:27.521 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:27 smithi183 ceph-mon[36296]: Deploying daemon mgr.y on smithi183 2024-03-23T15:35:27.548 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:27 smithi076 ceph-mon[32176]: Deploying daemon mgr.y on smithi183 2024-03-23T15:35:27.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-mon[36653]: Deploying daemon mgr.y on smithi183 2024-03-23T15:35:27.736 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:27.440+0000 7f567731f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:35:27.737 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:27.549+0000 7f567731f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:35:27.737 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:27.634+0000 7f567731f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:35:27.737 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:27.733+0000 7f567731f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:35:27.772 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:27 smithi183 systemd[1]: Starting Ceph mgr.y for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:35:27.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.920+0000 7fbca4fe1700 1 -- 172.21.15.76:0/3024971445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102cc0 msgr2=0x7fbca0109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:27.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.920+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/3024971445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102cc0 0x7fbca0109570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc900099c0 tx=0x7fbc90009cd0 comp rx=0 tx=0).stop 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.920+0000 7fbca4fe1700 1 -- 172.21.15.76:0/3024971445 shutdown_connections 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.920+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/3024971445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102cc0 0x7fbca0109570 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.920+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/3024971445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbca0101120 0x7fbca0101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.920+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/3024971445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbca0102320 0x7fbca0102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.920+0000 7fbca4fe1700 1 -- 172.21.15.76:0/3024971445 >> 172.21.15.76:0/3024971445 conn(0x7fbca00fc6c0 msgr2=0x7fbca00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.921+0000 7fbca4fe1700 1 -- 172.21.15.76:0/3024971445 shutdown_connections 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.921+0000 7fbca4fe1700 1 -- 172.21.15.76:0/3024971445 wait complete. 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.921+0000 7fbca4fe1700 1 Processor -- start 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.921+0000 7fbca4fe1700 1 -- start start 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.921+0000 7fbca4fe1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbca0101120 0x7fbca010ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.921+0000 7fbca4fe1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102320 0x7fbca010a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.921+0000 7fbca4fe1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbca0102cc0 0x7fbca010a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:27.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbca4fe1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010ad40 con 0x7fbca0102320 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbca4fe1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010aeb0 con 0x7fbca0102cc0 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbca4fe1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a2180 con 0x7fbca0101120 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102320 0x7fbca010a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102320 0x7fbca010a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45598/0 (socket says 172.21.15.76:45598) 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 -- 172.21.15.76:0/2172907766 learned_addr learned my addr 172.21.15.76:0/2172907766 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9e59c700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbca0101120 0x7fbca010ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 -- 172.21.15.76:0/2172907766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbca0101120 msgr2=0x7fbca010ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbca0101120 0x7fbca010ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 -- 172.21.15.76:0/2172907766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbca0102cc0 msgr2=0x7fbca010a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbca0102cc0 0x7fbca010a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 -- 172.21.15.76:0/2172907766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc900095f0 con 0x7fbca0102320 2024-03-23T15:35:27.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc9dd9b700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102320 0x7fbca010a080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400ef10 tx=0x7fbc9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9400ca70 con 0x7fbca0102320 2024-03-23T15:35:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9400f040 con 0x7fbca0102320 2024-03-23T15:35:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca01a2400 con 0x7fbca0102320 2024-03-23T15:35:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc940146e0 con 0x7fbca0102320 2024-03-23T15:35:27.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.922+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a2900 con 0x7fbca0102320 2024-03-23T15:35:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.923+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 13) ==== 45984+0+0 (secure 0 0 0) 0x7fbc94007500 con 0x7fbca0102320 2024-03-23T15:35:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.923+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca0108410 con 0x7fbca0102320 2024-03-23T15:35:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.923+0000 7fbc8f7fe700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbc880391c0 0x7fbc8803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.924+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbc9404bf20 con 0x7fbca0102320 2024-03-23T15:35:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.924+0000 7fbc9e59c700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbc880391c0 0x7fbc8803b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:27.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.924+0000 7fbc9e59c700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbc880391c0 0x7fbc8803b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbca0102180 tx=0x7fbc9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:27.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:27.928+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fbc9400cbd0 con 0x7fbca0102320 2024-03-23T15:35:28.037 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:27 smithi183 podman[37670]: 2024-03-23 15:35:27.901492294 +0000 UTC m=+0.102739359 container create 31d95d411636bb14be63dd9bf412ee5cb2888401fb53d636e97c9d41cc93bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-23T15:35:28.038 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:27 smithi183 podman[37670]: 2024-03-23 15:35:27.816438138 +0000 UTC m=+0.017685197 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:35:28.067 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:27.917+0000 7f567731f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:35:28.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:28.131+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbca002dc30 con 0x7fbc880391c0 2024-03-23T15:35:28.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:28 smithi183 podman[37670]: 2024-03-23 15:35:28.120371213 +0000 UTC m=+0.321618265 container init 31d95d411636bb14be63dd9bf412ee5cb2888401fb53d636e97c9d41cc93bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-03-23T15:35:28.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:28 smithi183 podman[37670]: 2024-03-23 15:35:28.122966191 +0000 UTC m=+0.324213238 container start 31d95d411636bb14be63dd9bf412ee5cb2888401fb53d636e97c9d41cc93bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:35:28.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mgr[37689]: -- 172.21.15.183:0/3109804589 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557cd16801a0 con 0x557cd1656800 2024-03-23T15:35:28.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:28 smithi183 bash[37670]: 31d95d411636bb14be63dd9bf412ee5cb2888401fb53d636e97c9d41cc93bfcf 2024-03-23T15:35:28.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:28 smithi183 systemd[1]: Started Ceph mgr.y for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:35:28.691 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:28.691 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:35:28.692 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:35:28.692 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:28.692 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.692 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.692 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.692 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.764 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:28.764 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:35:28.764 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:35:28.764 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:28.764 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.765 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.765 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.765 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:28.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:35:28.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:35:28.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:28.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:28.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:29.133 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:28.688+0000 7f86f8a98200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:35:29.133 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:28 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:28.785+0000 7f86f8a98200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:35:29.287 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:28.939+0000 7f567731f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:35:29.287 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:29.081+0000 7f567731f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:35:29.481 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:29 smithi183 ceph-mon[36296]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:29.481 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:29.562 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:29 smithi076 ceph-mon[32176]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:29.562 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:29.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:29 smithi192 ceph-mon[36653]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:29.567 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:29.567 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:29.419+0000 7f567731f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:35:29.568 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:29.509+0000 7f567731f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:35:29.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:29.709+0000 7f567731f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:35:29.823 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:29.478+0000 7f86f8a98200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:35:29.823 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:29.570+0000 7f86f8a98200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:35:29.823 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:29.724+0000 7f86f8a98200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:35:29.823 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:29.820+0000 7f86f8a98200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:35:30.132 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:29.902+0000 7f86f8a98200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:35:30.188 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:29.900+0000 7f567731f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:35:30.438 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:30.185+0000 7f567731f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:35:30.439 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:30.169+0000 7f86f8a98200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:35:30.439 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:30.277+0000 7f86f8a98200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:35:30.439 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:30.362+0000 7f86f8a98200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:35:30.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:30.445+0000 7f567731f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:35:30.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:30.680+0000 7f567731f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:35:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:30.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:30.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:30.883 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:30.457+0000 7f86f8a98200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:35:30.883 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:30.634+0000 7f86f8a98200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:35:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:30.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:30.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:30.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T15:35:30.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T15:35:30.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:31.742 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:31.607+0000 7f86f8a98200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:35:31.742 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:31.739+0000 7f86f8a98200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:35:31.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T15:35:31.742 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: Reconfiguring daemon mgr.x on smithi076 2024-03-23T15:35:31.743 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:31.743 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1615808818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec188648-1f9e-47f7-a11c-31c3080d91dd"}]: dispatch 2024-03-23T15:35:31.743 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1615808818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec188648-1f9e-47f7-a11c-31c3080d91dd"}]': finished 2024-03-23T15:35:31.743 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T15:35:31.743 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:31.743 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:31 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2238503508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:35:31.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:31.477+0000 7f567731f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:35:31.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T15:35:31.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: Reconfiguring daemon mgr.x on smithi076 2024-03-23T15:35:31.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:31.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1615808818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec188648-1f9e-47f7-a11c-31c3080d91dd"}]: dispatch 2024-03-23T15:35:31.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1615808818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec188648-1f9e-47f7-a11c-31c3080d91dd"}]': finished 2024-03-23T15:35:31.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T15:35:31.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:31.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:31 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2238503508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: Reconfiguring mgr.x (unknown last config time)... 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: Reconfiguring daemon mgr.x on smithi076 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1615808818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec188648-1f9e-47f7-a11c-31c3080d91dd"}]: dispatch 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1615808818' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec188648-1f9e-47f7-a11c-31c3080d91dd"}]': finished 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: osdmap e5: 1 total, 0 up, 1 in 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:31.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:31 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2238503508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:35:32.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:32.044+0000 7f567731f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:35:32.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:32.142+0000 7f567731f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:35:32.362 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:32.075+0000 7f86f8a98200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:35:32.362 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:32.164+0000 7f86f8a98200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:35:32.362 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:32.359+0000 7f86f8a98200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:35:32.633 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:32.541+0000 7f86f8a98200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:35:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-mon[36296]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-mon[36296]: Standby manager daemon z started 2024-03-23T15:35:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T15:35:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:35:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T15:35:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:35:32.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:32.704+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 14) ==== 91342+0+0 (secure 0 0 0) 0x7fbc9400fd40 con 0x7fbca0102320 2024-03-23T15:35:32.819 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:35:32.357+0000 7f567731f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:35:32.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-mon[36653]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:32.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-mon[36653]: Standby manager daemon z started 2024-03-23T15:35:32.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T15:35:32.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:35:32.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T15:35:32.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:32 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:35:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:32 smithi076 ceph-mon[32176]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:32 smithi076 ceph-mon[32176]: Standby manager daemon z started 2024-03-23T15:35:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:32 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T15:35:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:32 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:35:32.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:32 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T15:35:32.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:32 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/3598151307' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:35:33.084 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:32.822+0000 7f86f8a98200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:35:33.084 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:33.081+0000 7f86f8a98200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:35:33.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:33.317+0000 7f86f8a98200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:35:34.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:33 smithi192 ceph-mon[36653]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-23T15:35:34.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:33 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T15:35:34.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:33 smithi192 ceph-mon[36653]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:34.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:33 smithi076 ceph-mon[32176]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-23T15:35:34.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:33 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T15:35:34.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:33 smithi076 ceph-mon[32176]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:34.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:33 smithi183 ceph-mon[36296]: mgrmap e14: x(active, since 77s), standbys: z 2024-03-23T15:35:34.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:33 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T15:35:34.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:33 smithi183 ceph-mon[36296]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:34.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:34.120+0000 7f86f8a98200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:35:35.033 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:34.722+0000 7f86f8a98200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:35:35.033 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:34.818+0000 7f86f8a98200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:35:35.033 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:35:35 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:35:35.030+0000 7f86f8a98200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:35:35.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:35.101+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mon.0 v2:172.21.15.76:3300/0 8 ==== mgrmap(e 15) ==== 136700+0+0 (secure 0 0 0) 0x7fbc9401cc60 con 0x7fbca0102320 2024-03-23T15:35:35.243 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:35 smithi076 ceph-mon[32176]: Standby manager daemon y started 2024-03-23T15:35:35.243 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:35 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T15:35:35.243 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:35 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:35:35.243 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:35 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T15:35:35.243 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:35 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:35:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:35 smithi183 ceph-mon[36296]: Standby manager daemon y started 2024-03-23T15:35:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:35 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T15:35:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:35 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:35:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:35 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T15:35:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:35 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:35:35.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:35 smithi192 ceph-mon[36653]: Standby manager daemon y started 2024-03-23T15:35:35.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:35 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T15:35:35.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:35 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:35:35.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:35 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T15:35:35.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:35 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/216591027' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:35:36.314 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:36 smithi076 ceph-mon[32176]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-23T15:35:36.315 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:36 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T15:35:36.315 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:36 smithi076 ceph-mon[32176]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:36.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:36 smithi183 ceph-mon[36296]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-23T15:35:36.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:36 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T15:35:36.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:36 smithi183 ceph-mon[36296]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:36.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:36 smithi192 ceph-mon[36653]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-23T15:35:36.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:36 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T15:35:36.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:36 smithi192 ceph-mon[36653]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:37.146 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.146 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.146 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:37.146 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:37.146 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:37.146 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:37.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:37.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:37.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:37.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:38.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:38 smithi183 ceph-mon[36296]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:38.413 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:38 smithi076 ceph-mon[32176]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:38 smithi192 ceph-mon[36653]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:39.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:39 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T15:35:39.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:39 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:39.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:39 smithi192 ceph-mon[36653]: Deploying daemon osd.0 on smithi076 2024-03-23T15:35:39.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T15:35:39.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:39.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:39 smithi076 ceph-mon[32176]: Deploying daemon osd.0 on smithi076 2024-03-23T15:35:39.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:39 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T15:35:39.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:39 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:39.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:39 smithi183 ceph-mon[36296]: Deploying daemon osd.0 on smithi076 2024-03-23T15:35:40.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:40 smithi076 ceph-mon[32176]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:40.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:40 smithi192 ceph-mon[36653]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:40.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:40 smithi183 ceph-mon[36296]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:42.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:42 smithi192 ceph-mon[36653]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:42 smithi076 ceph-mon[32176]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:42 smithi183 ceph-mon[36296]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:44.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:44 smithi076 ceph-mon[32176]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:44.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:44.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:44.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:44 smithi192 ceph-mon[36653]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:44 smithi183 ceph-mon[36296]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:46 smithi192 ceph-mon[36653]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:46 smithi076 ceph-mon[32176]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:46 smithi183 ceph-mon[36296]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:47.192 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 0 on host 'smithi076' 2024-03-23T15:35:47.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.189+0000 7fbc8f7fe700 1 -- 172.21.15.76:0/2172907766 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbca002dc30 con 0x7fbc880391c0 2024-03-23T15:35:47.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbc880391c0 msgr2=0x7fbc8803b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:47.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbc880391c0 0x7fbc8803b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbca0102180 tx=0x7fbc9800b410 comp rx=0 tx=0).stop 2024-03-23T15:35:47.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102320 msgr2=0x7fbca010a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:47.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102320 0x7fbca010a080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400ef10 tx=0x7fbc9400c3a0 comp rx=0 tx=0).stop 2024-03-23T15:35:47.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 shutdown_connections 2024-03-23T15:35:47.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbc880391c0 0x7fbc8803b680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbca0102cc0 0x7fbca010a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbca0101120 0x7fbca010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 --2- 172.21.15.76:0/2172907766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbca0102320 0x7fbca010a080 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 >> 172.21.15.76:0/2172907766 conn(0x7fbca00fc6c0 msgr2=0x7fbca0105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 shutdown_connections 2024-03-23T15:35:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:35:47.191+0000 7fbca4fe1700 1 -- 172.21.15.76:0/2172907766 wait complete. 2024-03-23T15:35:47.910 DEBUG:teuthology.orchestra.run.smithi076:osd.0> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.0.service 2024-03-23T15:35:47.913 INFO:tasks.cephadm:Deploying osd.1 on smithi183 with /dev/vg_nvme/lv_4... 2024-03-23T15:35:47.913 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T15:35:48.156 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:35:48.278 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:48 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.278 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:48 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.278 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:48 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.278 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:48 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.278 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:48 smithi183 ceph-mon[36296]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:48.278 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:48 smithi183 ceph-mon[36296]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T15:35:48.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:48 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:48 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:48 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:48 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.423 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:48 smithi076 ceph-mon[32176]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:48.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:48 smithi076 ceph-mon[32176]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T15:35:48.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:48 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:48 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:48 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:48 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:48.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:48 smithi192 ceph-mon[36653]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:48.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:48 smithi192 ceph-mon[36653]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T15:35:49.482 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:49 smithi183 ceph-mon[36296]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T15:35:49.482 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:49 smithi183 ceph-mon[36296]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T15:35:49.482 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:49 smithi183 ceph-mon[36296]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:35:49.482 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:49 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:49.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:49 smithi192 ceph-mon[36653]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T15:35:49.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:49 smithi192 ceph-mon[36653]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T15:35:49.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:49 smithi192 ceph-mon[36653]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:35:49.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:49 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:49.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:49 smithi076 ceph-mon[32176]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T15:35:49.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:49 smithi076 ceph-mon[32176]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T15:35:49.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:49 smithi076 ceph-mon[32176]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:35:49.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:49 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:49.576 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:35:49 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0[44534]: 2024-03-23T15:35:49.307+0000 7fee05a51700 -1 osd.0 0 waiting for initial osdmap 2024-03-23T15:35:49.576 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:35:49 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0[44534]: 2024-03-23T15:35:49.309+0000 7fee01844700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:35:50.026 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T15:35:50.026 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T15:35:50.026 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-23T15:35:50.026 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-23T15:35:50.026 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132896 s, 789 MB/s 2024-03-23T15:35:50.026 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-23T15:35:50.411 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:50 smithi076 ceph-mon[32176]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:35:50.412 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:50 smithi076 ceph-mon[32176]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T15:35:50.412 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:50.412 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:50.412 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:50 smithi076 ceph-mon[32176]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:50.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:50 smithi192 ceph-mon[36653]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:35:50.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:50 smithi192 ceph-mon[36653]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T15:35:50.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:50.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:50.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:50 smithi192 ceph-mon[36653]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:50.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:50 smithi183 ceph-mon[36296]: from='osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:35:50.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:50 smithi183 ceph-mon[36296]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T15:35:50.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:50.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:50.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:50 smithi183 ceph-mon[36296]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T15:35:50.662 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-03-23T15:35:50.906 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:35:51.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:35:51.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:35:51.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] boot 2024-03-23T15:35:51.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T15:35:51.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:51.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:35:51.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:35:51.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] boot 2024-03-23T15:35:51.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:51.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:35:51.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:35:51.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: osd.0 [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] boot 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: osdmap e8: 1 total, 1 up, 1 in 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:35:52.409 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:52 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi076 2024-03-23T15:35:52.409 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:52 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi076 to 17088M 2024-03-23T15:35:52.409 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:52 smithi183 ceph-mon[36296]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T15:35:52.409 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:52 smithi183 ceph-mon[36296]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:52.409 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:52.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:52.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:52.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 -- 172.21.15.183:0/440834027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac1008f0 msgr2=0x7f2eac100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:52.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 --2- 172.21.15.183:0/440834027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac1008f0 0x7f2eac100d10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00099c0 tx=0x7f2ea0009cd0 comp rx=0 tx=0).stop 2024-03-23T15:35:52.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 -- 172.21.15.183:0/440834027 shutdown_connections 2024-03-23T15:35:52.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 --2- 172.21.15.183:0/440834027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eac102490 0x7f2eac108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:52.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 --2- 172.21.15.183:0/440834027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac1008f0 0x7f2eac100d10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:52.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 --2- 172.21.15.183:0/440834027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2eac101af0 0x7f2eac101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:52.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 -- 172.21.15.183:0/440834027 >> 172.21.15.183:0/440834027 conn(0x7f2eac0fbe90 msgr2=0x7f2eac0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 -- 172.21.15.183:0/440834027 shutdown_connections 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.412+0000 7f2eb2202700 1 -- 172.21.15.183:0/440834027 wait complete. 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 Processor -- start 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 -- start start 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2eac1008f0 0x7f2eac104880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac101af0 0x7f2eac102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eac102490 0x7f2eac103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac103b90 con 0x7f2eac1008f0 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac103d00 con 0x7f2eac101af0 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eb2202700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1a2310 con 0x7f2eac102490 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eaaffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac101af0 0x7f2eac102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:52.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eaaffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac101af0 0x7f2eac102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:56298/0 (socket says 172.21.15.183:56298) 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eaaffd700 1 -- 172.21.15.183:0/392989067 learned_addr learned my addr 172.21.15.183:0/392989067 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eabfff700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eac102490 0x7f2eac103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.413+0000 7f2eab7fe700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2eac1008f0 0x7f2eac104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eaaffd700 1 -- 172.21.15.183:0/392989067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eac102490 msgr2=0x7f2eac103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eaaffd700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eac102490 0x7f2eac103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eaaffd700 1 -- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2eac1008f0 msgr2=0x7f2eac104880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eaaffd700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2eac1008f0 0x7f2eac104880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eaaffd700 1 -- 172.21.15.183:0/392989067 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea00095f0 con 0x7f2eac101af0 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eabfff700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eac102490 0x7f2eac103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eaaffd700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac101af0 0x7f2eac102ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b530 tx=0x7f2e9c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2ea8ff9700 1 -- 172.21.15.183:0/392989067 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c011830 con 0x7f2eac101af0 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac1a25f0 con 0x7f2eac101af0 2024-03-23T15:35:52.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2ea8ff9700 1 -- 172.21.15.183:0/392989067 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9c011e70 con 0x7f2eac101af0 2024-03-23T15:35:52.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2ea8ff9700 1 -- 172.21.15.183:0/392989067 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00f2e0 con 0x7f2eac101af0 2024-03-23T15:35:52.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.414+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac1a2af0 con 0x7f2eac101af0 2024-03-23T15:35:52.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.415+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eac068ec0 con 0x7f2eac101af0 2024-03-23T15:35:52.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.416+0000 7f2ea8ff9700 1 -- 172.21.15.183:0/392989067 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 136700+0+0 (secure 0 0 0) 0x7f2e9c0105f0 con 0x7f2eac101af0 2024-03-23T15:35:52.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.417+0000 7f2ea8ff9700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2e940a20c0 0x7f2e940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:35:52.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.417+0000 7f2ea8ff9700 1 -- 172.21.15.183:0/392989067 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f2e9c0cbce0 con 0x7f2eac101af0 2024-03-23T15:35:52.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.417+0000 7f2eab7fe700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2e940a20c0 0x7f2e940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:35:52.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.418+0000 7f2eab7fe700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2e940a20c0 0x7f2e940a4580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000b580 tx=0x7f2ea0009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:35:52.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.422+0000 7f2ea8ff9700 1 -- 172.21.15.183:0/392989067 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2e9c096720 con 0x7f2eac101af0 2024-03-23T15:35:52.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:52 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi076 2024-03-23T15:35:52.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:52 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi076 to 17088M 2024-03-23T15:35:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:52 smithi076 ceph-mon[32176]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T15:35:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:52 smithi076 ceph-mon[32176]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:52.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:52.623 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:35:52.621+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2eac107f40 con 0x7f2e940a20c0 2024-03-23T15:35:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:52 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi076 2024-03-23T15:35:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:52 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi076 to 17088M 2024-03-23T15:35:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:52 smithi192 ceph-mon[36653]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T15:35:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:52 smithi192 ceph-mon[36653]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:35:52.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:35:53.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:53 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:35:53.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:53 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:35:53.537 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:53 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:53.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:53 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:35:53.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:53 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:35:53.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:53 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:53 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:35:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:53 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:35:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:53 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:35:54.444 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:54 smithi183 ceph-mon[36296]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:54.444 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:54 smithi183 ceph-mon[36296]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:54.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:54 smithi192 ceph-mon[36653]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:54.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:54 smithi192 ceph-mon[36653]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:54 smithi076 ceph-mon[32176]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:35:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:54 smithi076 ceph-mon[32176]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:55.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:55 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/528746801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]: dispatch 2024-03-23T15:35:55.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:55 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]: dispatch 2024-03-23T15:35:55.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:55 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]': finished 2024-03-23T15:35:55.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:55 smithi183 ceph-mon[36296]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T15:35:55.590 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:55 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:35:55.720 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:55 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/528746801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]: dispatch 2024-03-23T15:35:55.720 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:55 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]: dispatch 2024-03-23T15:35:55.720 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:55 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]': finished 2024-03-23T15:35:55.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:55 smithi192 ceph-mon[36653]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T15:35:55.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:55 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:35:55.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:55 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/528746801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]: dispatch 2024-03-23T15:35:55.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:55 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]: dispatch 2024-03-23T15:35:55.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:55 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b709e5-af13-424e-9dc5-3b4c7e078d65"}]': finished 2024-03-23T15:35:55.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:55 smithi076 ceph-mon[32176]: osdmap e10: 2 total, 1 up, 2 in 2024-03-23T15:35:55.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:55 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:35:56.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:56 smithi183 ceph-mon[36296]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:56.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:56 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/1789801738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:35:56.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:56 smithi192 ceph-mon[36653]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:56.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:56 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/1789801738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:35:56.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:56 smithi076 ceph-mon[32176]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:56.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:56 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/1789801738' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:35:58.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:35:58 smithi192 ceph-mon[36653]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:35:58 smithi076 ceph-mon[32176]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:35:58.861 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:35:58 smithi183 ceph-mon[36296]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:00.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:00 smithi183 ceph-mon[36296]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:00.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:00 smithi192 ceph-mon[36653]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:00.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:00 smithi076 ceph-mon[32176]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:02 smithi192 ceph-mon[36653]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T15:36:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:02 smithi076 ceph-mon[32176]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T15:36:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:02 smithi183 ceph-mon[36296]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T15:36:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:03.772 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:03 smithi183 ceph-mon[36296]: Deploying daemon osd.1 on smithi183 2024-03-23T15:36:03.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:03 smithi192 ceph-mon[36653]: Deploying daemon osd.1 on smithi183 2024-03-23T15:36:03.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:03 smithi076 ceph-mon[32176]: Deploying daemon osd.1 on smithi183 2024-03-23T15:36:04.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:04 smithi183 ceph-mon[36296]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:04.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:04 smithi192 ceph-mon[36653]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:04.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:04 smithi076 ceph-mon[32176]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:06.710 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:06 smithi183 ceph-mon[36296]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:06.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:06 smithi192 ceph-mon[36653]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:06.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:06 smithi076 ceph-mon[32176]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:08.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:08 smithi183 ceph-mon[36296]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:08.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:08.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:08.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:08 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:08 smithi192 ceph-mon[36653]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:08 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:08 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:08 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:08.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:08 smithi076 ceph-mon[32176]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:08.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:08 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:08.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:08 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:08.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:08 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:10.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:10 smithi183 ceph-mon[36296]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:10.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:10.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:10.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:10 smithi192 ceph-mon[36653]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:10.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:10 smithi076 ceph-mon[32176]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.985+0000 7f2ea8ff9700 1 -- 172.21.15.183:0/392989067 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2eac107f40 con 0x7f2e940a20c0 2024-03-23T15:36:10.988 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 1 on host 'smithi183' 2024-03-23T15:36:10.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2e940a20c0 msgr2=0x7f2e940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:10.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2e940a20c0 0x7f2e940a4580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000b580 tx=0x7f2ea0009530 comp rx=0 tx=0).stop 2024-03-23T15:36:10.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac101af0 msgr2=0x7f2eac102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac101af0 0x7f2eac102ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b530 tx=0x7f2e9c00b8f0 comp rx=0 tx=0).stop 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 shutdown_connections 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2e940a20c0 0x7f2e940a4580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2eac102490 0x7f2eac103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2eac1008f0 0x7f2eac104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 --2- 172.21.15.183:0/392989067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eac101af0 0x7f2eac102ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 >> 172.21.15.183:0/392989067 conn(0x7f2eac0fbe90 msgr2=0x7f2eac0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 shutdown_connections 2024-03-23T15:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:36:10.988+0000 7f2eb2202700 1 -- 172.21.15.183:0/392989067 wait complete. 2024-03-23T15:36:11.634 DEBUG:teuthology.orchestra.run.smithi183:osd.1> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.1.service 2024-03-23T15:36:11.636 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2024-03-23T15:36:11.637 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T15:36:11.880 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:36:12.216 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:12.216 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:12.217 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:12.217 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:11 smithi192 ceph-mon[36653]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:12.217 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:11 smithi192 ceph-mon[36653]: from='osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T15:36:12.217 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:11 smithi192 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T15:36:12.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:12.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:12.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:12.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:11 smithi076 ceph-mon[32176]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:12.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:11 smithi076 ceph-mon[32176]: from='osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T15:36:12.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:11 smithi076 ceph-mon[32176]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T15:36:12.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:12.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:12.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:12.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:11 smithi183 ceph-mon[36296]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:12.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:11 smithi183 ceph-mon[36296]: from='osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T15:36:12.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:11 smithi183 ceph-mon[36296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T15:36:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:12 smithi183 ceph-mon[36296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T15:36:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:12 smithi183 ceph-mon[36296]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T15:36:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:12 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:12 smithi183 ceph-mon[36296]: from='osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:36:13.166 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:12 smithi183 ceph-mon[36296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:36:13.219 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:12 smithi192 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T15:36:13.220 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:12 smithi192 ceph-mon[36653]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T15:36:13.220 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:12 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:13.220 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:12 smithi192 ceph-mon[36653]: from='osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:36:13.220 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:12 smithi192 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:36:13.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:12 smithi076 ceph-mon[32176]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T15:36:13.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:12 smithi076 ceph-mon[32176]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T15:36:13.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:12 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:13.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:12 smithi076 ceph-mon[32176]: from='osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:36:13.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:12 smithi076 ceph-mon[32176]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:36:13.633 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1[40908]: 2024-03-23T15:36:13.349+0000 7f55b7463700 -1 osd.1 0 waiting for initial osdmap 2024-03-23T15:36:13.633 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1[40908]: 2024-03-23T15:36:13.353+0000 7f55af24e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:36:14.016 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T15:36:14.016 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T15:36:14.016 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-23T15:36:14.016 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-23T15:36:14.016 INFO:teuthology.orchestra.run.smithi192.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011259 s, 931 MB/s 2024-03-23T15:36:14.016 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 2024-03-23T15:36:14.016 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-23T15:36:14.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-mon[36296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:36:14.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-mon[36296]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T15:36:14.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:14.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:14.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-mon[36296]: from='osd.1 ' entity='osd.1' 2024-03-23T15:36:14.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:13 smithi183 ceph-mon[36296]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:14.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:13 smithi192 ceph-mon[36653]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:36:14.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:13 smithi192 ceph-mon[36653]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T15:36:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:13 smithi192 ceph-mon[36653]: from='osd.1 ' entity='osd.1' 2024-03-23T15:36:14.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:13 smithi192 ceph-mon[36653]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:14.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:13 smithi076 ceph-mon[32176]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:36:14.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:13 smithi076 ceph-mon[32176]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T15:36:14.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:14.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:14.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:13 smithi076 ceph-mon[32176]: from='osd.1 ' entity='osd.1' 2024-03-23T15:36:14.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:13 smithi076 ceph-mon[32176]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-23T15:36:14.716 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-03-23T15:36:14.954 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] boot 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi183 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.262 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:36:15.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi183 to 17080M 2024-03-23T15:36:15.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:15.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:15.263 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] boot 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi183 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi183 to 17080M 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:15.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:36:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:36:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: osd.1 [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] boot 2024-03-23T15:36:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: osdmap e13: 2 total, 2 up, 2 in 2024-03-23T15:36:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi183 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi183 to 17080M 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:16.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1972393037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 msgr2=0x7f8bc0101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:16.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1972393037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc0101540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009990 tx=0x7f8bbc009ca0 comp rx=0 tx=0).stop 2024-03-23T15:36:16.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1972393037 shutdown_connections 2024-03-23T15:36:16.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1972393037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc0102cc0 0x7f8bc0109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:16.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1972393037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc0101540 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:16.443 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1972393037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc0102320 0x7f8bc0102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1972393037 >> 172.21.15.192:0/1972393037 conn(0x7f8bc00fc700 msgr2=0x7f8bc00feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1972393037 shutdown_connections 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1972393037 wait complete. 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 Processor -- start 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- start start 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc006b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc0102320 0x7f8bc019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc0102cc0 0x7f8bc01a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc01a1f70 con 0x7f8bc0102cc0 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc01a20e0 con 0x7f8bc0102320 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.438+0000 7f8bc77cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc01a22a0 con 0x7f8bc0101120 2024-03-23T15:36:16.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc006b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc006b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54764/0 (socket says 172.21.15.192:54764) 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 -- 172.21.15.192:0/1750600238 learned_addr learned my addr 172.21.15.192:0/1750600238 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc5d6c700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc0102cc0 0x7f8bc01a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc4d6a700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc0102320 0x7f8bc019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 -- 172.21.15.192:0/1750600238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc0102320 msgr2=0x7f8bc019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc0102320 0x7f8bc019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 -- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc0102cc0 msgr2=0x7f8bc01a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc0102cc0 0x7f8bc01a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 -- 172.21.15.192:0/1750600238 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bbc0095f0 con 0x7f8bc0101120 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc556b700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc006b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0095c0 tx=0x7f8bbc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bb67fc700 1 -- 172.21.15.192:0/1750600238 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc01d070 con 0x7f8bc0101120 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bb67fc700 1 -- 172.21.15.192:0/1750600238 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc0039d0 con 0x7f8bc0101120 2024-03-23T15:36:16.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc01a2520 con 0x7f8bc0101120 2024-03-23T15:36:16.446 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bb67fc700 1 -- 172.21.15.192:0/1750600238 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0056c0 con 0x7f8bc0101120 2024-03-23T15:36:16.446 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.439+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc01a2930 con 0x7f8bc0101120 2024-03-23T15:36:16.446 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.440+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc01085b0 con 0x7f8bc0101120 2024-03-23T15:36:16.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.441+0000 7f8bb67fc700 1 -- 172.21.15.192:0/1750600238 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) ==== 136700+0+0 (secure 0 0 0) 0x7f8bbc003b40 con 0x7f8bc0101120 2024-03-23T15:36:16.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.442+0000 7f8bb67fc700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8bac0a20c0 0x7f8bac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:16.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.442+0000 7f8bb67fc700 1 -- 172.21.15.192:0/1750600238 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f8bbc0cd780 con 0x7f8bc0101120 2024-03-23T15:36:16.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.442+0000 7f8bc4d6a700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8bac0a20c0 0x7f8bac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:16.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.442+0000 7f8bc4d6a700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8bac0a20c0 0x7f8bac0a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006ba20 tx=0x7f8bb000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:36:16.449 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.446+0000 7f8bb67fc700 1 -- 172.21.15.192:0/1750600238 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8bbc098030 con 0x7f8bc0101120 2024-03-23T15:36:16.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:16.650+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8bc002db90 con 0x7f8bac0a20c0 2024-03-23T15:36:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:16 smithi192 ceph-mon[36653]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:16 smithi192 ceph-mon[36653]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T15:36:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:16 smithi076 ceph-mon[32176]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:16 smithi076 ceph-mon[32176]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T15:36:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:16 smithi183 ceph-mon[36296]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:16 smithi183 ceph-mon[36296]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T15:36:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:17 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:36:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:17 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:36:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:17 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:36:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:36:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:17 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:36:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:17 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:36:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:17 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:18.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:18 smithi192 ceph-mon[36653]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:36:18.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:18 smithi192 ceph-mon[36653]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:18.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:18 smithi076 ceph-mon[32176]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:36:18.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:18 smithi076 ceph-mon[32176]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:18.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:18 smithi183 ceph-mon[36296]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:36:18.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:18 smithi183 ceph-mon[36296]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/2195198987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]: dispatch 2024-03-23T15:36:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:19 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]: dispatch 2024-03-23T15:36:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:19 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]': finished 2024-03-23T15:36:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:19 smithi192 ceph-mon[36653]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T15:36:19.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:19 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/2195198987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]: dispatch 2024-03-23T15:36:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:19 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]: dispatch 2024-03-23T15:36:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:19 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]': finished 2024-03-23T15:36:19.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:19 smithi076 ceph-mon[32176]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T15:36:19.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/2195198987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]: dispatch 2024-03-23T15:36:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:19 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]: dispatch 2024-03-23T15:36:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:19 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e53f5387-30ec-46d0-bd1e-0950c58b3101"}]': finished 2024-03-23T15:36:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:19 smithi183 ceph-mon[36296]: osdmap e15: 3 total, 2 up, 3 in 2024-03-23T15:36:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:19 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:20.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:20 smithi192 ceph-mon[36653]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:20.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:20 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/1332441428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:36:20.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:20 smithi076 ceph-mon[32176]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:20.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:20 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/1332441428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:36:20.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:20 smithi183 ceph-mon[36296]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:20.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:20 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/1332441428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:36:22.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:22 smithi192 ceph-mon[36653]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:22 smithi076 ceph-mon[32176]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:22 smithi183 ceph-mon[36296]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:24.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:24 smithi192 ceph-mon[36653]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:24.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:24 smithi076 ceph-mon[32176]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:24 smithi183 ceph-mon[36296]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:26.679 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:26 smithi192 ceph-mon[36653]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:26.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:26 smithi076 ceph-mon[32176]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:26.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:26 smithi183 ceph-mon[36296]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:27.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:27 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T15:36:27.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:27 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:27.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T15:36:27.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:27.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:27 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T15:36:27.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:27 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:28.655 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:28 smithi192 ceph-mon[36653]: Deploying daemon osd.2 on smithi192 2024-03-23T15:36:28.655 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:28 smithi192 ceph-mon[36653]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:28.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:28 smithi076 ceph-mon[32176]: Deploying daemon osd.2 on smithi192 2024-03-23T15:36:28.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:28 smithi076 ceph-mon[32176]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:28.868 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:28 smithi183 ceph-mon[36296]: Deploying daemon osd.2 on smithi192 2024-03-23T15:36:28.868 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:28 smithi183 ceph-mon[36296]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:30 smithi192 ceph-mon[36653]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:30.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:30 smithi076 ceph-mon[32176]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:30 smithi183 ceph-mon[36296]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:32.760 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:32 smithi192 ceph-mon[36653]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:32.760 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:32 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:32.760 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:32 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:32.760 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:32 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:32 smithi076 ceph-mon[32176]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:32 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:32 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:32.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:32 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:32.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:32 smithi183 ceph-mon[36296]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:32.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:32 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:32.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:32 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:32.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:32 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:34.748 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:34 smithi192 ceph-mon[36653]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:34.749 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.749 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.749 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:34.749 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:34.749 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:34 smithi076 ceph-mon[32176]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:34 smithi183 ceph-mon[36296]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:35.775 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.772+0000 7f8bb67fc700 1 -- 172.21.15.192:0/1750600238 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8bc002db90 con 0x7f8bac0a20c0 2024-03-23T15:36:35.775 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2024-03-23T15:36:35.777 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.774+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8bac0a20c0 msgr2=0x7f8bac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:35.777 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.774+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8bac0a20c0 0x7f8bac0a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006ba20 tx=0x7f8bb000a300 comp rx=0 tx=0).stop 2024-03-23T15:36:35.777 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.774+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 msgr2=0x7f8bc006b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:35.777 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.774+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc006b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0095c0 tx=0x7f8bbc00f690 comp rx=0 tx=0).stop 2024-03-23T15:36:35.777 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 shutdown_connections 2024-03-23T15:36:35.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8bac0a20c0 0x7f8bac0a4580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:35.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc0102cc0 0x7f8bc01a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:35.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc0101120 0x7f8bc006b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:35.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 --2- 172.21.15.192:0/1750600238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bc0102320 0x7f8bc019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:35.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 >> 172.21.15.192:0/1750600238 conn(0x7f8bc00fc700 msgr2=0x7f8bc01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:36:35.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 shutdown_connections 2024-03-23T15:36:35.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:36:35.775+0000 7f8bc77cf700 1 -- 172.21.15.192:0/1750600238 wait complete. 2024-03-23T15:36:36.473 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.2.service 2024-03-23T15:36:36.476 INFO:tasks.cephadm:Deploying osd.3 on smithi076 with /dev/vg_nvme/lv_3... 2024-03-23T15:36:36.476 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T15:36:36.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:36 smithi192 ceph-mon[36653]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:36.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:36 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:36.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:36 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:36.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:36 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:36.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:36 smithi192 ceph-mon[36653]: from='osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T15:36:36.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:36 smithi192 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T15:36:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:36:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:36 smithi076 ceph-mon[32176]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:36 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:36 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:36 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:36 smithi076 ceph-mon[32176]: from='osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T15:36:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:36 smithi076 ceph-mon[32176]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T15:36:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:36 smithi183 ceph-mon[36296]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:36 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:36 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:36 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:36 smithi183 ceph-mon[36296]: from='osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T15:36:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:36 smithi183 ceph-mon[36296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T15:36:38.058 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:37 smithi076 ceph-mon[32176]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T15:36:38.059 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:37 smithi076 ceph-mon[32176]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T15:36:38.059 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:38.059 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:37 smithi076 ceph-mon[32176]: from='osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:36:38.059 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:37 smithi076 ceph-mon[32176]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:36:38.059 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:37 smithi076 ceph-mon[32176]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:38.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T15:36:38.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-mon[36653]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T15:36:38.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:38.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-mon[36653]: from='osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:36:38.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:36:38.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-mon[36653]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:38.070 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2[41811]: 2024-03-23T15:36:37.744+0000 7f00a3f0f700 -1 osd.2 0 waiting for initial osdmap 2024-03-23T15:36:38.070 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2[41811]: 2024-03-23T15:36:37.747+0000 7f009fd02700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:36:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:37 smithi183 ceph-mon[36296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T15:36:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:37 smithi183 ceph-mon[36296]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T15:36:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:37 smithi183 ceph-mon[36296]: from='osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:36:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:37 smithi183 ceph-mon[36296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:36:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:37 smithi183 ceph-mon[36296]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T15:36:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T15:36:38.654 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-03-23T15:36:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-03-23T15:36:38.654 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114678 s, 914 MB/s 2024-03-23T15:36:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-03-23T15:36:38.976 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:38 smithi076 ceph-mon[32176]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:36:38.976 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:38 smithi076 ceph-mon[32176]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T15:36:38.976 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:38.976 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:39.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:38 smithi192 ceph-mon[36653]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:36:39.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:38 smithi192 ceph-mon[36653]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T15:36:39.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:39.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:39.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:38 smithi183 ceph-mon[36296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:36:39.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:38 smithi183 ceph-mon[36296]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T15:36:39.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:39.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:39.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_3 2024-03-23T15:36:39.523 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:36:39.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:39 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:36:39.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:39 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:36:39.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:39.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:39 smithi076 ceph-mon[32176]: osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] boot 2024-03-23T15:36:39.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:39 smithi076 ceph-mon[32176]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T15:36:39.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:40.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:39 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:36:40.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:39 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:36:40.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:39 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:40.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:39 smithi192 ceph-mon[36653]: osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] boot 2024-03-23T15:36:40.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:39 smithi192 ceph-mon[36653]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T15:36:40.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:39 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:39 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:36:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:39 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:36:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:39 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:39 smithi183 ceph-mon[36296]: osd.2 [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] boot 2024-03-23T15:36:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:39 smithi183 ceph-mon[36296]: osdmap e18: 3 total, 3 up, 3 in 2024-03-23T15:36:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:39 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:36:40.976 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:40.976 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi192 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi192 to 17080M 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:40.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:40.978 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:40.978 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:40 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:41.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi192 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi192 to 17080M 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:41.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:40 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-23T15:36:41.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:36:41.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi192 2024-03-23T15:36:41.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi192 to 17080M 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:36:41.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:40 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 -- 172.21.15.76:0/3943463855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd081055f0 msgr2=0x7fcd081079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3943463855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd081055f0 0x7fcd081079e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009990 tx=0x7fccf8009ca0 comp rx=0 tx=0).stop 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 -- 172.21.15.76:0/3943463855 shutdown_connections 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3943463855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd08107f20 0x7fcd0810a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3943463855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd08102cc0 0x7fcd081050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3943463855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd081055f0 0x7fcd081079e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 -- 172.21.15.76:0/3943463855 >> 172.21.15.76:0/3943463855 conn(0x7fcd080fc6d0 msgr2=0x7fcd080feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 -- 172.21.15.76:0/3943463855 shutdown_connections 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 -- 172.21.15.76:0/3943463855 wait complete. 2024-03-23T15:36:41.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.201+0000 7fcd0db92700 1 Processor -- start 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd0db92700 1 -- start start 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd0db92700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd08102cc0 0x7fcd08102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd0db92700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd081055f0 0x7fcd08100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd0db92700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd08107f20 0x7fcd081011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd0db92700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08101930 con 0x7fcd08102cc0 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd0db92700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08101aa0 con 0x7fcd08107f20 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd0db92700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081a2220 con 0x7fcd081055f0 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd08102cc0 0x7fcd08102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd08102cc0 0x7fcd08102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44634/0 (socket says 172.21.15.76:44634) 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 -- 172.21.15.76:0/3076282216 learned_addr learned my addr 172.21.15.76:0/3076282216 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:36:41.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd06ffd700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd081055f0 0x7fcd08100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd07fff700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd08107f20 0x7fcd081011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 -- 172.21.15.76:0/3076282216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd081055f0 msgr2=0x7fcd08100c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd081055f0 0x7fcd08100c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 -- 172.21.15.76:0/3076282216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd08107f20 msgr2=0x7fcd081011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd08107f20 0x7fcd081011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 -- 172.21.15.76:0/3076282216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf80095f0 con 0x7fcd08102cc0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd07fff700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd08107f20 0x7fcd081011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.202+0000 7fcd077fe700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd08102cc0 0x7fcd08102620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00d950 tx=0x7fccfc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.203+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc009940 con 0x7fcd08102cc0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.203+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd081a2500 con 0x7fcd08102cc0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.203+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc010460 con 0x7fcd08102cc0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.203+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00f620 con 0x7fcd08102cc0 2024-03-23T15:36:41.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.203+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd081a2a00 con 0x7fcd08102cc0 2024-03-23T15:36:41.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.204+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd080fe250 con 0x7fcd08102cc0 2024-03-23T15:36:41.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.205+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 15) ==== 136700+0+0 (secure 0 0 0) 0x7fccfc0105d0 con 0x7fcd08102cc0 2024-03-23T15:36:41.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.205+0000 7fcd04ff9700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a20c0 0x7fccf00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:36:41.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.205+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fccfc0cc680 con 0x7fcd08102cc0 2024-03-23T15:36:41.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.205+0000 7fcd06ffd700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a20c0 0x7fccf00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:36:41.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.206+0000 7fcd06ffd700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a20c0 0x7fccf00a4580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b5c0 tx=0x7fccf8003860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:36:41.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.209+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fccfc096b00 con 0x7fcd08102cc0 2024-03-23T15:36:41.326 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49180]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T15:36:41.326 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49180]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:41.326 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49180]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:41.326 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49180]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:41.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:41.422+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcd0802dc00 con 0x7fccf00a20c0 2024-03-23T15:36:41.764 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T15:36:41.764 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:41.764 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49203]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:41.764 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:36:41 smithi076 sudo[49203]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:41.900 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T15:36:41.900 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:41.900 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44623]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:41.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44623]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:36:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:42.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T15:36:42.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T15:36:42.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:42.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:42.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:36:42.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:36:42.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:36:42.030 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T15:36:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T15:36:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:36:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:36:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:36:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:42.383 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T15:36:42.383 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:42.383 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44626]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:42.383 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:36:41 smithi183 sudo[44626]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:42.426 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T15:36:42.427 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:42.427 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45592]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:42.427 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45592]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:42.819 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45595]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T15:36:42.819 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:42.819 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45595]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:42.820 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:36:42 smithi192 sudo[45595]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:42.957 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49347]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T15:36:42.958 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:42.958 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49347]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:42.958 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49347]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:42.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:36:42.971+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fccfc096ed0 con 0x7fcd08102cc0 2024-03-23T15:36:43.221 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:42 smithi192 ceph-mon[36653]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T15:36:43.221 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:42 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:43.221 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:42 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:43.221 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:42 smithi192 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:43.221 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T15:36:43.221 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:43.222 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:43.222 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45598]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:43.226 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49353]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T15:36:43.226 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49353]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:43.226 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49353]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:43.226 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 sudo[49353]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:43.226 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 ceph-mon[32176]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T15:36:43.226 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:43.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:43.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:42 smithi076 ceph-mon[32176]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:43.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:42 smithi183 ceph-mon[36296]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T15:36:43.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:42 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:43.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:42 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:43.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:42 smithi183 ceph-mon[36296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:43.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T15:36:43.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:43.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45601]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:43.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 sudo[45601]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:43.950 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44629]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T15:36:43.950 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:43.950 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44629]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:43.950 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44629]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:44.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:43 smithi192 ceph-mon[36653]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:44.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:44.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:43 smithi076 ceph-mon[32176]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44632]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T15:36:44.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44632]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T15:36:44.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44632]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T15:36:44.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 sudo[44632]: pam_unix(sudo:session): session closed for user root 2024-03-23T15:36:44.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:36:44.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:36:44.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:43 smithi183 ceph-mon[36296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T15:36:45.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:44 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3836673077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f911b5-5902-413d-9e35-98630a33251d"}]: dispatch 2024-03-23T15:36:45.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:44 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3836673077' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f911b5-5902-413d-9e35-98630a33251d"}]': finished 2024-03-23T15:36:45.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:44 smithi192 ceph-mon[36653]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T15:36:45.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:36:45.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:44 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3750843395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:36:45.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3836673077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f911b5-5902-413d-9e35-98630a33251d"}]: dispatch 2024-03-23T15:36:45.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3836673077' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f911b5-5902-413d-9e35-98630a33251d"}]': finished 2024-03-23T15:36:45.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:44 smithi076 ceph-mon[32176]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T15:36:45.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:36:45.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3750843395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:36:45.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:44 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3836673077' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70f911b5-5902-413d-9e35-98630a33251d"}]: dispatch 2024-03-23T15:36:45.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:44 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3836673077' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70f911b5-5902-413d-9e35-98630a33251d"}]': finished 2024-03-23T15:36:45.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:44 smithi183 ceph-mon[36296]: osdmap e22: 4 total, 3 up, 4 in 2024-03-23T15:36:45.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:36:45.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:44 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3750843395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:36:46.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:45 smithi192 ceph-mon[36653]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:46.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:45 smithi076 ceph-mon[32176]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:46.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:45 smithi183 ceph-mon[36296]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:48.492 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:48 smithi076 ceph-mon[32176]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:48.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:48 smithi192 ceph-mon[36653]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:48.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:48 smithi183 ceph-mon[36296]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:50.597 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:50 smithi076 ceph-mon[32176]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:50 smithi192 ceph-mon[36653]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:50 smithi183 ceph-mon[36296]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:51.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T15:36:51.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:51 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:51.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T15:36:51.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:51 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T15:36:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:51 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:36:52.700 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:52 smithi076 ceph-mon[32176]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:52.700 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:52 smithi076 ceph-mon[32176]: Deploying daemon osd.3 on smithi076 2024-03-23T15:36:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:52 smithi192 ceph-mon[36653]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:52 smithi192 ceph-mon[36653]: Deploying daemon osd.3 on smithi076 2024-03-23T15:36:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:52 smithi183 ceph-mon[36296]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:52 smithi183 ceph-mon[36296]: Deploying daemon osd.3 on smithi076 2024-03-23T15:36:54.697 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:54 smithi076 ceph-mon[32176]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:54.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:54 smithi192 ceph-mon[36653]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:54.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:54 smithi183 ceph-mon[36296]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:56.758 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:56 smithi076 ceph-mon[32176]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:56.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:56 smithi192 ceph-mon[36653]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:56.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:56 smithi183 ceph-mon[36296]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:58.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:57 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:58.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:57 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:58.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:57 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:58.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:36:57 smithi192 ceph-mon[36653]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:58.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:57 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:58.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:57 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:58.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:57 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:58.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:36:57 smithi076 ceph-mon[32176]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:36:58.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:57 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:58.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:57 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:36:58.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:57 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:36:58.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:36:57 smithi183 ceph-mon[36296]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:00.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.577+0000 7fcd04ff9700 1 -- 172.21.15.76:0/3076282216 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcd0802dc00 con 0x7fccf00a20c0 2024-03-23T15:37:00.581 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 3 on host 'smithi076' 2024-03-23T15:37:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a20c0 msgr2=0x7fccf00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:00.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a20c0 0x7fccf00a4580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b5c0 tx=0x7fccf8003860 comp rx=0 tx=0).stop 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd08102cc0 msgr2=0x7fcd08102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd08102cc0 0x7fcd08102620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00d950 tx=0x7fccfc00dc60 comp rx=0 tx=0).stop 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 shutdown_connections 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a20c0 0x7fccf00a4580 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd08107f20 0x7fcd081011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd08102cc0 0x7fcd08102620 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 --2- 172.21.15.76:0/3076282216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd081055f0 0x7fcd08100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.579+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 >> 172.21.15.76:0/3076282216 conn(0x7fcd080fc6d0 msgr2=0x7fcd08103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.580+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 shutdown_connections 2024-03-23T15:37:00.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:00.580+0000 7fcd0db92700 1 -- 172.21.15.76:0/3076282216 wait complete. 2024-03-23T15:37:00.793 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:00 smithi076 ceph-mon[32176]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:00.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:00 smithi192 ceph-mon[36653]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:00 smithi183 ceph-mon[36296]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:01.268 DEBUG:teuthology.orchestra.run.smithi076:osd.3> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.3.service 2024-03-23T15:37:01.270 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_3... 2024-03-23T15:37:01.270 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T15:37:01.513 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:37:01.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:01 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:01 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:01 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:01 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:01 smithi183 ceph-mon[36296]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T15:37:01.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:01 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:01 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.805 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:01 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:01 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.806 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:01 smithi076 ceph-mon[32176]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T15:37:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:01 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:01 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:01 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:01 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:01.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:01 smithi192 ceph-mon[36653]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T15:37:02.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:02 smithi076 ceph-mon[32176]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:02.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:02 smithi076 ceph-mon[32176]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T15:37:02.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:02 smithi076 ceph-mon[32176]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T15:37:02.750 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:02 smithi076 ceph-mon[32176]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:37:02.751 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:02 smithi192 ceph-mon[36653]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:02 smithi192 ceph-mon[36653]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T15:37:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:02 smithi192 ceph-mon[36653]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T15:37:02.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:02 smithi192 ceph-mon[36653]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:37:02.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:02.833 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:02 smithi183 ceph-mon[36296]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:02.833 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:02 smithi183 ceph-mon[36296]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T15:37:02.833 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:02 smithi183 ceph-mon[36296]: osdmap e23: 4 total, 3 up, 4 in 2024-03-23T15:37:02.833 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:02 smithi183 ceph-mon[36296]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:37:02.833 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:03.386 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T15:37:03.386 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T15:37:03.387 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-23T15:37:03.387 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-23T15:37:03.387 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011273 s, 930 MB/s 2024-03-23T15:37:03.387 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-23T15:37:03.569 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3[51477]: 2024-03-23T15:37:03.077+0000 7f2f785d3700 -1 osd.3 0 waiting for initial osdmap 2024-03-23T15:37:03.569 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3[51477]: 2024-03-23T15:37:03.083+0000 7f2f703be700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:37:03.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] boot 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T15:37:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:04.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:37:04.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T15:37:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' 2024-03-23T15:37:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] boot 2024-03-23T15:37:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T15:37:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:04.077 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: osdmap e24: 4 total, 3 up, 4 in 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: from='osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591]' entity='osd.3' 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: osd.3 [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] boot 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: osdmap e25: 4 total, 4 up, 4 in 2024-03-23T15:37:04.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:37:04.107 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-03-23T15:37:04.350 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:37:04.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:37:04.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:37:04.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi076 2024-03-23T15:37:04.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:04.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi076 to 8544M 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:04.806 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:04 smithi183 ceph-mon[36296]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T15:37:05.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi076 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi076 to 8544M 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:05.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:05.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:05.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:04 smithi192 ceph-mon[36653]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T15:37:05.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:37:05.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi076 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi076 to 8544M 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:05.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:05.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:05.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:04 smithi076 ceph-mon[32176]: osdmap e26: 4 total, 4 up, 4 in 2024-03-23T15:37:05.723 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:05 smithi183 ceph-mon[36296]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:05.723 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:05 smithi183 ceph-mon[36296]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T15:37:05.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/485934318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8102230 msgr2=0x7fa7e8102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/485934318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8102230 0x7fa7e8102650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400b0d0 tx=0x7fa7e400b3e0 comp rx=0 tx=0).stop 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/485934318 shutdown_connections 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/485934318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7e8103dd0 0x7fa7e810a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/485934318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8102230 0x7fa7e8102650 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/485934318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e8103430 0x7fa7e8103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/485934318 >> 172.21.15.183:0/485934318 conn(0x7fa7e80fd810 msgr2=0x7fa7e80ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/485934318 shutdown_connections 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/485934318 wait complete. 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 Processor -- start 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.964+0000 7fa7ef1ae700 1 -- start start 2024-03-23T15:37:05.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ef1ae700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7e8102230 0x7fa7e8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ef1ae700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e8103430 0x7fa7e8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ef1ae700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8103dd0 0x7fa7e819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ef1ae700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e819ff90 con 0x7fa7e8102230 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ef1ae700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81a00d0 con 0x7fa7e8103dd0 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ef1ae700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81a0260 con 0x7fa7e8103430 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8103dd0 0x7fa7e819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8103dd0 0x7fa7e819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:60358/0 (socket says 172.21.15.183:60358) 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 -- 172.21.15.183:0/252470708 learned_addr learned my addr 172.21.15.183:0/252470708 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ecf4a700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7e8102230 0x7fa7e8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7dffff700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e8103430 0x7fa7e8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:05.970 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 -- 172.21.15.183:0/252470708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e8103430 msgr2=0x7fa7e8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e8103430 0x7fa7e8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 -- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7e8102230 msgr2=0x7fa7e8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7e8102230 0x7fa7e8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 -- 172.21.15.183:0/252470708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e4009d00 con 0x7fa7e8103dd0 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ed74b700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8103dd0 0x7fa7e819f9c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000d980 tx=0x7fa7e000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ddffb700 1 -- 172.21.15.183:0/252470708 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0004220 con 0x7fa7e8103dd0 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e81a0540 con 0x7fa7e8103dd0 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.965+0000 7fa7ddffb700 1 -- 172.21.15.183:0/252470708 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e00075e0 con 0x7fa7e8103dd0 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.966+0000 7fa7ddffb700 1 -- 172.21.15.183:0/252470708 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00056f0 con 0x7fa7e8103dd0 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.966+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81a0a40 con 0x7fa7e8103dd0 2024-03-23T15:37:05.971 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.967+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e81096c0 con 0x7fa7e8103dd0 2024-03-23T15:37:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.968+0000 7fa7ddffb700 1 -- 172.21.15.183:0/252470708 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fa7e0004750 con 0x7fa7e8103dd0 2024-03-23T15:37:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.968+0000 7fa7ddffb700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7d00a21d0 0x7fa7d00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.969+0000 7fa7ddffb700 1 -- 172.21.15.183:0/252470708 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fa7e00cc820 con 0x7fa7e8103dd0 2024-03-23T15:37:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.969+0000 7fa7ecf4a700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7d00a21d0 0x7fa7d00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:05.972 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.969+0000 7fa7ecf4a700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7d00a21d0 0x7fa7d00a4690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4007b60 tx=0x7fa7e400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:37:05.975 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:05.973+0000 7fa7ddffb700 1 -- 172.21.15.183:0/252470708 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa7e00969f0 con 0x7fa7e8103dd0 2024-03-23T15:37:06.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:05 smithi192 ceph-mon[36653]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:06.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:05 smithi192 ceph-mon[36653]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T15:37:06.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:05 smithi076 ceph-mon[32176]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:06.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:05 smithi076 ceph-mon[32176]: osdmap e27: 4 total, 4 up, 4 in 2024-03-23T15:37:06.176 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:06.174+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa7e802dc00 con 0x7fa7d00a21d0 2024-03-23T15:37:06.915 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:06 smithi183 ceph-mon[36296]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:06.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:06 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:06.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:06 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:06.916 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:06 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:07.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:06 smithi192 ceph-mon[36653]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:07.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:06 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:07.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:06 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:07.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:06 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:07.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:06 smithi076 ceph-mon[32176]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:07.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:06 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:07.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:06 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:07.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:06 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:07.944 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:07 smithi183 ceph-mon[36296]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T15:37:07.944 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:07 smithi183 ceph-mon[36296]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:08.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:07 smithi192 ceph-mon[36653]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T15:37:08.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:07 smithi192 ceph-mon[36653]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:08.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:07 smithi076 ceph-mon[32176]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T15:37:08.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:07 smithi076 ceph-mon[32176]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:08.977 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:08 smithi076 ceph-mon[32176]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T15:37:09.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:08 smithi192 ceph-mon[36653]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T15:37:09.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:08 smithi183 ceph-mon[36296]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T15:37:10.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/3793277928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]: dispatch 2024-03-23T15:37:10.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:09 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]: dispatch 2024-03-23T15:37:10.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:09 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]': finished 2024-03-23T15:37:10.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:09 smithi192 ceph-mon[36653]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T15:37:10.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:09 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:10.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:09 smithi192 ceph-mon[36653]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:10.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/3821606530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:37:10.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/3793277928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]: dispatch 2024-03-23T15:37:10.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:09 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]: dispatch 2024-03-23T15:37:10.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:09 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]': finished 2024-03-23T15:37:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:09 smithi076 ceph-mon[32176]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T15:37:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:09 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:09 smithi076 ceph-mon[32176]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/3821606530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:37:10.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/3793277928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]: dispatch 2024-03-23T15:37:10.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:09 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]: dispatch 2024-03-23T15:37:10.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:09 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c4ca544-320c-4f9f-ba0c-54737df4a1a5"}]': finished 2024-03-23T15:37:10.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:09 smithi183 ceph-mon[36296]: osdmap e30: 5 total, 4 up, 5 in 2024-03-23T15:37:10.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:09 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:10.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:09 smithi183 ceph-mon[36296]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:10.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/3821606530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:37:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:12 smithi192 ceph-mon[36653]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-23T15:37:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:12 smithi076 ceph-mon[32176]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-23T15:37:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:12 smithi183 ceph-mon[36296]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-23T15:37:14.783 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:14 smithi183 ceph-mon[36296]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-23T15:37:14.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:14 smithi192 ceph-mon[36653]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-23T15:37:14.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:14 smithi076 ceph-mon[32176]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-23T15:37:16.783 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:16 smithi183 ceph-mon[36296]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T15:37:16.784 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T15:37:16.784 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:16 smithi192 ceph-mon[36653]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T15:37:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T15:37:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:16 smithi076 ceph-mon[32176]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T15:37:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T15:37:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:17.730 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:17 smithi183 ceph-mon[36296]: Deploying daemon osd.4 on smithi183 2024-03-23T15:37:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:17 smithi192 ceph-mon[36653]: Deploying daemon osd.4 on smithi183 2024-03-23T15:37:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:17 smithi076 ceph-mon[32176]: Deploying daemon osd.4 on smithi183 2024-03-23T15:37:18.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:18 smithi192 ceph-mon[36653]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-23T15:37:18.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:18 smithi076 ceph-mon[32176]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-23T15:37:18.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:18 smithi183 ceph-mon[36296]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-23T15:37:20.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:20 smithi076 ceph-mon[32176]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-23T15:37:20.769 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:20 smithi183 ceph-mon[36296]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-23T15:37:20.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:20 smithi192 ceph-mon[36653]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-23T15:37:22.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:22 smithi183 ceph-mon[36296]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T15:37:22.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:22 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:22.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:22 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:22.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:22 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:22.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:22 smithi192 ceph-mon[36653]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T15:37:22.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:22 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:22.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:22 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:22.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:22 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:22 smithi076 ceph-mon[32176]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T15:37:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:22 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:22 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:22.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:22 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:24.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:24 smithi183 ceph-mon[36296]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:24.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:24 smithi192 ceph-mon[36653]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:24.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:24 smithi076 ceph-mon[32176]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:25.094 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.092+0000 7fa7ddffb700 1 -- 172.21.15.183:0/252470708 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7e802dc00 con 0x7fa7d00a21d0 2024-03-23T15:37:25.095 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.094+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7d00a21d0 msgr2=0x7fa7d00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.094+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7d00a21d0 0x7fa7d00a4690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4007b60 tx=0x7fa7e400b040 comp rx=0 tx=0).stop 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.094+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8103dd0 msgr2=0x7fa7e819f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.094+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8103dd0 0x7fa7e819f9c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000d980 tx=0x7fa7e000dc90 comp rx=0 tx=0).stop 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.094+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 shutdown_connections 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.095+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7d00a21d0 0x7fa7d00a4690 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.095+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e8103dd0 0x7fa7e819f9c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.095+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7e8102230 0x7fa7e8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.095+0000 7fa7ef1ae700 1 --2- 172.21.15.183:0/252470708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e8103430 0x7fa7e8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.095+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 >> 172.21.15.183:0/252470708 conn(0x7fa7e80fd810 msgr2=0x7fa7e8106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.095+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 shutdown_connections 2024-03-23T15:37:25.097 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:37:25.095+0000 7fa7ef1ae700 1 -- 172.21.15.183:0/252470708 wait complete. 2024-03-23T15:37:25.716 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.4.service 2024-03-23T15:37:25.719 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2024-03-23T15:37:25.719 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T15:37:25.962 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:37:25.994 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:25.994 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:25.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:25.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:25.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:25.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:25.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:25.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:25.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:25.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T15:37:25.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:25 smithi183 ceph-mon[36296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T15:37:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T15:37:26.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:25 smithi192 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T15:37:26.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T15:37:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:25 smithi076 ceph-mon[32176]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T15:37:27.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:27 smithi192 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T15:37:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:27 smithi192 ceph-mon[36653]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T15:37:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:27 smithi192 ceph-mon[36653]: from='osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:37:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:27 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:27 smithi192 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:37:27.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:27 smithi076 ceph-mon[32176]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T15:37:27.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:27 smithi076 ceph-mon[32176]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T15:37:27.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:27 smithi076 ceph-mon[32176]: from='osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:37:27.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:27.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:27 smithi076 ceph-mon[32176]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:37:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:27 smithi183 ceph-mon[36296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T15:37:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:27 smithi183 ceph-mon[36296]: osdmap e31: 5 total, 4 up, 5 in 2024-03-23T15:37:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:27 smithi183 ceph-mon[36296]: from='osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:37:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:27 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:27 smithi183 ceph-mon[36296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:37:27.384 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:37:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4[47459]: 2024-03-23T15:37:27.071+0000 7fd0e59b7700 -1 osd.4 0 waiting for initial osdmap 2024-03-23T15:37:27.384 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:37:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4[47459]: 2024-03-23T15:37:27.077+0000 7fd0e17aa700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:37:27.897 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T15:37:27.898 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T15:37:27.898 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-23T15:37:27.898 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-23T15:37:27.898 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112039 s, 936 MB/s 2024-03-23T15:37:27.898 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-23T15:37:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:28 smithi183 ceph-mon[36296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:37:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:28 smithi183 ceph-mon[36296]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T15:37:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:28 smithi183 ceph-mon[36296]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:28.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:28 smithi076 ceph-mon[32176]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:37:28.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:28 smithi076 ceph-mon[32176]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T15:37:28.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:28.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:28.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:28 smithi076 ceph-mon[32176]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:28.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:28 smithi192 ceph-mon[36653]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:37:28.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:28 smithi192 ceph-mon[36653]: osdmap e32: 5 total, 4 up, 5 in 2024-03-23T15:37:28.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:28.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:28.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:28 smithi192 ceph-mon[36653]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-23T15:37:28.596 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-03-23T15:37:28.843 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:37:29.076 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:37:29.076 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:37:29.210 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] boot 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi183 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi183 to 8540M 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:29.211 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:29.212 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:29 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] boot 2024-03-23T15:37:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T15:37:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi183 2024-03-23T15:37:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi183 to 8540M 2024-03-23T15:37:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:29 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: osd.4 [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] boot 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: osdmap e33: 5 total, 5 up, 5 in 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi183 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi183 to 8540M 2024-03-23T15:37:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:29.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:29.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:29.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:29 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:30.277 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:30 smithi192 ceph-mon[36653]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T15:37:30.277 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:30 smithi192 ceph-mon[36653]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:30.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:30 smithi183 ceph-mon[36296]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T15:37:30.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:30 smithi183 ceph-mon[36296]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:30.478 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 -- 172.21.15.192:0/2960283945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe81008b0 msgr2=0x7fbfe8100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:30.478 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/2960283945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe81008b0 0x7fbfe8100cd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009990 tx=0x7fbfd8009ca0 comp rx=0 tx=0).stop 2024-03-23T15:37:30.478 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 -- 172.21.15.192:0/2960283945 shutdown_connections 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/2960283945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe8102450 0x7fbfe8108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/2960283945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe81008b0 0x7fbfe8100cd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/2960283945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe8101ab0 0x7fbfe8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 -- 172.21.15.192:0/2960283945 >> 172.21.15.192:0/2960283945 conn(0x7fbfe80fbe70 msgr2=0x7fbfe80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 -- 172.21.15.192:0/2960283945 shutdown_connections 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.475+0000 7fbfed0b2700 1 -- 172.21.15.192:0/2960283945 wait complete. 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 Processor -- start 2024-03-23T15:37:30.479 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 -- start start 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe81008b0 0x7fbfe81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe8101ab0 0x7fbfe8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102450 0x7fbfe819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe819fd60 con 0x7fbfe8101ab0 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe819fea0 con 0x7fbfe81008b0 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.476+0000 7fbfed0b2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe81a0030 con 0x7fbfe8102450 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe759e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102450 0x7fbfe819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe6d9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe81008b0 0x7fbfe81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe759e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102450 0x7fbfe819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:35696/0 (socket says 172.21.15.192:35696) 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe759e700 1 -- 172.21.15.192:0/3305618095 learned_addr learned my addr 172.21.15.192:0/3305618095 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:37:30.481 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe659c700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe8101ab0 0x7fbfe8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe6d9d700 1 -- 172.21.15.192:0/3305618095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102450 msgr2=0x7fbfe819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe6d9d700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102450 0x7fbfe819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe6d9d700 1 -- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe8101ab0 msgr2=0x7fbfe8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe6d9d700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe8101ab0 0x7fbfe8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe6d9d700 1 -- 172.21.15.192:0/3305618095 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80095f0 con 0x7fbfe81008b0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfe6d9d700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe81008b0 0x7fbfe81981d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8000c00 tx=0x7fbfd8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfcffff700 1 -- 172.21.15.192:0/3305618095 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd801d070 con 0x7fbfe81008b0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe81a02b0 con 0x7fbfe81008b0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfcffff700 1 -- 172.21.15.192:0/3305618095 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfd80173f0 con 0x7fbfe81008b0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfcffff700 1 -- 172.21.15.192:0/3305618095 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800e640 con 0x7fbfe81008b0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.477+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe81a06c0 con 0x7fbfe81008b0 2024-03-23T15:37:30.482 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.478+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe8068ec0 con 0x7fbfe81008b0 2024-03-23T15:37:30.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.479+0000 7fbfcffff700 1 -- 172.21.15.192:0/3305618095 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fbfd8017560 con 0x7fbfe81008b0 2024-03-23T15:37:30.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.480+0000 7fbfcffff700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbfd00a2150 0x7fbfd00a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:30.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.480+0000 7fbfcffff700 1 -- 172.21.15.192:0/3305618095 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fbfd80cdff0 con 0x7fbfe81008b0 2024-03-23T15:37:30.483 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.480+0000 7fbfe659c700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbfd00a2150 0x7fbfd00a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:30.484 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.481+0000 7fbfe659c700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbfd00a2150 0x7fbfd00a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc005950 tx=0x7fbfdc00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:37:30.486 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.483+0000 7fbfcffff700 1 -- 172.21.15.192:0/3305618095 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fbfd8098060 con 0x7fbfe81008b0 2024-03-23T15:37:30.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:30 smithi076 ceph-mon[32176]: osdmap e34: 5 total, 5 up, 5 in 2024-03-23T15:37:30.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:30 smithi076 ceph-mon[32176]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:30.690 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:30.687+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbfe80fcbb0 con 0x7fbfd00a2150 2024-03-23T15:37:31.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:31 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:31.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:31 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:31.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:31 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:31.433 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:31 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:31.433 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:31 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:31.433 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:31 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:31.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:31.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:31.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:31 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:32.290 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:32 smithi192 ceph-mon[36653]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:32.290 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:32 smithi192 ceph-mon[36653]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:32.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:32 smithi183 ceph-mon[36296]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:32.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:32 smithi183 ceph-mon[36296]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:32.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:32 smithi076 ceph-mon[32176]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:32.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:32 smithi076 ceph-mon[32176]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:34 smithi192 ceph-mon[36653]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:34 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/2807822221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]: dispatch 2024-03-23T15:37:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:34 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]: dispatch 2024-03-23T15:37:34.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:34 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]': finished 2024-03-23T15:37:34.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:34 smithi192 ceph-mon[36653]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T15:37:34.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:34.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:34 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/2806868980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:37:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:34 smithi076 ceph-mon[32176]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:34 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/2807822221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]: dispatch 2024-03-23T15:37:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:34 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]: dispatch 2024-03-23T15:37:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:34 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]': finished 2024-03-23T15:37:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:34 smithi076 ceph-mon[32176]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T15:37:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:34 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/2806868980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:37:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:34 smithi183 ceph-mon[36296]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:34 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/2807822221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]: dispatch 2024-03-23T15:37:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:34 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]: dispatch 2024-03-23T15:37:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:34 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce29342e-6466-4e41-ba1e-2c8f10f4a559"}]': finished 2024-03-23T15:37:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:34 smithi183 ceph-mon[36296]: osdmap e35: 6 total, 5 up, 6 in 2024-03-23T15:37:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:34 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/2806868980' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:37:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:36 smithi192 ceph-mon[36653]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:36.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:36 smithi076 ceph-mon[32176]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:36 smithi183 ceph-mon[36296]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:38.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:38 smithi192 ceph-mon[36653]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:38.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:38 smithi076 ceph-mon[32176]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:38.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:38 smithi183 ceph-mon[36296]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:40.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:40 smithi192 ceph-mon[36653]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:40.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:40 smithi076 ceph-mon[32176]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:40.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:40 smithi183 ceph-mon[36296]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:41.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T15:37:41.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T15:37:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:41.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T15:37:41.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:42.786 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:42 smithi192 ceph-mon[36653]: Deploying daemon osd.5 on smithi192 2024-03-23T15:37:42.786 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:42 smithi192 ceph-mon[36653]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:42 smithi076 ceph-mon[32176]: Deploying daemon osd.5 on smithi192 2024-03-23T15:37:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:42 smithi076 ceph-mon[32176]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:42.882 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:42 smithi183 ceph-mon[36296]: Deploying daemon osd.5 on smithi192 2024-03-23T15:37:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:42 smithi183 ceph-mon[36296]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:44.765 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:44 smithi192 ceph-mon[36653]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:44 smithi076 ceph-mon[32176]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:44 smithi183 ceph-mon[36296]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:46 smithi192 ceph-mon[36653]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:46 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:46 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:46 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:46 smithi076 ceph-mon[32176]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:46 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:46 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:46 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:46 smithi183 ceph-mon[36296]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:46 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:46 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:46 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:48.761 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:48 smithi192 ceph-mon[36653]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:48.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:48 smithi076 ceph-mon[32176]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:48.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:48 smithi183 ceph-mon[36296]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:50.410 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.408+0000 7fbfcffff700 1 -- 172.21.15.192:0/3305618095 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbfe80fcbb0 con 0x7fbfd00a2150 2024-03-23T15:37:50.410 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-03-23T15:37:50.412 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbfd00a2150 msgr2=0x7fbfd00a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbfd00a2150 0x7fbfd00a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc005950 tx=0x7fbfdc00a300 comp rx=0 tx=0).stop 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe81008b0 msgr2=0x7fbfe81981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe81008b0 0x7fbfe81981d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8000c00 tx=0x7fbfd8004730 comp rx=0 tx=0).stop 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 shutdown_connections 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbfd00a2150 0x7fbfd00a4610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbfe8102450 0x7fbfe819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe81008b0 0x7fbfe81981d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 --2- 172.21.15.192:0/3305618095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfe8101ab0 0x7fbfe8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 >> 172.21.15.192:0/3305618095 conn(0x7fbfe80fbe70 msgr2=0x7fbfe80fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 shutdown_connections 2024-03-23T15:37:50.413 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:37:50.410+0000 7fbfed0b2700 1 -- 172.21.15.192:0/3305618095 wait complete. 2024-03-23T15:37:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:50.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T15:37:50.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T15:37:50.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:50.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T15:37:50.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T15:37:50.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T15:37:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T15:37:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:37:51.212 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.5.service 2024-03-23T15:37:51.214 INFO:tasks.cephadm:Deploying osd.6 on smithi076 with /dev/vg_nvme/lv_2... 2024-03-23T15:37:51.215 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T15:37:51.463 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:37:52.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:52 smithi183 ceph-mon[36296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T15:37:52.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:52 smithi183 ceph-mon[36296]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T15:37:52.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:52.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:52 smithi183 ceph-mon[36296]: from='osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:37:52.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:52 smithi183 ceph-mon[36296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:37:52.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:52 smithi183 ceph-mon[36296]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:52.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:52 smithi076 ceph-mon[32176]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T15:37:52.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:52 smithi076 ceph-mon[32176]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T15:37:52.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:52.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:52 smithi076 ceph-mon[32176]: from='osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:37:52.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:52 smithi076 ceph-mon[32176]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:37:52.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:52 smithi076 ceph-mon[32176]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:52.762 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T15:37:52.762 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-mon[36653]: osdmap e36: 6 total, 5 up, 6 in 2024-03-23T15:37:52.762 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:52.762 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-mon[36653]: from='osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:37:52.762 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:37:52.762 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-mon[36653]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:53.069 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:37:52.819+0000 7fbe19d4e700 -1 osd.5 0 waiting for initial osdmap 2024-03-23T15:37:53.069 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:37:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:37:52.825+0000 7fbe11b39700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:37:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:53 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:37:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:53 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:37:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:53 smithi183 ceph-mon[36296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:37:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:53 smithi183 ceph-mon[36296]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T15:37:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:53 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:53 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:53 smithi183 ceph-mon[36296]: from='osd.5 ' entity='osd.5' 2024-03-23T15:37:53.635 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T15:37:53.635 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T15:37:53.635 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-03-23T15:37:53.635 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-03-23T15:37:53.635 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114241 s, 918 MB/s 2024-03-23T15:37:53.635 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-03-23T15:37:53.635 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:53 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:37:53.636 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:53 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:37:53.636 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:53 smithi076 ceph-mon[32176]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:37:53.636 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:53 smithi076 ceph-mon[32176]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T15:37:53.636 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:53 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:53.636 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:53 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:53.636 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:53 smithi076 ceph-mon[32176]: from='osd.5 ' entity='osd.5' 2024-03-23T15:37:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:53 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:37:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:53 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:37:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:53 smithi192 ceph-mon[36653]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:37:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:53 smithi192 ceph-mon[36653]: osdmap e37: 6 total, 5 up, 6 in 2024-03-23T15:37:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:53 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:53 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:53 smithi192 ceph-mon[36653]: from='osd.5 ' entity='osd.5' 2024-03-23T15:37:54.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_2 2024-03-23T15:37:54.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:54.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] boot 2024-03-23T15:37:54.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T15:37:54.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:54.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:54 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:54.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] boot 2024-03-23T15:37:54.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T15:37:54.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:54.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:54 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.601 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: osd.5 [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] boot 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: osdmap e38: 6 total, 6 up, 6 in 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: osdmap e39: 6 total, 6 up, 6 in 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:54.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:37:54.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:54.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:54.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:37:54.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:54 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:37:55.810 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:55 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi192 2024-03-23T15:37:55.810 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:55 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi192 to 8540M 2024-03-23T15:37:55.810 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:55 smithi076 ceph-mon[32176]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T15:37:55.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:55 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi192 2024-03-23T15:37:55.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:55 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi192 to 8540M 2024-03-23T15:37:55.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:55 smithi192 ceph-mon[36653]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T15:37:55.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:55 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi192 2024-03-23T15:37:55.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:55 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi192 to 8540M 2024-03-23T15:37:55.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:55 smithi183 ceph-mon[36296]: osdmap e40: 6 total, 6 up, 6 in 2024-03-23T15:37:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 -- 172.21.15.76:0/1600686204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c100530 msgr2=0x7fdb2c100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 --2- 172.21.15.76:0/1600686204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c100530 0x7fdb2c100950 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b0a0 tx=0x7fdb2800b3b0 comp rx=0 tx=0).stop 2024-03-23T15:37:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 -- 172.21.15.76:0/1600686204 shutdown_connections 2024-03-23T15:37:56.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 --2- 172.21.15.76:0/1600686204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2c103eb0 0x7fdb2c10a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 --2- 172.21.15.76:0/1600686204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c100530 0x7fdb2c100950 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 --2- 172.21.15.76:0/1600686204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb2c103510 0x7fdb2c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 -- 172.21.15.76:0/1600686204 >> 172.21.15.76:0/1600686204 conn(0x7fdb2c0fbe90 msgr2=0x7fdb2c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 -- 172.21.15.76:0/1600686204 shutdown_connections 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 -- 172.21.15.76:0/1600686204 wait complete. 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.364+0000 7fdb33506700 1 Processor -- start 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb33506700 1 -- start start 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb33506700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb2c100530 0x7fdb2c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb33506700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c103510 0x7fdb2c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb33506700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2c103eb0 0x7fdb2c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb33506700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c19fec0 con 0x7fdb2c103510 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb33506700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c1a0000 con 0x7fdb2c103eb0 2024-03-23T15:37:56.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb33506700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c1a0190 con 0x7fdb2c100530 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c103510 0x7fdb2c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c103510 0x7fdb2c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34900/0 (socket says 172.21.15.76:34900) 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 -- 172.21.15.76:0/1325700572 learned_addr learned my addr 172.21.15.76:0/1325700572 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb31aa3700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2c103eb0 0x7fdb2c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb312a2700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb2c100530 0x7fdb2c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 -- 172.21.15.76:0/1325700572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb2c100530 msgr2=0x7fdb2c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb2c100530 0x7fdb2c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 -- 172.21.15.76:0/1325700572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2c103eb0 msgr2=0x7fdb2c19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2c103eb0 0x7fdb2c19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 -- 172.21.15.76:0/1325700572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb28009d00 con 0x7fdb2c103510 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb30aa1700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c103510 0x7fdb2c198870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00c950 tx=0x7fdb1c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb227fc700 1 -- 172.21.15.76:0/1325700572 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c007ac0 con 0x7fdb2c103510 2024-03-23T15:37:56.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.365+0000 7fdb227fc700 1 -- 172.21.15.76:0/1325700572 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c004d10 con 0x7fdb2c103510 2024-03-23T15:37:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.366+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2c1a0470 con 0x7fdb2c103510 2024-03-23T15:37:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.366+0000 7fdb227fc700 1 -- 172.21.15.76:0/1325700572 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c005610 con 0x7fdb2c103510 2024-03-23T15:37:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.366+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2c1a09c0 con 0x7fdb2c103510 2024-03-23T15:37:56.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.366+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2c1096d0 con 0x7fdb2c103510 2024-03-23T15:37:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.368+0000 7fdb227fc700 1 -- 172.21.15.76:0/1325700572 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fdb1c004750 con 0x7fdb2c103510 2024-03-23T15:37:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.368+0000 7fdb227fc700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdb180a21d0 0x7fdb180a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:37:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.368+0000 7fdb312a2700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdb180a21d0 0x7fdb180a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:37:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.369+0000 7fdb227fc700 1 -- 172.21.15.76:0/1325700572 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fdb1c0ccdb0 con 0x7fdb2c103510 2024-03-23T15:37:56.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.369+0000 7fdb312a2700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdb180a21d0 0x7fdb180a4690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b830 tx=0x7fdb280060c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:37:56.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.372+0000 7fdb227fc700 1 -- 172.21.15.76:0/1325700572 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fdb1c096ca0 con 0x7fdb2c103510 2024-03-23T15:37:56.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:37:56.578+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdb2c02dc00 con 0x7fdb180a21d0 2024-03-23T15:37:56.720 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:56 smithi192 ceph-mon[36653]: pgmap v140: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:37:56.784 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:56 smithi183 ceph-mon[36296]: pgmap v140: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:37:56.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:56 smithi076 ceph-mon[32176]: pgmap v140: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:37:57.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:57 smithi076 ceph-mon[32176]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-23T15:37:57.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:57 smithi076 ceph-mon[32176]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T15:37:57.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:57 smithi076 ceph-mon[32176]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:57.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:57 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:57.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:57 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:57.440 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:57 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:57.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:57 smithi192 ceph-mon[36653]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-23T15:37:57.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:57 smithi192 ceph-mon[36653]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T15:37:57.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:57 smithi192 ceph-mon[36653]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:57.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:57 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:57.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:57 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:57.734 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:57 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:57 smithi183 ceph-mon[36296]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-23T15:37:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:57 smithi183 ceph-mon[36296]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T15:37:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:57 smithi183 ceph-mon[36296]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:37:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:57 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:37:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:57 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:37:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:57 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:37:58.473 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:58 smithi076 ceph-mon[32176]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T15:37:58.473 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:58 smithi076 ceph-mon[32176]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:37:58.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:58 smithi192 ceph-mon[36653]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T15:37:58.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:58 smithi192 ceph-mon[36653]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:37:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:58 smithi183 ceph-mon[36296]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T15:37:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:58 smithi183 ceph-mon[36296]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:37:59.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:59 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3616438449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3260ee11-29d2-493e-9bd1-156fd317542d"}]: dispatch 2024-03-23T15:37:59.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:59 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3616438449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3260ee11-29d2-493e-9bd1-156fd317542d"}]': finished 2024-03-23T15:37:59.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:59 smithi192 ceph-mon[36653]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T15:37:59.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:37:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:37:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:59 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3616438449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3260ee11-29d2-493e-9bd1-156fd317542d"}]: dispatch 2024-03-23T15:37:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:59 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3616438449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3260ee11-29d2-493e-9bd1-156fd317542d"}]': finished 2024-03-23T15:37:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:59 smithi076 ceph-mon[32176]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T15:37:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:37:59 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:37:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:59 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3616438449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3260ee11-29d2-493e-9bd1-156fd317542d"}]: dispatch 2024-03-23T15:37:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:59 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3616438449' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3260ee11-29d2-493e-9bd1-156fd317542d"}]': finished 2024-03-23T15:37:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:59 smithi183 ceph-mon[36296]: osdmap e43: 7 total, 6 up, 7 in 2024-03-23T15:37:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:37:59 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:00.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:00 smithi192 ceph-mon[36653]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:38:00.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:00 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/135547316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:00.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:00 smithi076 ceph-mon[32176]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:38:00.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:00 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/135547316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:00.882 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:00 smithi183 ceph-mon[36296]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-03-23T15:38:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:00 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/135547316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:01 smithi192 ceph-mon[36653]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-23T15:38:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:01 smithi192 ceph-mon[36653]: Cluster is now healthy 2024-03-23T15:38:01.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:01 smithi076 ceph-mon[32176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-23T15:38:01.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:01 smithi076 ceph-mon[32176]: Cluster is now healthy 2024-03-23T15:38:01.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:01 smithi183 ceph-mon[36296]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-23T15:38:01.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:01 smithi183 ceph-mon[36296]: Cluster is now healthy 2024-03-23T15:38:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:02 smithi192 ceph-mon[36653]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T15:38:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:02 smithi076 ceph-mon[32176]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T15:38:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:02 smithi183 ceph-mon[36296]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T15:38:04.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:04 smithi192 ceph-mon[36653]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-23T15:38:04.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:04 smithi076 ceph-mon[32176]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-23T15:38:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:04 smithi183 ceph-mon[36296]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-23T15:38:06.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:06 smithi192 ceph-mon[36653]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T15:38:06.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:06 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T15:38:06.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:06 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:06.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:06 smithi076 ceph-mon[32176]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T15:38:06.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:06 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T15:38:06.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:06 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:06.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:06 smithi183 ceph-mon[36296]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-23T15:38:06.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:06 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-23T15:38:06.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:06 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:07.810 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:07 smithi076 ceph-mon[32176]: Deploying daemon osd.6 on smithi076 2024-03-23T15:38:07.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:07 smithi192 ceph-mon[36653]: Deploying daemon osd.6 on smithi076 2024-03-23T15:38:07.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:07 smithi183 ceph-mon[36296]: Deploying daemon osd.6 on smithi076 2024-03-23T15:38:08.588 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:08 smithi076 ceph-mon[32176]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T15:38:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:08 smithi192 ceph-mon[36653]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T15:38:08.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:08 smithi183 ceph-mon[36296]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-23T15:38:10.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:10 smithi192 ceph-mon[36653]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-23T15:38:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:10 smithi076 ceph-mon[32176]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-23T15:38:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:10 smithi183 ceph-mon[36296]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-23T15:38:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:12 smithi192 ceph-mon[36653]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T15:38:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:12 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:12 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:12 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:12 smithi076 ceph-mon[32176]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T15:38:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:12 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:12 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:12 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:12 smithi183 ceph-mon[36296]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-23T15:38:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:12 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:12 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:12 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:14.751 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:14 smithi076 ceph-mon[32176]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:14.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:14 smithi192 ceph-mon[36653]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:14 smithi183 ceph-mon[36296]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:16.249 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.250 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.250 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:16.250 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:16.250 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.250 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:16 smithi076 ceph-mon[32176]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:16.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:16.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:16.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:16 smithi183 ceph-mon[36296]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:16.395 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 6 on host 'smithi076' 2024-03-23T15:38:16.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.393+0000 7fdb227fc700 1 -- 172.21.15.76:0/1325700572 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdb2c02dc00 con 0x7fdb180a21d0 2024-03-23T15:38:16.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdb180a21d0 msgr2=0x7fdb180a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:16.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdb180a21d0 0x7fdb180a4690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b830 tx=0x7fdb280060c0 comp rx=0 tx=0).stop 2024-03-23T15:38:16.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c103510 msgr2=0x7fdb2c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:16.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c103510 0x7fdb2c198870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00c950 tx=0x7fdb1c00cd10 comp rx=0 tx=0).stop 2024-03-23T15:38:16.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 shutdown_connections 2024-03-23T15:38:16.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fdb180a21d0 0x7fdb180a4690 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:16.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2c103eb0 0x7fdb2c19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:16.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb2c100530 0x7fdb2c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:16.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 --2- 172.21.15.76:0/1325700572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb2c103510 0x7fdb2c198870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:16.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 >> 172.21.15.76:0/1325700572 conn(0x7fdb2c0fbe90 msgr2=0x7fdb2c106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:38:16.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 shutdown_connections 2024-03-23T15:38:16.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:38:16.395+0000 7fdb33506700 1 -- 172.21.15.76:0/1325700572 wait complete. 2024-03-23T15:38:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:16 smithi192 ceph-mon[36653]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:17.180 DEBUG:teuthology.orchestra.run.smithi076:osd.6> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.6.service 2024-03-23T15:38:17.182 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_2... 2024-03-23T15:38:17.183 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T15:38:17.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:17 smithi183 ceph-mon[36296]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T15:38:17.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:17 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:17.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:17 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:17.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:17 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:17.397 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:17 smithi076 ceph-mon[32176]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T15:38:17.397 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:17.397 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:17.397 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:17 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:17.428 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:38:17.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:17 smithi192 ceph-mon[36653]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-23T15:38:17.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:17 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:17.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:17 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:17.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:17 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:18.319 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:18 smithi076 ceph-mon[32176]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T15:38:18.319 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:18 smithi076 ceph-mon[32176]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T15:38:18.319 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:18 smithi076 ceph-mon[32176]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:38:18.319 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:18 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:18.319 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:18 smithi076 ceph-mon[32176]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:18.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:18 smithi183 ceph-mon[36296]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T15:38:18.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:18 smithi183 ceph-mon[36296]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T15:38:18.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:18 smithi183 ceph-mon[36296]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:38:18.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:18 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:18.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:18 smithi183 ceph-mon[36296]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:18.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:18 smithi192 ceph-mon[36653]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-23T15:38:18.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:18 smithi192 ceph-mon[36653]: osdmap e44: 7 total, 6 up, 7 in 2024-03-23T15:38:18.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:18 smithi192 ceph-mon[36653]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:38:18.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:18 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:18.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:18 smithi192 ceph-mon[36653]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:18.575 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:38:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:38:18.511+0000 7f10c7444700 -1 osd.6 0 waiting for initial osdmap 2024-03-23T15:38:18.575 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:38:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:38:18.523+0000 7f10bf22f700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:38:19.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:19 smithi076 ceph-mon[32176]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:38:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:19 smithi076 ceph-mon[32176]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T15:38:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:19 smithi076 ceph-mon[32176]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' 2024-03-23T15:38:19.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:19 smithi183 ceph-mon[36296]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:38:19.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:19 smithi183 ceph-mon[36296]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T15:38:19.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:19 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:19.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:19 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:19.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:19 smithi183 ceph-mon[36296]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' 2024-03-23T15:38:19.486 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T15:38:19.486 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T15:38:19.486 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-23T15:38:19.486 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-23T15:38:19.486 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112809 s, 930 MB/s 2024-03-23T15:38:19.487 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-23T15:38:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:19 smithi192 ceph-mon[36653]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:38:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:19 smithi192 ceph-mon[36653]: osdmap e45: 7 total, 6 up, 7 in 2024-03-23T15:38:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:19 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:19 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:19 smithi192 ceph-mon[36653]: from='osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561]' entity='osd.6' 2024-03-23T15:38:20.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-03-23T15:38:20.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:20 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:38:20.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:20 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:38:20.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:20 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:20.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:20 smithi183 ceph-mon[36296]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:20.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:20 smithi183 ceph-mon[36296]: osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] boot 2024-03-23T15:38:20.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:20 smithi183 ceph-mon[36296]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T15:38:20.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:20 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:20.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:20 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:38:20.401 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:20 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:38:20.402 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:20.402 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:20 smithi076 ceph-mon[32176]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:20.402 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:20 smithi076 ceph-mon[32176]: osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] boot 2024-03-23T15:38:20.402 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:20 smithi076 ceph-mon[32176]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T15:38:20.402 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:20 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:20.450 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:38:20.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:20 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:38:20.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:20 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:38:20.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:20 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:20.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:20 smithi192 ceph-mon[36653]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-23T15:38:20.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:20 smithi192 ceph-mon[36653]: osd.6 [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] boot 2024-03-23T15:38:20.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:20 smithi192 ceph-mon[36653]: osdmap e46: 7 total, 7 up, 7 in 2024-03-23T15:38:20.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:20 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:38:21.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi076 2024-03-23T15:38:21.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:21.995 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi076 to 5696M 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T15:38:21.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:21 smithi183 ceph-mon[36296]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:22.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 -- 172.21.15.183:0/1247857456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328102230 msgr2=0x7f0328102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:22.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 --2- 172.21.15.183:0/1247857456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328102230 0x7f0328102650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f03240099c0 tx=0x7f0324009cd0 comp rx=0 tx=0).stop 2024-03-23T15:38:22.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 -- 172.21.15.183:0/1247857456 shutdown_connections 2024-03-23T15:38:22.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 --2- 172.21.15.183:0/1247857456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0328103dd0 0x7f032810a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:22.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 --2- 172.21.15.183:0/1247857456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328102230 0x7f0328102650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:22.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 --2- 172.21.15.183:0/1247857456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0328103430 0x7f0328103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:22.008 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 -- 172.21.15.183:0/1247857456 >> 172.21.15.183:0/1247857456 conn(0x7f03280fd810 msgr2=0x7f03280ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:38:22.008 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 -- 172.21.15.183:0/1247857456 shutdown_connections 2024-03-23T15:38:22.008 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.005+0000 7f032fa5e700 1 -- 172.21.15.183:0/1247857456 wait complete. 2024-03-23T15:38:22.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 Processor -- start 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 -- start start 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0328102230 0x7f032819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328103430 0x7f032819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0328103dd0 0x7f03281a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03281a2030 con 0x7f0328102230 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03281a2170 con 0x7f0328103430 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f03281a2300 con 0x7f0328103dd0 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328103430 0x7f032819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328103430 0x7f032819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57102/0 (socket says 172.21.15.183:57102) 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 -- 172.21.15.183:0/583010265 learned_addr learned my addr 172.21.15.183:0/583010265 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-23T15:38:22.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032d7fa700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0328102230 0x7f032819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032dffb700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0328103dd0 0x7f03281a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 -- 172.21.15.183:0/583010265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0328103dd0 msgr2=0x7f03281a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0328103dd0 0x7f03281a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 -- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0328102230 msgr2=0x7f032819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0328102230 0x7f032819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 -- 172.21.15.183:0/583010265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03240095f0 con 0x7f0328103430 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032cff9700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328103430 0x7f032819a9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f031800b530 tx=0x7f031800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f031e7fc700 1 -- 172.21.15.183:0/583010265 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318011830 con 0x7f0328103430 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f031e7fc700 1 -- 172.21.15.183:0/583010265 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0318011e70 con 0x7f0328103430 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.006+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03281a25e0 con 0x7f0328103430 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.007+0000 7f031e7fc700 1 -- 172.21.15.183:0/583010265 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800f2e0 con 0x7f0328103430 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.007+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281a2a90 con 0x7f0328103430 2024-03-23T15:38:22.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.007+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0328068ec0 con 0x7f0328103430 2024-03-23T15:38:22.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.008+0000 7f031e7fc700 1 -- 172.21.15.183:0/583010265 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f0318010610 con 0x7f0328103430 2024-03-23T15:38:22.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.009+0000 7f031e7fc700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f03140a2150 0x7f03140a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:22.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.009+0000 7f031e7fc700 1 -- 172.21.15.183:0/583010265 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f03180cc960 con 0x7f0328103430 2024-03-23T15:38:22.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.009+0000 7f032d7fa700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f03140a2150 0x7f03140a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:22.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.010+0000 7f032d7fa700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f03140a2150 0x7f03140a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0324006010 tx=0x7f0324000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:38:22.016 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.014+0000 7f031e7fc700 1 -- 172.21.15.183:0/583010265 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0318096640 con 0x7f0328103430 2024-03-23T15:38:22.219 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:22.217+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0328075b50 con 0x7f03140a2150 2024-03-23T15:38:22.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi076 2024-03-23T15:38:22.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi076 to 5696M 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T15:38:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:21 smithi192 ceph-mon[36653]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:22.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi076 2024-03-23T15:38:22.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi076 to 5696M 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: osdmap e47: 7 total, 7 up, 7 in 2024-03-23T15:38:22.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:21 smithi076 ceph-mon[32176]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:23.359 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:23 smithi183 ceph-mon[36296]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:38:23.359 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:38:23.359 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:38:23.359 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:23 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:23 smithi192 ceph-mon[36653]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:38:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:38:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:38:23.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:23 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:23.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:23 smithi076 ceph-mon[32176]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:38:23.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:38:23.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:38:23.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:23 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:24.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:24 smithi183 ceph-mon[36296]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:24.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:24 smithi192 ceph-mon[36653]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:24.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:24 smithi076 ceph-mon[32176]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:25 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/244233291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]: dispatch 2024-03-23T15:38:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:25 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]: dispatch 2024-03-23T15:38:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:25 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]': finished 2024-03-23T15:38:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:25 smithi183 ceph-mon[36296]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T15:38:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:25.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:25 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/244233291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]: dispatch 2024-03-23T15:38:25.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:25 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]: dispatch 2024-03-23T15:38:25.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:25 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]': finished 2024-03-23T15:38:25.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:25 smithi192 ceph-mon[36653]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T15:38:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:25.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/244233291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]: dispatch 2024-03-23T15:38:25.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:25 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]: dispatch 2024-03-23T15:38:25.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:25 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d430ca1a-9ec0-43c2-9f12-01ca89b117fb"}]': finished 2024-03-23T15:38:25.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:25 smithi076 ceph-mon[32176]: osdmap e48: 8 total, 7 up, 8 in 2024-03-23T15:38:25.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:26.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:26 smithi183 ceph-mon[36296]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:26.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:26 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/1685579598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:26.470 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:26 smithi192 ceph-mon[36653]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:26.471 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:26 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/1685579598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:26.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:26 smithi076 ceph-mon[32176]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:26.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:26 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/1685579598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:28.776 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:28 smithi183 ceph-mon[36296]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:28.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:28 smithi192 ceph-mon[36653]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:28.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:28 smithi076 ceph-mon[32176]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:30.790 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:30 smithi183 ceph-mon[36296]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:30 smithi192 ceph-mon[36653]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:30.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:30 smithi076 ceph-mon[32176]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:32.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:32 smithi192 ceph-mon[36653]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:32.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:32 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T15:38:32.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:32 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:32 smithi076 ceph-mon[32176]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:32 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T15:38:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:32 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:32.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:32 smithi183 ceph-mon[36296]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:32.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:32 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-23T15:38:32.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:32 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:33.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:33 smithi183 ceph-mon[36296]: Deploying daemon osd.7 on smithi183 2024-03-23T15:38:33.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:33 smithi192 ceph-mon[36653]: Deploying daemon osd.7 on smithi183 2024-03-23T15:38:33.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:33 smithi076 ceph-mon[32176]: Deploying daemon osd.7 on smithi183 2024-03-23T15:38:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:34 smithi192 ceph-mon[36653]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:34 smithi076 ceph-mon[32176]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:34 smithi183 ceph-mon[36296]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:36 smithi192 ceph-mon[36653]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:36.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:36 smithi076 ceph-mon[32176]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:36.831 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:36 smithi183 ceph-mon[36296]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:38 smithi192 ceph-mon[36653]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:38.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:38 smithi076 ceph-mon[32176]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:38.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:38 smithi183 ceph-mon[36296]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:38.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:38.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:38.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:40.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:40 smithi183 ceph-mon[36296]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:40.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:40 smithi192 ceph-mon[36653]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:40.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:40 smithi076 ceph-mon[32176]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:41.484 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.481+0000 7f031e7fc700 1 -- 172.21.15.183:0/583010265 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0328075b50 con 0x7f03140a2150 2024-03-23T15:38:41.484 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-03-23T15:38:41.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f03140a2150 msgr2=0x7f03140a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:41.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f03140a2150 0x7f03140a4610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0324006010 tx=0x7f0324000bc0 comp rx=0 tx=0).stop 2024-03-23T15:38:41.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328103430 msgr2=0x7f032819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:41.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328103430 0x7f032819a9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f031800b530 tx=0x7f031800b840 comp rx=0 tx=0).stop 2024-03-23T15:38:41.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 shutdown_connections 2024-03-23T15:38:41.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f03140a2150 0x7f03140a4610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:41.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0328103dd0 0x7f03281a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:41.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0328102230 0x7f032819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:41.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 --2- 172.21.15.183:0/583010265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0328103430 0x7f032819a9e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:41.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 >> 172.21.15.183:0/583010265 conn(0x7f03280fd810 msgr2=0x7f0328074430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:38:41.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 shutdown_connections 2024-03-23T15:38:41.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:38:41.484+0000 7f032fa5e700 1 -- 172.21.15.183:0/583010265 wait complete. 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:41.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:42.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:38:42.166 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.7.service 2024-03-23T15:38:42.168 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2024-03-23T15:38:42.168 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T15:38:42.414 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:38:42.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:42 smithi183 ceph-mon[36296]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:42.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:42 smithi183 ceph-mon[36296]: from='osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T15:38:42.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:42 smithi183 ceph-mon[36296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T15:38:42.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:42 smithi192 ceph-mon[36653]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:42.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:42 smithi192 ceph-mon[36653]: from='osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T15:38:42.803 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:42 smithi192 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T15:38:43.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:42 smithi076 ceph-mon[32176]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:43.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:42 smithi076 ceph-mon[32176]: from='osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T15:38:43.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:42 smithi076 ceph-mon[32176]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-23T15:38:43.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:43 smithi183 ceph-mon[36296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T15:38:43.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:43 smithi183 ceph-mon[36296]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T15:38:43.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:43 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:43.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:43 smithi183 ceph-mon[36296]: from='osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:38:43.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:43 smithi183 ceph-mon[36296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:38:43.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:43 smithi183 ceph-mon[36296]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:43.886 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:43 smithi192 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T15:38:43.887 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:43 smithi192 ceph-mon[36653]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T15:38:43.887 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:43 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:43.887 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:43 smithi192 ceph-mon[36653]: from='osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:38:43.887 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:43 smithi192 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:38:43.887 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:43 smithi192 ceph-mon[36653]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:44.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:43 smithi076 ceph-mon[32176]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-23T15:38:44.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:43 smithi076 ceph-mon[32176]: osdmap e49: 8 total, 7 up, 8 in 2024-03-23T15:38:44.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:43 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:44.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:43 smithi076 ceph-mon[32176]: from='osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:38:44.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:43 smithi076 ceph-mon[32176]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:38:44.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:43 smithi076 ceph-mon[32176]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-23T15:38:44.133 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:38:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:38:44.049+0000 7fad67574700 -1 osd.7 0 waiting for initial osdmap 2024-03-23T15:38:44.133 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:38:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:38:44.060+0000 7fad5eb5e700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:38:44.605 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T15:38:44.605 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T15:38:44.605 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-23T15:38:44.605 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-23T15:38:44.605 INFO:teuthology.orchestra.run.smithi192.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0158749 s, 661 MB/s 2024-03-23T15:38:44.606 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-23T15:38:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:44 smithi183 ceph-mon[36296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:38:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:44 smithi183 ceph-mon[36296]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T15:38:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:44 smithi183 ceph-mon[36296]: from='osd.7 ' entity='osd.7' 2024-03-23T15:38:44.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:44 smithi192 ceph-mon[36653]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:38:44.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:44 smithi192 ceph-mon[36653]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T15:38:44.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:44.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:44.990 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:44 smithi192 ceph-mon[36653]: from='osd.7 ' entity='osd.7' 2024-03-23T15:38:45.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:44 smithi076 ceph-mon[32176]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:38:45.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:44 smithi076 ceph-mon[32176]: osdmap e50: 8 total, 7 up, 8 in 2024-03-23T15:38:45.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:45.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:45.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:44 smithi076 ceph-mon[32176]: from='osd.7 ' entity='osd.7' 2024-03-23T15:38:45.339 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-03-23T15:38:45.578 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:38:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:38:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:38:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] boot 2024-03-23T15:38:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T15:38:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi183 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi183 to 5693M 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:45 smithi183 ceph-mon[36296]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] boot 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi183 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:45.986 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:45.987 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi183 to 5693M 2024-03-23T15:38:45.987 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.987 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:45.987 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:45.987 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:45.987 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:45 smithi192 ceph-mon[36653]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: osd.7 [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] boot 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: osdmap e51: 8 total, 8 up, 8 in 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi183 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:46.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:38:46.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi183 to 5693M 2024-03-23T15:38:46.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:46.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:46.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:38:46.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:38:46.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:45 smithi076 ceph-mon[32176]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:46.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:46 smithi192 ceph-mon[36653]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T15:38:46.756 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:46 smithi192 ceph-mon[36653]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T15:38:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:46 smithi183 ceph-mon[36296]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T15:38:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:46 smithi183 ceph-mon[36296]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T15:38:47.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:46 smithi076 ceph-mon[32176]: osdmap e52: 8 total, 8 up, 8 in 2024-03-23T15:38:47.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:46 smithi076 ceph-mon[32176]: osdmap e53: 8 total, 8 up, 8 in 2024-03-23T15:38:47.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 -- 172.21.15.192:0/2108537921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc102d40 msgr2=0x7efcfc1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:47.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2108537921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc102d40 0x7efcfc1095f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b0a0 tx=0x7efcf400b3b0 comp rx=0 tx=0).stop 2024-03-23T15:38:47.312 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 -- 172.21.15.192:0/2108537921 shutdown_connections 2024-03-23T15:38:47.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2108537921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc102d40 0x7efcfc1095f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:47.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2108537921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcfc1011a0 0x7efcfc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:47.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2108537921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc1023a0 0x7efcfc102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:47.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 -- 172.21.15.192:0/2108537921 >> 172.21.15.192:0/2108537921 conn(0x7efcfc0fc760 msgr2=0x7efcfc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:38:47.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 -- 172.21.15.192:0/2108537921 shutdown_connections 2024-03-23T15:38:47.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.310+0000 7efd03ee3700 1 -- 172.21.15.192:0/2108537921 wait complete. 2024-03-23T15:38:47.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 Processor -- start 2024-03-23T15:38:47.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 -- start start 2024-03-23T15:38:47.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcfc1011a0 0x7efcfc06b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:47.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc1023a0 0x7efcfc19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:47.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc102d40 0x7efcfc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:47.315 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc19fd70 con 0x7efcfc102d40 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc19fee0 con 0x7efcfc1011a0 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd03ee3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1a00a0 con 0x7efcfc1023a0 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc1023a0 0x7efcfc19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc1023a0 0x7efcfc19cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52942/0 (socket says 172.21.15.192:52942) 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 -- 172.21.15.192:0/2047336368 learned_addr learned my addr 172.21.15.192:0/2047336368 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd01c7f700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcfc1011a0 0x7efcfc06b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd02480700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc102d40 0x7efcfc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 -- 172.21.15.192:0/2047336368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcfc1011a0 msgr2=0x7efcfc06b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcfc1011a0 0x7efcfc06b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 -- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc102d40 msgr2=0x7efcfc19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc102d40 0x7efcfc19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 -- 172.21.15.192:0/2047336368 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf4009d00 con 0x7efcfc1023a0 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.311+0000 7efd0147e700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc1023a0 0x7efcfc19cf90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c9b0 tx=0x7efcec00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:38:47.316 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.312+0000 7efcf2ffd700 1 -- 172.21.15.192:0/2047336368 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec007b00 con 0x7efcfc1023a0 2024-03-23T15:38:47.317 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.312+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc1a0380 con 0x7efcfc1023a0 2024-03-23T15:38:47.317 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.312+0000 7efcf2ffd700 1 -- 172.21.15.192:0/2047336368 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcec007c60 con 0x7efcfc1023a0 2024-03-23T15:38:47.317 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.312+0000 7efcf2ffd700 1 -- 172.21.15.192:0/2047336368 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec018660 con 0x7efcfc1023a0 2024-03-23T15:38:47.317 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.312+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc1a08d0 con 0x7efcfc1023a0 2024-03-23T15:38:47.317 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.312+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcfc108500 con 0x7efcfc1023a0 2024-03-23T15:38:47.318 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.314+0000 7efcf2ffd700 1 -- 172.21.15.192:0/2047336368 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7efcec00f450 con 0x7efcfc1023a0 2024-03-23T15:38:47.318 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.314+0000 7efcf2ffd700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7efce80a2150 0x7efce80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:38:47.318 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.315+0000 7efcf2ffd700 1 -- 172.21.15.192:0/2047336368 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7efcec0cd130 con 0x7efcfc1023a0 2024-03-23T15:38:47.318 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.315+0000 7efd01c7f700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7efce80a2150 0x7efce80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:38:47.318 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.315+0000 7efd01c7f700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7efce80a2150 0x7efce80a4610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efcfc102200 tx=0x7efcf800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:38:47.321 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.319+0000 7efcf2ffd700 1 -- 172.21.15.192:0/2047336368 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7efcec096c40 con 0x7efcfc1023a0 2024-03-23T15:38:47.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:38:47.522+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efcfc02dc00 con 0x7efce80a2150 2024-03-23T15:38:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:47 smithi192 ceph-mon[36653]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:47 smithi192 ceph-mon[36653]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:38:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:38:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:38:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:47 smithi183 ceph-mon[36296]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:47 smithi183 ceph-mon[36296]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:38:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:38:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:38:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:48.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:47 smithi076 ceph-mon[32176]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:47 smithi076 ceph-mon[32176]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:38:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:38:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:38:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:48.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:48 smithi192 ceph-mon[36653]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T15:38:49.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:48 smithi076 ceph-mon[32176]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T15:38:49.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:48 smithi183 ceph-mon[36296]: osdmap e54: 8 total, 8 up, 8 in 2024-03-23T15:38:49.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:49 smithi192 ceph-mon[36653]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T15:38:49.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:49 smithi192 ceph-mon[36653]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:49.903 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:49 smithi076 ceph-mon[32176]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T15:38:49.904 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:49 smithi076 ceph-mon[32176]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:50.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:49 smithi183 ceph-mon[36296]: osdmap e55: 8 total, 8 up, 8 in 2024-03-23T15:38:50.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:49 smithi183 ceph-mon[36296]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:50 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/2708808363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]: dispatch 2024-03-23T15:38:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:50 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]: dispatch 2024-03-23T15:38:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:50 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]': finished 2024-03-23T15:38:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:50 smithi192 ceph-mon[36653]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T15:38:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:38:51.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:50 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/2708808363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]: dispatch 2024-03-23T15:38:51.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:50 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]: dispatch 2024-03-23T15:38:51.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:50 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]': finished 2024-03-23T15:38:51.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:50 smithi076 ceph-mon[32176]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T15:38:51.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:38:51.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:50 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/2708808363' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]: dispatch 2024-03-23T15:38:51.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:50 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]: dispatch 2024-03-23T15:38:51.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:50 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4a7d4f-99a1-4109-a142-427ef07c1333"}]': finished 2024-03-23T15:38:51.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:50 smithi183 ceph-mon[36296]: osdmap e56: 9 total, 8 up, 9 in 2024-03-23T15:38:51.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:38:52.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:51 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/2033438969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:52.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:51 smithi192 ceph-mon[36653]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:52.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:51 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/2033438969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:52.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:51 smithi076 ceph-mon[32176]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:52.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:51 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/2033438969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:38:52.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:51 smithi183 ceph-mon[36296]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:54.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:54 smithi192 ceph-mon[36653]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:54 smithi076 ceph-mon[32176]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:54.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:54 smithi183 ceph-mon[36296]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:56.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:56 smithi192 ceph-mon[36653]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:56.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:56 smithi076 ceph-mon[32176]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:56.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:56 smithi183 ceph-mon[36296]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:58.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:58 smithi192 ceph-mon[36653]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:58.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T15:38:58.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:58 smithi076 ceph-mon[32176]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:58.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T15:38:58.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:58 smithi183 ceph-mon[36296]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:38:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-23T15:38:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:38:59.651 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:38:59 smithi192 ceph-mon[36653]: Deploying daemon osd.8 on smithi192 2024-03-23T15:38:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:38:59 smithi076 ceph-mon[32176]: Deploying daemon osd.8 on smithi192 2024-03-23T15:38:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:38:59 smithi183 ceph-mon[36296]: Deploying daemon osd.8 on smithi192 2024-03-23T15:39:00.817 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:00 smithi192 ceph-mon[36653]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:00.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:00 smithi076 ceph-mon[32176]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:00 smithi183 ceph-mon[36296]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:02.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:02 smithi192 ceph-mon[36653]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:02.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:02 smithi076 ceph-mon[32176]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:02 smithi183 ceph-mon[36296]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:04.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:04 smithi192 ceph-mon[36653]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:04.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:04.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:04.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:04.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:04 smithi076 ceph-mon[32176]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:04.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:04.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:04.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:04 smithi183 ceph-mon[36296]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:06.790 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:06 smithi192 ceph-mon[36653]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:06.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:06 smithi076 ceph-mon[32176]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:06.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:06 smithi183 ceph-mon[36296]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:07.489 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.486+0000 7efcf2ffd700 1 -- 172.21.15.192:0/2047336368 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efcfc02dc00 con 0x7efce80a2150 2024-03-23T15:39:07.489 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.488+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7efce80a2150 msgr2=0x7efce80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.488+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7efce80a2150 0x7efce80a4610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efcfc102200 tx=0x7efcf800b410 comp rx=0 tx=0).stop 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.488+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc1023a0 msgr2=0x7efcfc19cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.488+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc1023a0 0x7efcfc19cf90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c9b0 tx=0x7efcec00ccc0 comp rx=0 tx=0).stop 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 shutdown_connections 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7efce80a2150 0x7efce80a4610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc102d40 0x7efcfc19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcfc1011a0 0x7efcfc06b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:07.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 --2- 172.21.15.192:0/2047336368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc1023a0 0x7efcfc19cf90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:07.493 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 >> 172.21.15.192:0/2047336368 conn(0x7efcfc0fc760 msgr2=0x7efcfc105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:39:07.493 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 shutdown_connections 2024-03-23T15:39:07.493 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:39:07.489+0000 7efd03ee3700 1 -- 172.21.15.192:0/2047336368 wait complete. 2024-03-23T15:39:07.781 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.781 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.781 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:07.781 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:07.781 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:07.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:07.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T15:39:07.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:07 smithi192 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.953 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:07.954 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T15:39:07.954 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:07 smithi183 ceph-mon[36296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T15:39:08.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T15:39:08.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:07 smithi076 ceph-mon[32176]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-23T15:39:08.148 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.8.service 2024-03-23T15:39:08.151 INFO:tasks.cephadm:Deploying osd.9 on smithi076 with /dev/vg_nvme/lv_1... 2024-03-23T15:39:08.151 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T15:39:08.406 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:39:09.688 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:09 smithi192 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T15:39:09.689 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:09 smithi192 ceph-mon[36653]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T15:39:09.689 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:09 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:09.689 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:09 smithi192 ceph-mon[36653]: from='osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:39:09.689 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:09 smithi192 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:39:09.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:09 smithi076 ceph-mon[32176]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T15:39:09.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:09 smithi076 ceph-mon[32176]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T15:39:09.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:09 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:09.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:09 smithi076 ceph-mon[32176]: from='osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:39:09.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:09 smithi076 ceph-mon[32176]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:39:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:09 smithi183 ceph-mon[36296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-23T15:39:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:09 smithi183 ceph-mon[36296]: osdmap e57: 9 total, 8 up, 9 in 2024-03-23T15:39:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:09 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:09 smithi183 ceph-mon[36296]: from='osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:39:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:09 smithi183 ceph-mon[36296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:39:09.940 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:39:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:39:09.923+0000 7f5f67678700 -1 osd.8 0 waiting for initial osdmap 2024-03-23T15:39:09.941 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:39:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:39:09.938+0000 7f5f5f463700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:39:10.636 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T15:39:10.636 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T15:39:10.636 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-03-23T15:39:10.636 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-03-23T15:39:10.636 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114689 s, 914 MB/s 2024-03-23T15:39:10.637 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-03-23T15:39:10.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:10 smithi192 ceph-mon[36653]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:10.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:10 smithi192 ceph-mon[36653]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:39:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:10 smithi192 ceph-mon[36653]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T15:39:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:10 smithi192 ceph-mon[36653]: from='osd.8 ' entity='osd.8' 2024-03-23T15:39:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:10 smithi076 ceph-mon[32176]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:10 smithi076 ceph-mon[32176]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:39:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:10 smithi076 ceph-mon[32176]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T15:39:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:10 smithi076 ceph-mon[32176]: from='osd.8 ' entity='osd.8' 2024-03-23T15:39:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:10 smithi183 ceph-mon[36296]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:10 smithi183 ceph-mon[36296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:39:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:10 smithi183 ceph-mon[36296]: osdmap e58: 9 total, 8 up, 9 in 2024-03-23T15:39:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:10 smithi183 ceph-mon[36296]: from='osd.8 ' entity='osd.8' 2024-03-23T15:39:11.292 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_1 2024-03-23T15:39:11.545 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:39:11.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] boot 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:11.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:11.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] boot 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:11.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:11.577 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:39:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:39:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: osd.8 [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] boot 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: osdmap e59: 9 total, 9 up, 9 in 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:11.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:12.654 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:12 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi192 2024-03-23T15:39:12.654 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:12 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi192 to 5693M 2024-03-23T15:39:12.654 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:12 smithi076 ceph-mon[32176]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:12.654 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:12 smithi076 ceph-mon[32176]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T15:39:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:12 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi192 2024-03-23T15:39:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:12 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi192 to 5693M 2024-03-23T15:39:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:12 smithi192 ceph-mon[36653]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:12.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:12 smithi192 ceph-mon[36653]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T15:39:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:12 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi192 2024-03-23T15:39:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:12 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi192 to 5693M 2024-03-23T15:39:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:12 smithi183 ceph-mon[36296]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-23T15:39:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:12 smithi183 ceph-mon[36296]: osdmap e60: 9 total, 9 up, 9 in 2024-03-23T15:39:13.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 -- 172.21.15.76:0/1621492836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 msgr2=0x7fd53c109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 --2- 172.21.15.76:0/1621492836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c109580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd53400b0d0 tx=0x7fd53400b3e0 comp rx=0 tx=0).stop 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 -- 172.21.15.76:0/1621492836 shutdown_connections 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 --2- 172.21.15.76:0/1621492836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c109580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 --2- 172.21.15.76:0/1621492836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd53c101130 0x7fd53c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 --2- 172.21.15.76:0/1621492836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53c102330 0x7fd53c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 -- 172.21.15.76:0/1621492836 >> 172.21.15.76:0/1621492836 conn(0x7fd53c0fc6d0 msgr2=0x7fd53c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 -- 172.21.15.76:0/1621492836 shutdown_connections 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.168+0000 7fd544523700 1 -- 172.21.15.76:0/1621492836 wait complete. 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 Processor -- start 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 -- start start 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53c101130 0x7fd53c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:13.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd53c102330 0x7fd53c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c19edb0 con 0x7fd53c102cd0 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c19eef0 con 0x7fd53c101130 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd544523700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c19f080 con 0x7fd53c102330 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58688/0 (socket says 172.21.15.76:58688) 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 -- 172.21.15.76:0/2730541638 learned_addr learned my addr 172.21.15.76:0/2730541638 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd541abe700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd53c102330 0x7fd53c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd5422bf700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53c101130 0x7fd53c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 -- 172.21.15.76:0/2730541638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd53c102330 msgr2=0x7fd53c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd53c102330 0x7fd53c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 -- 172.21.15.76:0/2730541638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53c101130 msgr2=0x7fd53c197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53c101130 0x7fd53c197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd542ac0700 1 -- 172.21.15.76:0/2730541638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd534009d00 con 0x7fd53c102cd0 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.169+0000 7fd5422bf700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53c101130 0x7fd53c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.170+0000 7fd542ac0700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c19e7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5340078a0 tx=0x7fd534007980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.170+0000 7fd5337fe700 1 -- 172.21.15.76:0/2730541638 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53400e040 con 0x7fd53c102cd0 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.170+0000 7fd5337fe700 1 -- 172.21.15.76:0/2730541638 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd534003e10 con 0x7fd53c102cd0 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.170+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c19f300 con 0x7fd53c102cd0 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.170+0000 7fd5337fe700 1 -- 172.21.15.76:0/2730541638 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534012b50 con 0x7fd53c102cd0 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.170+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c19f740 con 0x7fd53c102cd0 2024-03-23T15:39:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.171+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53c108420 con 0x7fd53c102cd0 2024-03-23T15:39:13.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.172+0000 7fd5337fe700 1 -- 172.21.15.76:0/2730541638 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fd534012cb0 con 0x7fd53c102cd0 2024-03-23T15:39:13.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.172+0000 7fd5337fe700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd5280a2150 0x7fd5280a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:13.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.172+0000 7fd5422bf700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd5280a2150 0x7fd5280a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:13.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.172+0000 7fd5337fe700 1 -- 172.21.15.76:0/2730541638 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7fd5340ceeb0 con 0x7fd53c102cd0 2024-03-23T15:39:13.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.173+0000 7fd5422bf700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd5280a2150 0x7fd5280a4610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd53c102190 tx=0x7fd538008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:39:13.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.176+0000 7fd5337fe700 1 -- 172.21.15.76:0/2730541638 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd5340988a0 con 0x7fd53c102cd0 2024-03-23T15:39:13.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:13.381+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd53c02dc00 con 0x7fd5280a2150 2024-03-23T15:39:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:13 smithi192 ceph-mon[36653]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T15:39:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:39:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:39:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:13.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:13 smithi076 ceph-mon[32176]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T15:39:13.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:39:13.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:39:13.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:13 smithi183 ceph-mon[36296]: osdmap e61: 9 total, 9 up, 9 in 2024-03-23T15:39:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:39:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:39:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:14.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:14 smithi076 ceph-mon[32176]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:39:14.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:14 smithi076 ceph-mon[32176]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:14.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:14 smithi076 ceph-mon[32176]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T15:39:14.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:14 smithi192 ceph-mon[36653]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:39:14.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:14 smithi192 ceph-mon[36653]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:14.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:14 smithi192 ceph-mon[36653]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T15:39:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:14 smithi183 ceph-mon[36296]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:39:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:14 smithi183 ceph-mon[36296]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:14 smithi183 ceph-mon[36296]: osdmap e62: 9 total, 9 up, 9 in 2024-03-23T15:39:15.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:15 smithi192 ceph-mon[36653]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T15:39:15.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:15 smithi076 ceph-mon[32176]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T15:39:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:15 smithi183 ceph-mon[36296]: osdmap e63: 9 total, 9 up, 9 in 2024-03-23T15:39:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:16 smithi192 ceph-mon[36653]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T15:39:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:16 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/596762873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93eb38c-a6a9-453f-8b9a-da479e25a337"}]: dispatch 2024-03-23T15:39:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:16 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/596762873' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c93eb38c-a6a9-453f-8b9a-da479e25a337"}]': finished 2024-03-23T15:39:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:16 smithi192 ceph-mon[36653]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T15:39:16.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:16 smithi076 ceph-mon[32176]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T15:39:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:16 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/596762873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93eb38c-a6a9-453f-8b9a-da479e25a337"}]: dispatch 2024-03-23T15:39:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:16 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/596762873' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c93eb38c-a6a9-453f-8b9a-da479e25a337"}]': finished 2024-03-23T15:39:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:16 smithi076 ceph-mon[32176]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T15:39:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:16 smithi183 ceph-mon[36296]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T15:39:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:16 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/596762873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93eb38c-a6a9-453f-8b9a-da479e25a337"}]: dispatch 2024-03-23T15:39:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:16 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/596762873' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c93eb38c-a6a9-453f-8b9a-da479e25a337"}]': finished 2024-03-23T15:39:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:16 smithi183 ceph-mon[36296]: osdmap e64: 10 total, 9 up, 10 in 2024-03-23T15:39:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:17 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1131141180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:39:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:17 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1131141180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:39:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:17 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1131141180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:39:18.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:18 smithi192 ceph-mon[36653]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-23T15:39:18.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:18 smithi076 ceph-mon[32176]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-23T15:39:18.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:18 smithi183 ceph-mon[36296]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-23T15:39:20.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:20 smithi192 ceph-mon[36653]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T15:39:20.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:20 smithi076 ceph-mon[32176]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T15:39:20.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:20 smithi183 ceph-mon[36296]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T15:39:22.749 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:22 smithi076 ceph-mon[32176]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-23T15:39:22.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:22 smithi192 ceph-mon[36653]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-23T15:39:22.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:22 smithi183 ceph-mon[36296]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-23T15:39:24.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:24 smithi076 ceph-mon[32176]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-23T15:39:24.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:24 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T15:39:24.718 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:24 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:24.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:24 smithi192 ceph-mon[36653]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-23T15:39:24.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:24 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T15:39:24.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:24 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:24 smithi183 ceph-mon[36296]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-23T15:39:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:24 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-23T15:39:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:24 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:25.816 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:25 smithi076 ceph-mon[32176]: Deploying daemon osd.9 on smithi076 2024-03-23T15:39:25.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:25 smithi192 ceph-mon[36653]: Deploying daemon osd.9 on smithi076 2024-03-23T15:39:25.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:25 smithi183 ceph-mon[36296]: Deploying daemon osd.9 on smithi076 2024-03-23T15:39:26.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:26 smithi192 ceph-mon[36653]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:26.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:26 smithi076 ceph-mon[32176]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:26.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:26 smithi183 ceph-mon[36296]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:28.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:28 smithi192 ceph-mon[36653]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:28.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:28 smithi076 ceph-mon[32176]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:28 smithi183 ceph-mon[36296]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:30 smithi192 ceph-mon[36653]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:30.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:30 smithi076 ceph-mon[32176]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:30.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:30 smithi183 ceph-mon[36296]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:32.756 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:32 smithi076 ceph-mon[32176]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:32.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:32 smithi192 ceph-mon[36653]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:32.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:32 smithi183 ceph-mon[36296]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:33.446 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 9 on host 'smithi076' 2024-03-23T15:39:33.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.443+0000 7fd5337fe700 1 -- 172.21.15.76:0/2730541638 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd53c02dc00 con 0x7fd5280a2150 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd5280a2150 msgr2=0x7fd5280a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd5280a2150 0x7fd5280a4610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd53c102190 tx=0x7fd538008040 comp rx=0 tx=0).stop 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 msgr2=0x7fd53c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c19e7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5340078a0 tx=0x7fd534007980 comp rx=0 tx=0).stop 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 shutdown_connections 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd5280a2150 0x7fd5280a4610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd53c102cd0 0x7fd53c19e7e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53c101130 0x7fd53c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 --2- 172.21.15.76:0/2730541638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd53c102330 0x7fd53c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 >> 172.21.15.76:0/2730541638 conn(0x7fd53c0fc6d0 msgr2=0x7fd53c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:39:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 shutdown_connections 2024-03-23T15:39:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:39:33.445+0000 7fd544523700 1 -- 172.21.15.76:0/2730541638 wait complete. 2024-03-23T15:39:34.111 DEBUG:teuthology.orchestra.run.smithi076:osd.9> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.9.service 2024-03-23T15:39:34.114 INFO:tasks.cephadm:Deploying osd.10 on smithi183 with /dev/vg_nvme/lv_1... 2024-03-23T15:39:34.114 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T15:39:34.362 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:39:34.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:34 smithi183 ceph-mon[36296]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:34.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:34 smithi183 ceph-mon[36296]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T15:39:34.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:34 smithi192 ceph-mon[36653]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:34.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:34 smithi192 ceph-mon[36653]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T15:39:34.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:34 smithi076 ceph-mon[32176]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:34 smithi076 ceph-mon[32176]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-23T15:39:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:35.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:35 smithi076 ceph-mon[32176]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T15:39:35.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:35 smithi076 ceph-mon[32176]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T15:39:35.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:35 smithi076 ceph-mon[32176]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:39:35.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:35 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:39:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:39:35.450+0000 7f649e466700 -1 osd.9 0 waiting for initial osdmap 2024-03-23T15:39:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:39:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:39:35.462+0000 7f649a259700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:39:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:35 smithi183 ceph-mon[36296]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T15:39:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:35 smithi183 ceph-mon[36296]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T15:39:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:35 smithi183 ceph-mon[36296]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:39:35.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:35 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:35.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:35 smithi192 ceph-mon[36653]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-23T15:39:35.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:35 smithi192 ceph-mon[36653]: osdmap e65: 10 total, 9 up, 10 in 2024-03-23T15:39:35.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:35 smithi192 ceph-mon[36653]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-03-23T15:39:35.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:35 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:36.478 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T15:39:36.478 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T15:39:36.478 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-23T15:39:36.479 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-23T15:39:36.479 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153394 s, 684 MB/s 2024-03-23T15:39:36.479 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-23T15:39:36.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:36 smithi076 ceph-mon[32176]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:39:36.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:36 smithi076 ceph-mon[32176]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T15:39:36.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:36 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:36.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:36 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:36.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:36 smithi076 ceph-mon[32176]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:36 smithi183 ceph-mon[36296]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:39:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:36 smithi183 ceph-mon[36296]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T15:39:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:36 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:36 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:36 smithi183 ceph-mon[36296]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:36 smithi192 ceph-mon[36653]: from='osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-03-23T15:39:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:36 smithi192 ceph-mon[36653]: osdmap e66: 10 total, 9 up, 10 in 2024-03-23T15:39:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:36 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:36 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:36 smithi192 ceph-mon[36653]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-23T15:39:37.103 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-03-23T15:39:37.355 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:39:37.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:37 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:39:37.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:37 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:39:37.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:37.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:37 smithi183 ceph-mon[36296]: osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] boot 2024-03-23T15:39:37.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:37 smithi183 ceph-mon[36296]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T15:39:37.795 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:37 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:37 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:39:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:37 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:39:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:37 smithi192 ceph-mon[36653]: osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] boot 2024-03-23T15:39:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:37 smithi192 ceph-mon[36653]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T15:39:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:37 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:37 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:39:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:37 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:39:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:37 smithi076 ceph-mon[32176]: osd.9 [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] boot 2024-03-23T15:39:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:37 smithi076 ceph-mon[32176]: osdmap e67: 10 total, 10 up, 10 in 2024-03-23T15:39:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:37 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:39:38.778 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:38.778 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T15:39:38.778 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.778 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:38.779 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T15:39:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:38.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:38.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: osdmap e68: 10 total, 10 up, 10 in 2024-03-23T15:39:38.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:39:38.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 -- 172.21.15.183:0/3915165828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac101de0 msgr2=0x7f6bac1096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/3915165828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac101de0 0x7f6bac1096a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400b0a0 tx=0x7f6ba400b3b0 comp rx=0 tx=0).stop 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 -- 172.21.15.183:0/3915165828 shutdown_connections 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/3915165828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac101de0 0x7f6bac1096a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/3915165828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bac102700 0x7f6bac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/3915165828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bac101440 0x7f6bac1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 -- 172.21.15.183:0/3915165828 >> 172.21.15.183:0/3915165828 conn(0x7f6bac0fbe90 msgr2=0x7f6bac0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 -- 172.21.15.183:0/3915165828 shutdown_connections 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.003+0000 7f6bb10c3700 1 -- 172.21.15.183:0/3915165828 wait complete. 2024-03-23T15:39:39.009 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 Processor -- start 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 -- start start 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bac101440 0x7f6bac19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bac101de0 0x7f6bac19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac102700 0x7f6bac1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac1a21e0 con 0x7f6bac101440 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac1a2320 con 0x7f6bac102700 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bb10c3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac1a24b0 con 0x7f6bac101de0 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bab59e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac102700 0x7f6bac1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bab59e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac102700 0x7f6bac1a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:47332/0 (socket says 172.21.15.183:47332) 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.004+0000 7f6bab59e700 1 -- 172.21.15.183:0/1279441492 learned_addr learned my addr 172.21.15.183:0/1279441492 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6baa59c700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bac101de0 0x7f6bac19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:39.010 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6baad9d700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bac101440 0x7f6bac19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bab59e700 1 -- 172.21.15.183:0/1279441492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bac101de0 msgr2=0x7f6bac19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bab59e700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bac101de0 0x7f6bac19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bab59e700 1 -- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bac101440 msgr2=0x7f6bac19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bab59e700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bac101440 0x7f6bac19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bab59e700 1 -- 172.21.15.183:0/1279441492 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba4009d00 con 0x7f6bac102700 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6baad9d700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bac101440 0x7f6bac19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bab59e700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac102700 0x7f6bac1a1c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40094c0 tx=0x7f6ba40095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6b93fff700 1 -- 172.21.15.183:0/1279441492 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400e040 con 0x7f6bac102700 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac1a2730 con 0x7f6bac102700 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6b93fff700 1 -- 172.21.15.183:0/1279441492 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba401d960 con 0x7f6bac102700 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6b93fff700 1 -- 172.21.15.183:0/1279441492 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba401bab0 con 0x7f6bac102700 2024-03-23T15:39:39.011 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.005+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac1a2c70 con 0x7f6bac102700 2024-03-23T15:39:39.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.006+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac068ec0 con 0x7f6bac102700 2024-03-23T15:39:39.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.007+0000 7f6b93fff700 1 -- 172.21.15.183:0/1279441492 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f6ba401dad0 con 0x7f6bac102700 2024-03-23T15:39:39.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.008+0000 7f6b93fff700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f6b940a2150 0x7f6b940a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:39:39.013 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.008+0000 7f6b93fff700 1 -- 172.21.15.183:0/1279441492 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f6ba40cf9a0 con 0x7f6bac102700 2024-03-23T15:39:39.013 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.008+0000 7f6baad9d700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f6b940a2150 0x7f6b940a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:39:39.013 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.008+0000 7f6baad9d700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f6b940a2150 0x7f6b940a4610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6bac06a6c0 tx=0x7f6b9c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:39:39.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.012+0000 7f6b93fff700 1 -- 172.21.15.183:0/1279441492 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6ba4099200 con 0x7f6bac102700 2024-03-23T15:39:39.219 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:39.216+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6bac0fe000 con 0x7f6b940a2150 2024-03-23T15:39:39.600 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:39 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi076 2024-03-23T15:39:39.600 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:39 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi076 to 4272M 2024-03-23T15:39:39.600 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:39:39.601 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:39:39.601 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:39 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:39 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi076 2024-03-23T15:39:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:39 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi076 to 4272M 2024-03-23T15:39:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:39 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:39:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:39 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:39:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:39 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:39.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:39 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi076 2024-03-23T15:39:39.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:39 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi076 to 4272M 2024-03-23T15:39:39.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:39 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:39:39.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:39 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:39:39.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:39 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:40.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:40 smithi192 ceph-mon[36653]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:39:40.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:40 smithi192 ceph-mon[36653]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:40.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:40 smithi076 ceph-mon[32176]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:39:40.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:40 smithi076 ceph-mon[32176]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:40.856 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:40 smithi183 ceph-mon[36296]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:39:40.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:40 smithi183 ceph-mon[36296]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:42.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:42 smithi192 ceph-mon[36653]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:42.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:42 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/2312305229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]: dispatch 2024-03-23T15:39:42.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:42 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]: dispatch 2024-03-23T15:39:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:42 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]': finished 2024-03-23T15:39:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:42 smithi192 ceph-mon[36653]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T15:39:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:42 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:39:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:42 smithi076 ceph-mon[32176]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:42 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/2312305229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]: dispatch 2024-03-23T15:39:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:42 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]: dispatch 2024-03-23T15:39:42.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:42 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]': finished 2024-03-23T15:39:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:42 smithi076 ceph-mon[32176]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T15:39:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:42 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:39:42.882 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:42 smithi183 ceph-mon[36296]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:42 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/2312305229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]: dispatch 2024-03-23T15:39:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:42 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]: dispatch 2024-03-23T15:39:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:42 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef61a179-f812-4a25-8605-cf03e2d8a31c"}]': finished 2024-03-23T15:39:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:42 smithi183 ceph-mon[36296]: osdmap e69: 11 total, 10 up, 11 in 2024-03-23T15:39:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:42 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:39:43.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:43 smithi192 ceph-mon[36653]: from='client.? 172.21.15.183:0/435983001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:39:43.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:43 smithi076 ceph-mon[32176]: from='client.? 172.21.15.183:0/435983001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:39:43.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:43 smithi183 ceph-mon[36296]: from='client.? 172.21.15.183:0/435983001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:39:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:44 smithi192 ceph-mon[36653]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:44 smithi076 ceph-mon[32176]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:44 smithi183 ceph-mon[36296]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:46 smithi192 ceph-mon[36653]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:46 smithi076 ceph-mon[32176]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:46 smithi183 ceph-mon[36296]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:48.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:48 smithi192 ceph-mon[36653]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:48.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:48 smithi076 ceph-mon[32176]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:48.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:48 smithi183 ceph-mon[36296]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:50 smithi192 ceph-mon[36653]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T15:39:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:50 smithi076 ceph-mon[32176]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T15:39:50.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:50 smithi183 ceph-mon[36296]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-23T15:39:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:39:51.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:51 smithi192 ceph-mon[36653]: Deploying daemon osd.10 on smithi183 2024-03-23T15:39:51.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:51 smithi076 ceph-mon[32176]: Deploying daemon osd.10 on smithi183 2024-03-23T15:39:51.847 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:51 smithi183 ceph-mon[36296]: Deploying daemon osd.10 on smithi183 2024-03-23T15:39:52.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:52 smithi192 ceph-mon[36653]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:52.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:52 smithi076 ceph-mon[32176]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:52 smithi183 ceph-mon[36296]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:54.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:54 smithi183 ceph-mon[36296]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:54.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:54 smithi192 ceph-mon[36653]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:54 smithi076 ceph-mon[32176]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:56.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:56.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:56.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:56.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:56 smithi192 ceph-mon[36653]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:56.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:56.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:56.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:56.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:56 smithi076 ceph-mon[32176]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:56.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:56.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:39:56.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:39:56.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:56 smithi183 ceph-mon[36296]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:58.778 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:58 smithi183 ceph-mon[36296]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:58.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:58 smithi192 ceph-mon[36653]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:58 smithi076 ceph-mon[32176]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:39:58.835 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.832+0000 7f6b93fff700 1 -- 172.21.15.183:0/1279441492 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6bac0fe000 con 0x7f6b940a2150 2024-03-23T15:39:58.835 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 10 on host 'smithi183' 2024-03-23T15:39:58.838 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f6b940a2150 msgr2=0x7f6b940a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:58.838 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f6b940a2150 0x7f6b940a4610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6bac06a6c0 tx=0x7f6b9c008040 comp rx=0 tx=0).stop 2024-03-23T15:39:58.838 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac102700 msgr2=0x7f6bac1a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:39:58.838 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac102700 0x7f6bac1a1c10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40094c0 tx=0x7f6ba40095a0 comp rx=0 tx=0).stop 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 shutdown_connections 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f6b940a2150 0x7f6b940a4610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bac102700 0x7f6bac1a1c10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6bac101440 0x7f6bac19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 --2- 172.21.15.183:0/1279441492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bac101de0 0x7f6bac19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 >> 172.21.15.183:0/1279441492 conn(0x7f6bac0fbe90 msgr2=0x7f6bac0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 shutdown_connections 2024-03-23T15:39:58.839 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-23T15:39:58.836+0000 7f6bb10c3700 1 -- 172.21.15.183:0/1279441492 wait complete. 2024-03-23T15:39:59.457 DEBUG:teuthology.orchestra.run.smithi183:osd.10> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.10.service 2024-03-23T15:39:59.460 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2024-03-23T15:39:59.460 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-23T15:39:59.704 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:40:00.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:59 smithi192 ceph-mon[36653]: from='osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T15:40:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:59 smithi192 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T15:40:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:59 smithi192 ceph-mon[36653]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:40:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:39:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:59 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:59 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:59 smithi076 ceph-mon[32176]: from='osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T15:40:00.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:59 smithi076 ceph-mon[32176]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T15:40:00.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:59 smithi076 ceph-mon[32176]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:40:00.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:59 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:39:59 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:59 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:59 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:59 smithi183 ceph-mon[36296]: from='osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T15:40:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:59 smithi183 ceph-mon[36296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-23T15:40:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:59 smithi183 ceph-mon[36296]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:40:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:59 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:39:59 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:00 smithi183 ceph-mon[36296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T15:40:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:00 smithi183 ceph-mon[36296]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T15:40:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:00 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:00 smithi183 ceph-mon[36296]: from='osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:40:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:00 smithi183 ceph-mon[36296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:40:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:00 smithi183 ceph-mon[36296]: overall HEALTH_OK 2024-03-23T15:40:01.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:00 smithi192 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T15:40:01.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:00 smithi192 ceph-mon[36653]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T15:40:01.130 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:00 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:01.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:00 smithi192 ceph-mon[36653]: from='osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:40:01.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:00 smithi192 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:40:01.131 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:00 smithi192 ceph-mon[36653]: overall HEALTH_OK 2024-03-23T15:40:01.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:00 smithi076 ceph-mon[32176]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-23T15:40:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:00 smithi076 ceph-mon[32176]: osdmap e70: 11 total, 10 up, 11 in 2024-03-23T15:40:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:00 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:00 smithi076 ceph-mon[32176]: from='osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:40:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:00 smithi076 ceph-mon[32176]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-23T15:40:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:00 smithi076 ceph-mon[32176]: overall HEALTH_OK 2024-03-23T15:40:01.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:40:01.272+0000 7fb22b988700 -1 osd.10 0 waiting for initial osdmap 2024-03-23T15:40:01.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:40:01.283+0000 7fb223773700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:40:01.895 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-23T15:40:01.895 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-23T15:40:01.895 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-03-23T15:40:01.895 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-03-23T15:40:01.895 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113004 s, 928 MB/s 2024-03-23T15:40:01.895 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-03-23T15:40:01.919 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-mon[36296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:40:01.919 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-mon[36296]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T15:40:01.919 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:01.919 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:01.919 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-mon[36296]: from='osd.10 ' entity='osd.10' 2024-03-23T15:40:01.919 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:01 smithi183 ceph-mon[36296]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:40:02.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:01 smithi192 ceph-mon[36653]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:40:02.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:01 smithi192 ceph-mon[36653]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T15:40:02.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:01 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:02.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:01 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:02.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:01 smithi192 ceph-mon[36653]: from='osd.10 ' entity='osd.10' 2024-03-23T15:40:02.226 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:01 smithi192 ceph-mon[36653]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:40:02.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:01 smithi076 ceph-mon[32176]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-23T15:40:02.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:01 smithi076 ceph-mon[32176]: osdmap e71: 11 total, 10 up, 11 in 2024-03-23T15:40:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:01 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:01 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:01 smithi076 ceph-mon[32176]: from='osd.10 ' entity='osd.10' 2024-03-23T15:40:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:01 smithi076 ceph-mon[32176]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-23T15:40:02.568 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2024-03-23T15:40:02.833 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:40:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:02 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:40:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:02 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:40:02.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:02.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:02 smithi183 ceph-mon[36296]: osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] boot 2024-03-23T15:40:02.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:02 smithi183 ceph-mon[36296]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T15:40:02.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:03.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:02 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:40:03.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:02 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:40:03.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:03.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:02 smithi192 ceph-mon[36653]: osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] boot 2024-03-23T15:40:03.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:02 smithi192 ceph-mon[36653]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T15:40:03.101 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:03.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:02 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:40:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:02 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:40:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:02 smithi076 ceph-mon[32176]: osd.10 [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] boot 2024-03-23T15:40:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:02 smithi076 ceph-mon[32176]: osdmap e72: 11 total, 11 up, 11 in 2024-03-23T15:40:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:40:04.018 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T15:40:04.018 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi183 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi183 to 4270M 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:04.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:04.020 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:04.020 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:04.020 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:03 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T15:40:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi183 2024-03-23T15:40:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi183 to 4270M 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:03 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: osdmap e73: 11 total, 11 up, 11 in 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi183 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi183 to 4270M 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:04.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:04.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:04.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:04.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:04.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:04.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:03 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 -- 172.21.15.192:0/1312336387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 msgr2=0x7f8ef4105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 --2- 172.21.15.192:0/1312336387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4105100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8009ca0 comp rx=0 tx=0).stop 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 -- 172.21.15.192:0/1312336387 shutdown_connections 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 --2- 172.21.15.192:0/1312336387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef4107f70 0x7f8ef410a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 --2- 172.21.15.192:0/1312336387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4105100 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 --2- 172.21.15.192:0/1312336387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef4105640 0x7f8ef4107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 -- 172.21.15.192:0/1312336387 >> 172.21.15.192:0/1312336387 conn(0x7f8ef40fc6c0 msgr2=0x7f8ef40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 -- 172.21.15.192:0/1312336387 shutdown_connections 2024-03-23T15:40:04.558 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.555+0000 7f8ef9347700 1 -- 172.21.15.192:0/1312336387 wait complete. 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 Processor -- start 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 -- start start 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef4105640 0x7f8ef4100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef4107f70 0x7f8ef4101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4101980 con 0x7f8ef4107f70 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4101af0 con 0x7f8ef4105640 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef9347700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41a22b0 con 0x7f8ef4102d10 2024-03-23T15:40:04.560 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:38130/0 (socket says 172.21.15.192:38130) 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 -- 172.21.15.192:0/2707085350 learned_addr learned my addr 172.21.15.192:0/2707085350 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef27fc700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef4105640 0x7f8ef4100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef37fe700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef4107f70 0x7f8ef4101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 -- 172.21.15.192:0/2707085350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef4105640 msgr2=0x7f8ef4100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef4105640 0x7f8ef4100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 -- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef4107f70 msgr2=0x7f8ef4101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef4107f70 0x7f8ef4101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:04.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.556+0000 7f8ef2ffd700 1 -- 172.21.15.192:0/2707085350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee80095f0 con 0x7f8ef4102d10 2024-03-23T15:40:04.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.557+0000 7f8ef2ffd700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4102670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b650 tx=0x7f8ee800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:04.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.557+0000 7f8edbfff700 1 -- 172.21.15.192:0/2707085350 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee801d070 con 0x7f8ef4102d10 2024-03-23T15:40:04.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.557+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef41a2530 con 0x7f8ef4102d10 2024-03-23T15:40:04.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.557+0000 7f8edbfff700 1 -- 172.21.15.192:0/2707085350 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee800be60 con 0x7f8ef4102d10 2024-03-23T15:40:04.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.557+0000 7f8edbfff700 1 -- 172.21.15.192:0/2707085350 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8017610 con 0x7f8ef4102d10 2024-03-23T15:40:04.562 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.557+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41a2a20 con 0x7f8ef4102d10 2024-03-23T15:40:04.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.558+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef40fe240 con 0x7f8ef4102d10 2024-03-23T15:40:04.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.559+0000 7f8edbfff700 1 -- 172.21.15.192:0/2707085350 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f8ee8017850 con 0x7f8ef4102d10 2024-03-23T15:40:04.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.559+0000 7f8edbfff700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8edc0a2150 0x7f8edc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:04.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.560+0000 7f8edbfff700 1 -- 172.21.15.192:0/2707085350 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f8ee80cec40 con 0x7f8ef4102d10 2024-03-23T15:40:04.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.560+0000 7f8ef27fc700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8edc0a2150 0x7f8edc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:04.564 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.560+0000 7f8ef27fc700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8edc0a2150 0x7f8edc0a4610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006fd0 tx=0x7f8ee4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:04.565 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.563+0000 7f8edbfff700 1 -- 172.21.15.192:0/2707085350 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8ee80982d0 con 0x7f8ef4102d10 2024-03-23T15:40:04.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:04.766+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8ef4062640 con 0x7f8edc0a2150 2024-03-23T15:40:05.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:04 smithi192 ceph-mon[36653]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T15:40:05.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:40:05.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:40:05.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:05.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:04 smithi183 ceph-mon[36296]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T15:40:05.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:40:05.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:40:05.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:05.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:04 smithi076 ceph-mon[32176]: osdmap e74: 11 total, 11 up, 11 in 2024-03-23T15:40:05.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T15:40:05.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T15:40:05.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:06.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:05 smithi192 ceph-mon[36653]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:40:06.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:05 smithi192 ceph-mon[36653]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T15:40:06.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:05 smithi192 ceph-mon[36653]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T15:40:06.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:05 smithi183 ceph-mon[36296]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:40:06.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:05 smithi183 ceph-mon[36296]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T15:40:06.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:05 smithi183 ceph-mon[36296]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T15:40:06.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:05 smithi076 ceph-mon[32176]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:40:06.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:05 smithi076 ceph-mon[32176]: osdmap e75: 11 total, 11 up, 11 in 2024-03-23T15:40:06.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:05 smithi076 ceph-mon[32176]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T15:40:07.067 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:06 smithi192 ceph-mon[36653]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T15:40:07.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:06 smithi183 ceph-mon[36296]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T15:40:07.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:06 smithi076 ceph-mon[32176]: osdmap e76: 11 total, 11 up, 11 in 2024-03-23T15:40:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:07 smithi183 ceph-mon[36296]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-23T15:40:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:07 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/1443423613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]: dispatch 2024-03-23T15:40:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:07 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]: dispatch 2024-03-23T15:40:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:07 smithi183 ceph-mon[36296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]': finished 2024-03-23T15:40:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:07 smithi183 ceph-mon[36296]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T15:40:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:08.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:07 smithi076 ceph-mon[32176]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-23T15:40:08.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:07 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/1443423613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]: dispatch 2024-03-23T15:40:08.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:07 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]: dispatch 2024-03-23T15:40:08.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:07 smithi076 ceph-mon[32176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]': finished 2024-03-23T15:40:08.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:07 smithi076 ceph-mon[32176]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T15:40:08.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:08.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:07 smithi192 ceph-mon[36653]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-23T15:40:08.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:07 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/1443423613' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]: dispatch 2024-03-23T15:40:08.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:07 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]: dispatch 2024-03-23T15:40:08.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:07 smithi192 ceph-mon[36653]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab72b35e-6449-42d0-9a9b-7014e9586173"}]': finished 2024-03-23T15:40:08.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:07 smithi192 ceph-mon[36653]: osdmap e77: 12 total, 11 up, 12 in 2024-03-23T15:40:08.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:09.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.192:0/4208318880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:40:09.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:08 smithi192 ceph-mon[36653]: from='client.? 172.21.15.192:0/4208318880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:40:09.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:08 smithi076 ceph-mon[32176]: from='client.? 172.21.15.192:0/4208318880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T15:40:10.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:09 smithi183 ceph-mon[36296]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-23T15:40:10.322 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:09 smithi192 ceph-mon[36653]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-23T15:40:10.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:09 smithi076 ceph-mon[32176]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-23T15:40:12.818 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:12 smithi192 ceph-mon[36653]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:12 smithi076 ceph-mon[32176]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:12 smithi183 ceph-mon[36296]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:14.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:14 smithi192 ceph-mon[36653]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:14.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:14 smithi076 ceph-mon[32176]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:14 smithi183 ceph-mon[36296]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:15.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T15:40:15.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:15.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T15:40:15.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-23T15:40:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:16 smithi192 ceph-mon[36653]: Deploying daemon osd.11 on smithi192 2024-03-23T15:40:16.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:16 smithi192 ceph-mon[36653]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:16 smithi076 ceph-mon[32176]: Deploying daemon osd.11 on smithi192 2024-03-23T15:40:16.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:16 smithi076 ceph-mon[32176]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:16 smithi183 ceph-mon[36296]: Deploying daemon osd.11 on smithi192 2024-03-23T15:40:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:16 smithi183 ceph-mon[36296]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:18.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:18 smithi192 ceph-mon[36653]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:18.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:18 smithi076 ceph-mon[32176]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:18.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:18 smithi183 ceph-mon[36296]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:20.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:20 smithi192 ceph-mon[36653]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:20.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:20 smithi076 ceph-mon[32176]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:20.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:20 smithi183 ceph-mon[36296]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:22.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:22 smithi192 ceph-mon[36653]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:22.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:22 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:22.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:22 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:22.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:22 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:22 smithi076 ceph-mon[32176]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:22.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:22 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:22.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:22 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:22.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:22 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:22 smithi183 ceph-mon[36296]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:22 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:22 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:22 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:24.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:24 smithi192 ceph-mon[36653]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:24.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:24 smithi076 ceph-mon[32176]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:24 smithi183 ceph-mon[36296]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:25.689 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2024-03-23T15:40:25.689 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.686+0000 7f8edbfff700 1 -- 172.21.15.192:0/2707085350 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8ef4062640 con 0x7f8edc0a2150 2024-03-23T15:40:25.691 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8edc0a2150 msgr2=0x7f8edc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:25.691 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8edc0a2150 0x7f8edc0a4610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006fd0 tx=0x7f8ee4008040 comp rx=0 tx=0).stop 2024-03-23T15:40:25.691 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 msgr2=0x7f8ef4102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4102670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b650 tx=0x7f8ee800f740 comp rx=0 tx=0).stop 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 shutdown_connections 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8edc0a2150 0x7f8edc0a4610 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef4107f70 0x7f8ef4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ef4102d10 0x7f8ef4102670 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 --2- 172.21.15.192:0/2707085350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef4105640 0x7f8ef4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 >> 172.21.15.192:0/2707085350 conn(0x7f8ef40fc6c0 msgr2=0x7f8ef4103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 shutdown_connections 2024-03-23T15:40:25.692 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:25.689+0000 7f8ef9347700 1 -- 172.21.15.192:0/2707085350 wait complete. 2024-03-23T15:40:26.006 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.006 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.006 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.007 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:25 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:25 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-23T15:40:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:26.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:26.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:25 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:26.350 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.11.service 2024-03-23T15:40:26.353 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-23T15:40:26.353 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-23T15:40:26.598 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:40:27.032 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:26 smithi076 ceph-mon[32176]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T15:40:27.033 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:26 smithi076 ceph-mon[32176]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T15:40:27.033 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:26 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:27.033 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:26 smithi076 ceph-mon[32176]: from='osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:40:27.033 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:26 smithi076 ceph-mon[32176]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:40:27.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:26 smithi192 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T15:40:27.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:26 smithi192 ceph-mon[36653]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T15:40:27.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:26 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:27.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:26 smithi192 ceph-mon[36653]: from='osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:40:27.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:26 smithi192 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:40:27.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:26 smithi183 ceph-mon[36296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-23T15:40:27.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:26 smithi183 ceph-mon[36296]: osdmap e78: 12 total, 11 up, 12 in 2024-03-23T15:40:27.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:26 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:27.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:26 smithi183 ceph-mon[36296]: from='osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:40:27.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:26 smithi183 ceph-mon[36296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-03-23T15:40:27.566 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:40:27.262+0000 7f6c8e38c700 -1 osd.11 0 waiting for initial osdmap 2024-03-23T15:40:27.566 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:40:27.276+0000 7f6c86177700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T15:40:28.012 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:27 smithi076 ceph-mon[32176]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:40:28.012 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:27 smithi076 ceph-mon[32176]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T15:40:28.013 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:28.013 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:27 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:28.013 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:27 smithi076 ceph-mon[32176]: from='osd.11 ' entity='osd.11' 2024-03-23T15:40:28.013 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:27 smithi076 ceph-mon[32176]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:28.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-mon[36653]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:40:28.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-mon[36653]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T15:40:28.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:28.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:28.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-mon[36653]: from='osd.11 ' entity='osd.11' 2024-03-23T15:40:28.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:27 smithi192 ceph-mon[36653]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:27 smithi183 ceph-mon[36296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-03-23T15:40:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:27 smithi183 ceph-mon[36296]: osdmap e79: 12 total, 11 up, 12 in 2024-03-23T15:40:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:27 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:27 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:27 smithi183 ceph-mon[36296]: from='osd.11 ' entity='osd.11' 2024-03-23T15:40:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:27 smithi183 ceph-mon[36296]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-03-23T15:40:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 -- 172.21.15.76:0/2672844673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0103430 msgr2=0x7f4da0103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 --2- 172.21.15.76:0/2672844673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0103430 0x7f4da0103890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4d900099c0 tx=0x7f4d90009cd0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 -- 172.21.15.76:0/2672844673 shutdown_connections 2024-03-23T15:40:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 --2- 172.21.15.76:0/2672844673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4da0103dd0 0x7f4da010a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 --2- 172.21.15.76:0/2672844673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da0102230 0x7f4da0102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 --2- 172.21.15.76:0/2672844673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0103430 0x7f4da0103890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 -- 172.21.15.76:0/2672844673 >> 172.21.15.76:0/2672844673 conn(0x7f4da00fd7b0 msgr2=0x7f4da00ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 -- 172.21.15.76:0/2672844673 shutdown_connections 2024-03-23T15:40:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.379+0000 7f4da522b700 1 -- 172.21.15.76:0/2672844673 wait complete. 2024-03-23T15:40:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 Processor -- start 2024-03-23T15:40:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 -- start start 2024-03-23T15:40:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0102230 0x7f4da0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da0103430 0x7f4da0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4da0103dd0 0x7f4da019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4da019fd90 con 0x7f4da0102230 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4da019fed0 con 0x7f4da0103dd0 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4da522b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01a0060 con 0x7f4da0103430 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9ed9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0102230 0x7f4da0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9e59c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da0103430 0x7f4da0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9ed9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0102230 0x7f4da0198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36354/0 (socket says 172.21.15.76:36354) 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9ed9d700 1 -- 172.21.15.76:0/2640885355 learned_addr learned my addr 172.21.15.76:0/2640885355 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9f59e700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4da0103dd0 0x7f4da019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9e59c700 1 -- 172.21.15.76:0/2640885355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4da0103dd0 msgr2=0x7f4da019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9e59c700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4da0103dd0 0x7f4da019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9e59c700 1 -- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0102230 msgr2=0x7f4da0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9e59c700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0102230 0x7f4da0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9e59c700 1 -- 172.21.15.76:0/2640885355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d900095f0 con 0x7f4da0103430 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.380+0000 7f4d9ed9d700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0102230 0x7f4da0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.381+0000 7f4d9e59c700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da0103430 0x7f4da0198740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009f50 tx=0x7f4d90004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.381+0000 7f4d87fff700 1 -- 172.21.15.76:0/2640885355 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9001d070 con 0x7f4da0103430 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.381+0000 7f4d87fff700 1 -- 172.21.15.76:0/2640885355 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d90004050 con 0x7f4da0103430 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.381+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da01a02e0 con 0x7f4da0103430 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.381+0000 7f4d87fff700 1 -- 172.21.15.76:0/2640885355 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d900047e0 con 0x7f4da0103430 2024-03-23T15:40:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.381+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da01a0770 con 0x7f4da0103430 2024-03-23T15:40:28.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.382+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da01095f0 con 0x7f4da0103430 2024-03-23T15:40:28.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.383+0000 7f4d87fff700 1 -- 172.21.15.76:0/2640885355 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f4d90003b70 con 0x7f4da0103430 2024-03-23T15:40:28.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.384+0000 7f4d87fff700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d880a21d0 0x7f4d880a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:28.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.384+0000 7f4d9ed9d700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d880a21d0 0x7f4d880a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:28.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.384+0000 7f4d87fff700 1 -- 172.21.15.76:0/2640885355 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f4d900ce290 con 0x7f4da0103430 2024-03-23T15:40:28.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.384+0000 7f4d9ed9d700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d880a21d0 0x7f4d880a4690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d940105c0 tx=0x7f4d9400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:28.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.388+0000 7f4d87fff700 1 -- 172.21.15.76:0/2640885355 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4d90097790 con 0x7f4da0103430 2024-03-23T15:40:28.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.593+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4da010dcc0 con 0x7f4da0103430 2024-03-23T15:40:28.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.594+0000 7f4d87fff700 1 -- 172.21.15.76:0/2640885355 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f4d9009ae50 con 0x7f4da0103430 2024-03-23T15:40:28.596 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:28.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d880a21d0 msgr2=0x7f4d880a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:28.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d880a21d0 0x7f4d880a4690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d940105c0 tx=0x7f4d9400b4a0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da0103430 msgr2=0x7f4da0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:28.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da0103430 0x7f4da0198740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009f50 tx=0x7f4d90004730 comp rx=0 tx=0).stop 2024-03-23T15:40:28.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 shutdown_connections 2024-03-23T15:40:28.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4d880a21d0 0x7f4d880a4690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4da0103dd0 0x7f4da019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4da0102230 0x7f4da0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 --2- 172.21.15.76:0/2640885355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da0103430 0x7f4da0198740 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:28.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 >> 172.21.15.76:0/2640885355 conn(0x7f4da00fd7b0 msgr2=0x7f4da0106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:28.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 shutdown_connections 2024-03-23T15:40:28.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:28.596+0000 7f4da522b700 1 -- 172.21.15.76:0/2640885355 wait complete. 2024-03-23T15:40:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:28 smithi192 ceph-mon[36653]: purged_snaps scrub starts 2024-03-23T15:40:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:28 smithi192 ceph-mon[36653]: purged_snaps scrub ok 2024-03-23T15:40:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:28 smithi192 ceph-mon[36653]: osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] boot 2024-03-23T15:40:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:28 smithi192 ceph-mon[36653]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T15:40:29.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:28 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:29.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:28 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2640885355' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T15:40:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:28 smithi076 ceph-mon[32176]: purged_snaps scrub starts 2024-03-23T15:40:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:28 smithi076 ceph-mon[32176]: purged_snaps scrub ok 2024-03-23T15:40:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:28 smithi076 ceph-mon[32176]: osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] boot 2024-03-23T15:40:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:28 smithi076 ceph-mon[32176]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T15:40:29.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:28 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:29.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:28 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2640885355' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T15:40:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:28 smithi183 ceph-mon[36296]: purged_snaps scrub starts 2024-03-23T15:40:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:28 smithi183 ceph-mon[36296]: purged_snaps scrub ok 2024-03-23T15:40:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:28 smithi183 ceph-mon[36296]: osd.11 [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] boot 2024-03-23T15:40:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:28 smithi183 ceph-mon[36296]: osdmap e80: 12 total, 12 up, 12 in 2024-03-23T15:40:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:28 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:40:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:28 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2640885355' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T15:40:29.366 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1711208428,"num_in_osds":12,"osd_in_since":1711208407,"num_remapped_pgs":0} 2024-03-23T15:40:29.367 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T15:40:29.610 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:40:30.538 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:30.539 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:30 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T15:40:30.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:30.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:30.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:30.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:30 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: osdmap e81: 12 total, 12 up, 12 in 2024-03-23T15:40:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:40:30.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:30.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:30.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:30.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:30 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:31.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.413+0000 7f7412591700 1 -- 172.21.15.76:0/1445260404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c101c50 msgr2=0x7f740c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:31.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.413+0000 7f7412591700 1 --2- 172.21.15.76:0/1445260404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c101c50 0x7f740c1020d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7400009990 tx=0x7f7400009ca0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 -- 172.21.15.76:0/1445260404 shutdown_connections 2024-03-23T15:40:31.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 --2- 172.21.15.76:0/1445260404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f740c102610 0x7f740c10dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 --2- 172.21.15.76:0/1445260404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c106140 0x7f740c106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 --2- 172.21.15.76:0/1445260404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c101c50 0x7f740c1020d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 -- 172.21.15.76:0/1445260404 >> 172.21.15.76:0/1445260404 conn(0x7f740c076440 msgr2=0x7f740c076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 -- 172.21.15.76:0/1445260404 shutdown_connections 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 -- 172.21.15.76:0/1445260404 wait complete. 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 Processor -- start 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.414+0000 7f7412591700 1 -- start start 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c101c50 0x7f740c102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f740c102610 0x7f740c103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c106140 0x7f740c1038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a6b20 con 0x7f740c106140 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a6c90 con 0x7f740c102610 2024-03-23T15:40:31.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f740c1a6e50 con 0x7f740c101c50 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c106140 0x7f740c1038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c106140 0x7f740c1038c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36364/0 (socket says 172.21.15.76:36364) 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 -- 172.21.15.76:0/390785967 learned_addr learned my addr 172.21.15.76:0/390785967 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f740b7fe700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f740c102610 0x7f740c103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f740bfff700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c101c50 0x7f740c102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 -- 172.21.15.76:0/390785967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c101c50 msgr2=0x7f740c102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c101c50 0x7f740c102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 -- 172.21.15.76:0/390785967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f740c102610 msgr2=0x7f740c103380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f740c102610 0x7f740c103380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 -- 172.21.15.76:0/390785967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74000095f0 con 0x7f740c106140 2024-03-23T15:40:31.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f740bfff700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c101c50 0x7f740c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:40:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7410b2e700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c106140 0x7f740c1038c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f740400ef30 tx=0x7f740400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f74097fa700 1 -- 172.21.15.76:0/390785967 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404018070 con 0x7f740c106140 2024-03-23T15:40:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f74097fa700 1 -- 172.21.15.76:0/390785967 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740400f040 con 0x7f740c106140 2024-03-23T15:40:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f74097fa700 1 -- 172.21.15.76:0/390785967 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404003e70 con 0x7f740c106140 2024-03-23T15:40:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c1a70d0 con 0x7f740c106140 2024-03-23T15:40:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.415+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c1a7580 con 0x7f740c106140 2024-03-23T15:40:31.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.416+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f740c068ec0 con 0x7f740c106140 2024-03-23T15:40:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.417+0000 7f74097fa700 1 -- 172.21.15.76:0/390785967 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f7404004ad0 con 0x7f740c106140 2024-03-23T15:40:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.418+0000 7f74097fa700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f73f40a2150 0x7f73f40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.418+0000 7f740bfff700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f73f40a2150 0x7f73f40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.418+0000 7f74097fa700 1 -- 172.21.15.76:0/390785967 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f74040cde80 con 0x7f740c106140 2024-03-23T15:40:31.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.418+0000 7f740bfff700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f73f40a2150 0x7f73f40a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009c90 tx=0x7f73fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:31.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.422+0000 7f74097fa700 1 -- 172.21.15.76:0/390785967 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7404097380 con 0x7f740c106140 2024-03-23T15:40:31.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:31 smithi076 ceph-mon[32176]: Detected new or changed devices on smithi192 2024-03-23T15:40:31.424 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:31 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi192 to 4270M 2024-03-23T15:40:31.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:31 smithi192 ceph-mon[36653]: Detected new or changed devices on smithi192 2024-03-23T15:40:31.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:31 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi192 to 4270M 2024-03-23T15:40:31.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.617+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f740c1111e0 con 0x7f740c106140 2024-03-23T15:40:31.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.618+0000 7f74097fa700 1 -- 172.21.15.76:0/390785967 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18440 (secure 0 0 0) 0x7f740409aa40 con 0x7f740c106140 2024-03-23T15:40:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:31.621 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":81,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:40:29.276449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:40:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f73f40a2150 msgr2=0x7f73f40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f73f40a2150 0x7f73f40a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009c90 tx=0x7f73fc009260 comp rx=0 tx=0).stop 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c106140 msgr2=0x7f740c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c106140 0x7f740c1038c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f740400ef30 tx=0x7f740400c450 comp rx=0 tx=0).stop 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 shutdown_connections 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f73f40a2150 0x7f73f40a4610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f740c106140 0x7f740c1038c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f740c101c50 0x7f740c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 --2- 172.21.15.76:0/390785967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f740c102610 0x7f740c103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 >> 172.21.15.76:0/390785967 conn(0x7f740c076440 msgr2=0x7f740c100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 shutdown_connections 2024-03-23T15:40:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:31.620+0000 7f7412591700 1 -- 172.21.15.76:0/390785967 wait complete. 2024-03-23T15:40:31.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:31 smithi183 ceph-mon[36296]: Detected new or changed devices on smithi192 2024-03-23T15:40:31.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:31 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi192 to 4270M 2024-03-23T15:40:32.291 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:40:32.291 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-23T15:40:32.530 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:40:32.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:32 smithi192 ceph-mon[36653]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:32.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:32 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/390785967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:40:32.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:32 smithi076 ceph-mon[32176]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:32.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:32 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/390785967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:40:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:32 smithi183 ceph-mon[36296]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:32.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:32 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/390785967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.182+0000 7fb39a0a0700 1 -- 172.21.15.76:0/3988926716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 msgr2=0x7fb3941028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.182+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/3988926716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941028c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb3840099c0 tx=0x7fb384009cd0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 -- 172.21.15.76:0/3988926716 shutdown_connections 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/3988926716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb394107590 0x7fb39410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/3988926716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941028c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/3988926716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb394102e90 0x7fb394106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 -- 172.21.15.76:0/3988926716 >> 172.21.15.76:0/3988926716 conn(0x7fb3940792d0 msgr2=0x7fb3940796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:34.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 -- 172.21.15.76:0/3988926716 shutdown_connections 2024-03-23T15:40:34.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 -- 172.21.15.76:0/3988926716 wait complete. 2024-03-23T15:40:34.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 Processor -- start 2024-03-23T15:40:34.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.183+0000 7fb39a0a0700 1 -- start start 2024-03-23T15:40:34.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb394102e90 0x7fb394198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb394107590 0x7fb39419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb39419d530 con 0x7fb3941024e0 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb39419d6a0 con 0x7fb394107590 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb39419d860 con 0x7fb394102e90 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36382/0 (socket says 172.21.15.76:36382) 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 -- 172.21.15.76:0/2855529754 learned_addr learned my addr 172.21.15.76:0/2855529754 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb393fff700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb394107590 0x7fb39419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb392ffd700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb394102e90 0x7fb394198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 -- 172.21.15.76:0/2855529754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb394102e90 msgr2=0x7fb394198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb394102e90 0x7fb394198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 -- 172.21.15.76:0/2855529754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb394107590 msgr2=0x7fb39419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb394107590 0x7fb39419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 -- 172.21.15.76:0/2855529754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3840095f0 con 0x7fb3941024e0 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb3937fe700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941986a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb384005720 tx=0x7fb384005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb390ff9700 1 -- 172.21.15.76:0/2855529754 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38401d070 con 0x7fb3941024e0 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39419dae0 con 0x7fb3941024e0 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb390ff9700 1 -- 172.21.15.76:0/2855529754 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb384003f50 con 0x7fb3941024e0 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb390ff9700 1 -- 172.21.15.76:0/2855529754 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384022d30 con 0x7fb3941024e0 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.184+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3941a53a0 con 0x7fb3941024e0 2024-03-23T15:40:34.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.185+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39410a7d0 con 0x7fb3941024e0 2024-03-23T15:40:34.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.187+0000 7fb390ff9700 1 -- 172.21.15.76:0/2855529754 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fb384003a70 con 0x7fb3941024e0 2024-03-23T15:40:34.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.188+0000 7fb390ff9700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb37c0a2150 0x7fb37c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:34.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.188+0000 7fb392ffd700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb37c0a2150 0x7fb37c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:34.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.188+0000 7fb390ff9700 1 -- 172.21.15.76:0/2855529754 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb3840cf430 con 0x7fb3941024e0 2024-03-23T15:40:34.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.188+0000 7fb392ffd700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb37c0a2150 0x7fb37c0a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb394108850 tx=0x7fb38800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:34.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.191+0000 7fb390ff9700 1 -- 172.21.15.76:0/2855529754 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb384098930 con 0x7fb3941024e0 2024-03-23T15:40:34.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.387+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3941999b0 con 0x7fb3941024e0 2024-03-23T15:40:34.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.387+0000 7fb390ff9700 1 -- 172.21.15.76:0/2855529754 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fb38409bff0 con 0x7fb3941024e0 2024-03-23T15:40:34.391 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:40:34.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb37c0a2150 msgr2=0x7fb37c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb37c0a2150 0x7fb37c0a4610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb394108850 tx=0x7fb38800b5b0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 msgr2=0x7fb3941986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941986a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb384005720 tx=0x7fb384005800 comp rx=0 tx=0).stop 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 shutdown_connections 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb37c0a2150 0x7fb37c0a4610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb394107590 0x7fb39419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3941024e0 0x7fb3941986a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 --2- 172.21.15.76:0/2855529754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb394102e90 0x7fb394198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.389+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 >> 172.21.15.76:0/2855529754 conn(0x7fb3940792d0 msgr2=0x7fb3941002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.390+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 shutdown_connections 2024-03-23T15:40:34.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:34.390+0000 7fb39a0a0700 1 -- 172.21.15.76:0/2855529754 wait complete. 2024-03-23T15:40:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:34 smithi192 ceph-mon[36653]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:34 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2855529754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:40:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:34 smithi076 ceph-mon[32176]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:34 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2855529754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:40:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:34 smithi183 ceph-mon[36296]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:34 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2855529754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.a on smithi076 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.d on smithi076 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.g on smithi076 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.j on smithi076 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.b on smithi183 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.e on smithi183 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.h on smithi183 2024-03-23T15:40:35.147 INFO:tasks.cephadm:Adding mds.k on smithi183 2024-03-23T15:40:35.148 INFO:tasks.cephadm:Adding mds.c on smithi192 2024-03-23T15:40:35.148 INFO:tasks.cephadm:Adding mds.f on smithi192 2024-03-23T15:40:35.148 INFO:tasks.cephadm:Adding mds.i on smithi192 2024-03-23T15:40:35.148 INFO:tasks.cephadm:Adding mds.l on smithi192 2024-03-23T15:40:35.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2024-03-23T15:40:35.398 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:40:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:36 smithi192 ceph-mon[36653]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:36.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:36 smithi076 ceph-mon[32176]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:36.837 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:36 smithi183 ceph-mon[36296]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:37.095 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 -- 172.21.15.192:0/4063061809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 msgr2=0x7f3f44101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:37.095 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 --2- 172.21.15.192:0/4063061809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f44101580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009990 tx=0x7f3f34009ca0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.095 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 -- 172.21.15.192:0/4063061809 shutdown_connections 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 --2- 172.21.15.192:0/4063061809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f44102d00 0x7f3f441095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 --2- 172.21.15.192:0/4063061809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f44101580 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 --2- 172.21.15.192:0/4063061809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f44102360 0x7f3f441027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 -- 172.21.15.192:0/4063061809 >> 172.21.15.192:0/4063061809 conn(0x7f3f440fc720 msgr2=0x7f3f440feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 -- 172.21.15.192:0/4063061809 shutdown_connections 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.089+0000 7f3f49818700 1 -- 172.21.15.192:0/4063061809 wait complete. 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 Processor -- start 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 -- start start 2024-03-23T15:40:37.096 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f4406b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f44102360 0x7f3f4419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f44102d00 0x7f3f4419f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4419fd20 con 0x7f3f44102d00 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4419fe90 con 0x7f3f44102360 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.090+0000 7f3f49818700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f441a0050 con 0x7f3f44101160 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f4406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f4406b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:58692/0 (socket says 172.21.15.192:58692) 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 -- 172.21.15.192:0/425404486 learned_addr learned my addr 172.21.15.192:0/425404486 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f427fc700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f44102360 0x7f3f4419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:37.097 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f437fe700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f44102d00 0x7f3f4419f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 -- 172.21.15.192:0/425404486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f44102360 msgr2=0x7f3f4419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f44102360 0x7f3f4419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 -- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f44102d00 msgr2=0x7f3f4419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f44102d00 0x7f3f4419f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 -- 172.21.15.192:0/425404486 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f340095f0 con 0x7f3f44101160 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f42ffd700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f4406b440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f340095c0 tx=0x7f3f3400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f48816700 1 -- 172.21.15.192:0/425404486 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3401d070 con 0x7f3f44101160 2024-03-23T15:40:37.098 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f441a02d0 con 0x7f3f44101160 2024-03-23T15:40:37.099 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f48816700 1 -- 172.21.15.192:0/425404486 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f3400fb80 con 0x7f3f44101160 2024-03-23T15:40:37.099 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f48816700 1 -- 172.21.15.192:0/425404486 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340173f0 con 0x7f3f44101160 2024-03-23T15:40:37.099 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.091+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f441a0740 con 0x7f3f44101160 2024-03-23T15:40:37.099 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.092+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f441085f0 con 0x7f3f44101160 2024-03-23T15:40:37.100 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.093+0000 7f3f48816700 1 -- 172.21.15.192:0/425404486 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f3f3400fcf0 con 0x7f3f44101160 2024-03-23T15:40:37.100 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.094+0000 7f3f48816700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3f2c0a2150 0x7f3f2c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:37.100 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.094+0000 7f3f48816700 1 -- 172.21.15.192:0/425404486 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3f340ce9e0 con 0x7f3f44101160 2024-03-23T15:40:37.100 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.094+0000 7f3f427fc700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3f2c0a2150 0x7f3f2c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:37.100 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.095+0000 7f3f427fc700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3f2c0a2150 0x7f3f2c0a4610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3f4406b9c0 tx=0x7f3f3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:37.101 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.098+0000 7f3f48816700 1 -- 172.21.15.192:0/425404486 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f3f34097ee0 con 0x7f3f44101160 2024-03-23T15:40:37.303 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.300+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) -- 0x7f3f4402dce0 con 0x7f3f2c0a2150 2024-03-23T15:40:37.311 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2024-03-23T15:40:37.311 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.308+0000 7f3f48816700 1 -- 172.21.15.192:0/425404486 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3f4402dce0 con 0x7f3f2c0a2150 2024-03-23T15:40:37.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.310+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3f2c0a2150 msgr2=0x7f3f2c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:37.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.310+0000 7f3f49818700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3f2c0a2150 0x7f3f2c0a4610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3f4406b9c0 tx=0x7f3f3800b410 comp rx=0 tx=0).stop 2024-03-23T15:40:37.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 msgr2=0x7f3f4406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:37.313 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f4406b440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f340095c0 tx=0x7f3f3400f690 comp rx=0 tx=0).stop 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 shutdown_connections 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3f2c0a2150 0x7f3f2c0a4610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f44102d00 0x7f3f4419f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f44101160 0x7f3f4406b440 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 --2- 172.21.15.192:0/425404486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f44102360 0x7f3f4419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 >> 172.21.15.192:0/425404486 conn(0x7f3f440fc720 msgr2=0x7f3f44105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 shutdown_connections 2024-03-23T15:40:37.314 INFO:teuthology.orchestra.run.smithi192.stderr:2024-03-23T15:40:37.311+0000 7f3f49818700 1 -- 172.21.15.192:0/425404486 wait complete. 2024-03-23T15:40:38.036 DEBUG:teuthology.orchestra.run.smithi076:mds.a> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.a.service 2024-03-23T15:40:38.039 DEBUG:teuthology.orchestra.run.smithi076:mds.d> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.d.service 2024-03-23T15:40:38.082 DEBUG:teuthology.orchestra.run.smithi076:mds.g> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.g.service 2024-03-23T15:40:38.126 DEBUG:teuthology.orchestra.run.smithi076:mds.j> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.j.service 2024-03-23T15:40:38.169 DEBUG:teuthology.orchestra.run.smithi183:mds.b> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.b.service 2024-03-23T15:40:38.171 DEBUG:teuthology.orchestra.run.smithi183:mds.e> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.e.service 2024-03-23T15:40:38.214 DEBUG:teuthology.orchestra.run.smithi183:mds.h> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.h.service 2024-03-23T15:40:38.257 DEBUG:teuthology.orchestra.run.smithi183:mds.k> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.k.service 2024-03-23T15:40:38.300 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.c.service 2024-03-23T15:40:38.302 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.f.service 2024-03-23T15:40:38.345 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.i.service 2024-03-23T15:40:38.388 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.l.service 2024-03-23T15:40:38.431 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-23T15:40:38.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 -- 172.21.15.76:0/3081389649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674100650 msgr2=0x7f3674100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 --2- 172.21.15.76:0/3081389649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674100650 0x7f3674100a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3664009900 tx=0x7f366402bc30 comp rx=0 tx=0).stop 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 -- 172.21.15.76:0/3081389649 shutdown_connections 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 --2- 172.21.15.76:0/3081389649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3674102200 0x7f3674108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 --2- 172.21.15.76:0/3081389649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3674101840 0x7f3674101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 --2- 172.21.15.76:0/3081389649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674100650 0x7f3674100a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 -- 172.21.15.76:0/3081389649 >> 172.21.15.76:0/3081389649 conn(0x7f36740fbbc0 msgr2=0x7f36740fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 -- 172.21.15.76:0/3081389649 shutdown_connections 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.543+0000 7f36797af640 1 -- 172.21.15.76:0/3081389649 wait complete. 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 Processor -- start 2024-03-23T15:40:38.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 -- start start 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3674100650 0x7f367406a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3674101840 0x7f367419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674102200 0x7f367419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f367419f310 con 0x7f3674102200 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f367419f480 con 0x7f3674101840 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36797af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f367419f640 con 0x7f3674100650 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674102200 0x7f367419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f3672ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3674100650 0x7f367406a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674102200 0x7f367419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57282/0 (socket says 172.21.15.76:57282) 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f36737fe640 1 -- 172.21.15.76:0/2340503930 learned_addr learned my addr 172.21.15.76:0/2340503930 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f3672ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3674100650 0x7f367406a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:56504/0 (socket says 172.21.15.76:56504) 2024-03-23T15:40:38.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.544+0000 7f366bfff640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3674101840 0x7f367419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36737fe640 1 -- 172.21.15.76:0/2340503930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3674100650 msgr2=0x7f367406a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36737fe640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3674100650 0x7f367406a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36737fe640 1 -- 172.21.15.76:0/2340503930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3674101840 msgr2=0x7f367419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36737fe640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3674101840 0x7f367419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36737fe640 1 -- 172.21.15.76:0/2340503930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3664009560 con 0x7f3674102200 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f366bfff640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3674101840 0x7f367419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36737fe640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674102200 0x7f367419eb80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f366c00eaa0 tx=0x7f366c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f3670ff9640 1 -- 172.21.15.76:0/2340503930 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c00cdb0 con 0x7f3674102200 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367419f920 con 0x7f3674102200 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f3670ff9640 1 -- 172.21.15.76:0/2340503930 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366c005020 con 0x7f3674102200 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f3670ff9640 1 -- 172.21.15.76:0/2340503930 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c0116b0 con 0x7f3674102200 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.545+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367419fe30 con 0x7f3674102200 2024-03-23T15:40:38.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.546+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3674107970 con 0x7f3674102200 2024-03-23T15:40:38.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.547+0000 7f3670ff9640 1 -- 172.21.15.76:0/2340503930 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f366c005520 con 0x7f3674102200 2024-03-23T15:40:38.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.547+0000 7f3670ff9640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f36600a20c0 0x7f36600a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.547+0000 7f3672ffd640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f36600a20c0 0x7f36600a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.547+0000 7f3670ff9640 1 -- 172.21.15.76:0/2340503930 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f366c0cda30 con 0x7f3674102200 2024-03-23T15:40:38.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.548+0000 7f3672ffd640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f36600a20c0 0x7f36600a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3664005ab0 tx=0x7f3664005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:38.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.552+0000 7f3670ff9640 1 -- 172.21.15.76:0/2340503930 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f366c096ea0 con 0x7f3674102200 2024-03-23T15:40:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:40:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: Saving service mds.all spec with placement smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-23T15:40:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:38.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:38.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:38.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:38.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:38.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:38 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:38.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:40:38.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: Saving service mds.all spec with placement smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-23T15:40:38.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:38.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:38.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:38.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:38.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:38.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:38.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:38.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:38.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:38 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: Saving service mds.all spec with placement smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:38.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:38 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:38.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.729+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f367410c260 con 0x7f3674102200 2024-03-23T15:40:38.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.730+0000 7f3670ff9640 1 -- 172.21.15.76:0/2340503930 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18440 (secure 0 0 0) 0x7f366c09a560 con 0x7f3674102200 2024-03-23T15:40:38.733 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:38.734 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":81,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:40:29.276449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:40:38.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f36600a20c0 msgr2=0x7f36600a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f36600a20c0 0x7f36600a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3664005ab0 tx=0x7f3664005fd0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674102200 msgr2=0x7f367419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674102200 0x7f367419eb80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f366c00eaa0 tx=0x7f366c00ef60 comp rx=0 tx=0).stop 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 shutdown_connections 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3674102200 0x7f367419eb80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f36600a20c0 0x7f36600a4580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3674101840 0x7f367419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 --2- 172.21.15.76:0/2340503930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3674100650 0x7f367406a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 >> 172.21.15.76:0/2340503930 conn(0x7f36740fbbc0 msgr2=0x7f3674104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 shutdown_connections 2024-03-23T15:40:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.732+0000 7f36797af640 1 -- 172.21.15.76:0/2340503930 wait complete. 2024-03-23T15:40:38.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:40:38.747 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:40:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 -- 172.21.15.76:0/3726999020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368109630 msgr2=0x7f436810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 --2- 172.21.15.76:0/3726999020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368109630 0x7f436810bad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4360009ca0 tx=0x7f436002c190 comp rx=0 tx=0).stop 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 -- 172.21.15.76:0/3726999020 shutdown_connections 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 --2- 172.21.15.76:0/3726999020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368109630 0x7f436810bad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 --2- 172.21.15.76:0/3726999020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4368074110 0x7f4368074570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 --2- 172.21.15.76:0/3726999020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43680756f0 0x7f4368073b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 -- 172.21.15.76:0/3726999020 >> 172.21.15.76:0/3726999020 conn(0x7f43680fcd30 msgr2=0x7f43680ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 -- 172.21.15.76:0/3726999020 shutdown_connections 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.848+0000 7f436dca4640 1 -- 172.21.15.76:0/3726999020 wait complete. 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 Processor -- start 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 -- start start 2024-03-23T15:40:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368074110 0x7f43681998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43680756f0 0x7f4368199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4368109630 0x7f43681a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f43681a13c0 con 0x7f4368074110 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43681a1530 con 0x7f4368109630 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f436dca4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43681a16f0 con 0x7f43680756f0 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f43677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368074110 0x7f43681998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f43677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368074110 0x7f43681998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57304/0 (socket says 172.21.15.76:57304) 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.849+0000 7f43677fe640 1 -- 172.21.15.76:0/511075205 learned_addr learned my addr 172.21.15.76:0/511075205 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f435effd640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43680756f0 0x7f4368199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f4367fff640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4368109630 0x7f43681a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43677fe640 1 -- 172.21.15.76:0/511075205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43680756f0 msgr2=0x7f4368199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43677fe640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43680756f0 0x7f4368199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43677fe640 1 -- 172.21.15.76:0/511075205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4368109630 msgr2=0x7f43681a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:38.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43677fe640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4368109630 0x7f43681a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43677fe640 1 -- 172.21.15.76:0/511075205 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4360009900 con 0x7f4368074110 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f4367fff640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4368109630 0x7f43681a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43677fe640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368074110 0x7f43681998d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f435800c920 tx=0x7f435800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43657fa640 1 -- 172.21.15.76:0/511075205 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358007c70 con 0x7f4368074110 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43657fa640 1 -- 172.21.15.76:0/511075205 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4358007dd0 con 0x7f4368074110 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43681a19d0 con 0x7f4368074110 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f43657fa640 1 -- 172.21.15.76:0/511075205 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358018860 con 0x7f4368074110 2024-03-23T15:40:38.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.850+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43681a1ee0 con 0x7f4368074110 2024-03-23T15:40:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.851+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43681083f0 con 0x7f4368074110 2024-03-23T15:40:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.852+0000 7f43657fa640 1 -- 172.21.15.76:0/511075205 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f435800fac0 con 0x7f4368074110 2024-03-23T15:40:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.852+0000 7f43657fa640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f43540a20c0 0x7f43540a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.853+0000 7f435effd640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f43540a20c0 0x7f43540a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.853+0000 7f43657fa640 1 -- 172.21.15.76:0/511075205 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f43580cd960 con 0x7f4368074110 2024-03-23T15:40:38.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.853+0000 7f435effd640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f43540a20c0 0x7f43540a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f436819ade0 tx=0x7f4350005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:38.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:38.857+0000 7f43657fa640 1 -- 172.21.15.76:0/511075205 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4358096dd0 con 0x7f4368074110 2024-03-23T15:40:39.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.030+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f436810f5e0 con 0x7f4368074110 2024-03-23T15:40:39.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.030+0000 7f43657fa640 1 -- 172.21.15.76:0/511075205 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f435809a490 con 0x7f4368074110 2024-03-23T15:40:39.034 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.032+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f43540a20c0 msgr2=0x7f43540a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.032+0000 7f436dca4640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f43540a20c0 0x7f43540a4580 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f436819ade0 tx=0x7f4350005f90 comp rx=0 tx=0).stop 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.032+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368074110 msgr2=0x7f43681998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.032+0000 7f436dca4640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368074110 0x7f43681998d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f435800c920 tx=0x7f435800cde0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.032+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 shutdown_connections 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.032+0000 7f436dca4640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f43540a20c0 0x7f43540a4580 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.033+0000 7f436dca4640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4368109630 0x7f43681a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.033+0000 7f436dca4640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43680756f0 0x7f4368199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.033+0000 7f436dca4640 1 --2- 172.21.15.76:0/511075205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4368074110 0x7f43681998d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.033+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 >> 172.21.15.76:0/511075205 conn(0x7f43680fcd30 msgr2=0x7f43681055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:39.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.033+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 shutdown_connections 2024-03-23T15:40:39.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.033+0000 7f436dca4640 1 -- 172.21.15.76:0/511075205 wait complete. 2024-03-23T15:40:39.047 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:39.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 -- 172.21.15.76:0/1006762290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b81029f0 msgr2=0x7f78b8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:39.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 --2- 172.21.15.76:0/1006762290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b81029f0 0x7f78b8102e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009900 tx=0x7f78a802bc30 comp rx=0 tx=0).stop 2024-03-23T15:40:39.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 -- 172.21.15.76:0/1006762290 shutdown_connections 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 --2- 172.21.15.76:0/1006762290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78b81033b0 0x7f78b8109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 --2- 172.21.15.76:0/1006762290 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b81029f0 0x7f78b8102e70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 --2- 172.21.15.76:0/1006762290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b80ffa20 0x7f78b80ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 -- 172.21.15.76:0/1006762290 >> 172.21.15.76:0/1006762290 conn(0x7f78b80fb350 msgr2=0x7f78b80fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 -- 172.21.15.76:0/1006762290 shutdown_connections 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.146+0000 7f78bffd0640 1 -- 172.21.15.76:0/1006762290 wait complete. 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 Processor -- start 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 -- start start 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b80ffa20 0x7f78b8197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78b81029f0 0x7f78b8197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b81033b0 0x7f78b819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f78b819f2f0 con 0x7f78b80ffa20 2024-03-23T15:40:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78b819f460 con 0x7f78b81033b0 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78b819f620 con 0x7f78b81029f0 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b80ffa20 0x7f78b8197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b80ffa20 0x7f78b8197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57334/0 (socket says 172.21.15.76:57334) 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 -- 172.21.15.76:0/2323192958 learned_addr learned my addr 172.21.15.76:0/2323192958 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bd56b640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78b81029f0 0x7f78b8197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78be56d640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b81033b0 0x7f78b819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 -- 172.21.15.76:0/2323192958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78b81029f0 msgr2=0x7f78b8197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78b81029f0 0x7f78b8197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 -- 172.21.15.76:0/2323192958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b81033b0 msgr2=0x7f78b819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b81033b0 0x7f78b819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 -- 172.21.15.76:0/2323192958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a8009560 con 0x7f78b80ffa20 2024-03-23T15:40:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bdd6c640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b80ffa20 0x7f78b8197800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d770 tx=0x7f78b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78aeffd640 1 -- 172.21.15.76:0/2323192958 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4009c30 con 0x7f78b80ffa20 2024-03-23T15:40:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78aeffd640 1 -- 172.21.15.76:0/2323192958 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b400de80 con 0x7f78b80ffa20 2024-03-23T15:40:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b819f900 con 0x7f78b80ffa20 2024-03-23T15:40:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.147+0000 7f78aeffd640 1 -- 172.21.15.76:0/2323192958 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4010460 con 0x7f78b80ffa20 2024-03-23T15:40:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.148+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b819fe60 con 0x7f78b80ffa20 2024-03-23T15:40:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.149+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b8108b40 con 0x7f78b80ffa20 2024-03-23T15:40:39.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.150+0000 7f78aeffd640 1 -- 172.21.15.76:0/2323192958 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f78b4003f60 con 0x7f78b80ffa20 2024-03-23T15:40:39.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.150+0000 7f78aeffd640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a40a2140 0x7f78a40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:39.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.150+0000 7f78bd56b640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a40a2140 0x7f78a40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:39.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.150+0000 7f78aeffd640 1 -- 172.21.15.76:0/2323192958 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f78b40cd540 con 0x7f78b80ffa20 2024-03-23T15:40:39.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.151+0000 7f78bd56b640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a40a2140 0x7f78a40a4600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78b8198d10 tx=0x7f78a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:39.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.154+0000 7f78aeffd640 1 -- 172.21.15.76:0/2323192958 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f78b40969b0 con 0x7f78b80ffa20 2024-03-23T15:40:39.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.369+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78b810d150 con 0x7f78b80ffa20 2024-03-23T15:40:39.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.369+0000 7f78aeffd640 1 -- 172.21.15.76:0/2323192958 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f78b409a070 con 0x7f78b80ffa20 2024-03-23T15:40:39.373 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:39.374 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-23T15:40:39.374 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 1 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a40a2140 msgr2=0x7f78a40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a40a2140 0x7f78a40a4600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78b8198d10 tx=0x7f78a8000bc0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b80ffa20 msgr2=0x7f78b8197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b80ffa20 0x7f78b8197800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d770 tx=0x7f78b400dc30 comp rx=0 tx=0).stop 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 shutdown_connections 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a40a2140 0x7f78a40a4600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b81033b0 0x7f78b819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78b81029f0 0x7f78b8197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 --2- 172.21.15.76:0/2323192958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78b80ffa20 0x7f78b8197800 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:39.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.371+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 >> 172.21.15.76:0/2323192958 conn(0x7f78b80fb350 msgr2=0x7f78b8105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:39.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.372+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 shutdown_connections 2024-03-23T15:40:39.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:39.372+0000 7f78bffd0640 1 -- 172.21.15.76:0/2323192958 wait complete. 2024-03-23T15:40:39.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:39 smithi192 ceph-mon[36653]: Deploying daemon mds.l on smithi192 2024-03-23T15:40:39.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:39 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2340503930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:40:39.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:39 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/511075205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:40:39.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:39 smithi076 ceph-mon[32176]: Deploying daemon mds.l on smithi192 2024-03-23T15:40:39.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:39 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2340503930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:40:39.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:39 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/511075205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:40:39.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:39 smithi183 ceph-mon[36296]: Deploying daemon mds.l on smithi192 2024-03-23T15:40:39.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:39 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2340503930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:40:39.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:39 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/511075205' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:40:40.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:40 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2323192958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:40.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:40 smithi192 ceph-mon[36653]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:40.570 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 systemd[1]: Starting Ceph mds.l for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:40:40.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:40 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2323192958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:40.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:40 smithi076 ceph-mon[32176]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:40.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:40 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2323192958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:40.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:40 smithi183 ceph-mon[36296]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:40.882 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 podman[66993]: 2024-03-23 15:40:40.487980121 +0000 UTC m=+0.017725219 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:40:40.882 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 podman[66993]: 2024-03-23 15:40:40.61100307 +0000 UTC m=+0.140748152 container create ce914a2b35931c7a40e9071f7232c1920adc79e1dba73544194e563e41cce427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_BRANCH=HEAD) 2024-03-23T15:40:40.882 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 podman[66993]: 2024-03-23 15:40:40.880399762 +0000 UTC m=+0.410144848 container init ce914a2b35931c7a40e9071f7232c1920adc79e1dba73544194e563e41cce427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-23T15:40:41.319 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 podman[66993]: 2024-03-23 15:40:40.882984743 +0000 UTC m=+0.412729830 container start ce914a2b35931c7a40e9071f7232c1920adc79e1dba73544194e563e41cce427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-03-23T15:40:41.319 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 ceph-mds[67012]: -- 172.21.15.192:0/3087412221 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b50ffa1a0 con 0x555b50fd1000 2024-03-23T15:40:41.320 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l[67008]: starting mds.l at 2024-03-23T15:40:41.320 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 bash[66993]: ce914a2b35931c7a40e9071f7232c1920adc79e1dba73544194e563e41cce427 2024-03-23T15:40:41.320 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:40:40 smithi192 systemd[1]: Started Ceph mds.l for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:40:41.387 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:41.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.495+0000 7ff0b868f640 1 -- 172.21.15.76:0/723723785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0107490 msgr2=0x7ff0b0109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.495+0000 7ff0b868f640 1 --2- 172.21.15.76:0/723723785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0107490 0x7ff0b0109880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009ca0 tx=0x7ff0a802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- 172.21.15.76:0/723723785 shutdown_connections 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 --2- 172.21.15.76:0/723723785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0107490 0x7ff0b0109880 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 --2- 172.21.15.76:0/723723785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b0104b60 0x7ff0b0106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 --2- 172.21.15.76:0/723723785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b0102230 0x7ff0b0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- 172.21.15.76:0/723723785 >> 172.21.15.76:0/723723785 conn(0x7ff0b00fbba0 msgr2=0x7ff0b00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- 172.21.15.76:0/723723785 shutdown_connections 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- 172.21.15.76:0/723723785 wait complete. 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 Processor -- start 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- start start 2024-03-23T15:40:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0102230 0x7ff0b0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b0104b60 0x7ff0b0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b0107490 0x7ff0b019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b019f240 con 0x7ff0b0102230 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b019f3b0 con 0x7ff0b0107490 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b868f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b019f570 con 0x7ff0b0104b60 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b642b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0102230 0x7ff0b0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.496+0000 7ff0b642b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0102230 0x7ff0b0197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57348/0 (socket says 172.21.15.76:57348) 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b642b640 1 -- 172.21.15.76:0/2505382086 learned_addr learned my addr 172.21.15.76:0/2505382086 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b6c2c640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b0107490 0x7ff0b019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b5c2a640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b0104b60 0x7ff0b0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b642b640 1 -- 172.21.15.76:0/2505382086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b0104b60 msgr2=0x7ff0b0197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b642b640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b0104b60 0x7ff0b0197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b642b640 1 -- 172.21.15.76:0/2505382086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b0107490 msgr2=0x7ff0b019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b642b640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b0107490 0x7ff0b019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b642b640 1 -- 172.21.15.76:0/2505382086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a8009900 con 0x7ff0b0102230 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b642b640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0102230 0x7ff0b0197750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00e9f0 tx=0x7ff0ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0a77fe640 1 -- 172.21.15.76:0/2505382086 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac009700 con 0x7ff0b0102230 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0a77fe640 1 -- 172.21.15.76:0/2505382086 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ac00cd80 con 0x7ff0b0102230 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b019f850 con 0x7ff0b0102230 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0a77fe640 1 -- 172.21.15.76:0/2505382086 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac018920 con 0x7ff0b0102230 2024-03-23T15:40:41.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.497+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b019fd60 con 0x7ff0b0102230 2024-03-23T15:40:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.498+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b00fd690 con 0x7ff0b0102230 2024-03-23T15:40:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.499+0000 7ff0a77fe640 1 -- 172.21.15.76:0/2505382086 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7ff0ac018a80 con 0x7ff0b0102230 2024-03-23T15:40:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.500+0000 7ff0a77fe640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff09c0a20c0 0x7ff09c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.500+0000 7ff0b5c2a640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff09c0a20c0 0x7ff09c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.500+0000 7ff0a77fe640 1 -- 172.21.15.76:0/2505382086 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff0ac014070 con 0x7ff0b0102230 2024-03-23T15:40:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.500+0000 7ff0b5c2a640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff09c0a20c0 0x7ff09c0a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0198c60 tx=0x7ff0a000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:41.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.504+0000 7ff0a77fe640 1 -- 172.21.15.76:0/2505382086 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff0ac053280 con 0x7ff0b0102230 2024-03-23T15:40:41.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.718+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0b002d1d0 con 0x7ff0b0102230 2024-03-23T15:40:41.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.718+0000 7ff0a77fe640 1 -- 172.21.15.76:0/2505382086 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7ff0ac097090 con 0x7ff0b0102230 2024-03-23T15:40:41.721 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:41.721 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T15:40:41.722 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 2 2024-03-23T15:40:41.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff09c0a20c0 msgr2=0x7ff09c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff09c0a20c0 0x7ff09c0a4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0198c60 tx=0x7ff0a000b590 comp rx=0 tx=0).stop 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0102230 msgr2=0x7ff0b0197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0102230 0x7ff0b0197750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00e9f0 tx=0x7ff0ac00eeb0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 shutdown_connections 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff09c0a20c0 0x7ff09c0a4580 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b0107490 0x7ff0b019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0b0104b60 0x7ff0b0197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 --2- 172.21.15.76:0/2505382086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0b0102230 0x7ff0b0197750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 >> 172.21.15.76:0/2505382086 conn(0x7ff0b00fbba0 msgr2=0x7ff0b0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 shutdown_connections 2024-03-23T15:40:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:41.720+0000 7ff0b868f640 1 -- 172.21.15.76:0/2505382086 wait complete. 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6832/996338359,v1:172.21.15.192:6833/996338359] up:boot 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: fsmap 1 up:standby 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: Deploying daemon mds.k on smithi183 2024-03-23T15:40:42.025 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:42.026 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:41 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2505382086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6832/996338359,v1:172.21.15.192:6833/996338359] up:boot 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: fsmap 1 up:standby 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: Deploying daemon mds.k on smithi183 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:41 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2505382086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:42.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6832/996338359,v1:172.21.15.192:6833/996338359] up:boot 2024-03-23T15:40:42.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: fsmap 1 up:standby 2024-03-23T15:40:42.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: Deploying daemon mds.k on smithi183 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:42.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:41 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2505382086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:43.383 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 systemd[1]: Starting Ceph mds.k for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:40:43.735 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:43.777 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 podman[66058]: 2024-03-23 15:40:43.38340034 +0000 UTC m=+0.017718517 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:40:43.777 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 podman[66058]: 2024-03-23 15:40:43.505302503 +0000 UTC m=+0.139620689 container create db71d6822185d8474544eb54843b40ea26f8834303d12f1cd43d4684e5dcb20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:40:43.777 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 podman[66058]: 2024-03-23 15:40:43.774463691 +0000 UTC m=+0.408781875 container init db71d6822185d8474544eb54843b40ea26f8834303d12f1cd43d4684e5dcb20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.846+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/3274867620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 msgr2=0x7fcfac108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.846+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/3274867620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac108af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009ca0 tx=0x7fcfa402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.846+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/3274867620 shutdown_connections 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.846+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/3274867620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac108af0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.846+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/3274867620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfac1018a0 0x7fcfac101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/3274867620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfac1006b0 0x7fcfac100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/3274867620 >> 172.21.15.76:0/3274867620 conn(0x7fcfac0fbc40 msgr2=0x7fcfac0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/3274867620 shutdown_connections 2024-03-23T15:40:43.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/3274867620 wait complete. 2024-03-23T15:40:43.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 Processor -- start 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 -- start start 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfac1006b0 0x7fcfac1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfac1018a0 0x7fcfac196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac19e0d0 con 0x7fcfac102260 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac19e240 con 0x7fcfac1006b0 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.847+0000 7fcfb0b3e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac19e400 con 0x7fcfac1018a0 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac19db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57370/0 (socket says 172.21.15.76:57370) 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 -- 172.21.15.76:0/1813599550 learned_addr learned my addr 172.21.15.76:0/1813599550 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:43.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaa59c640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfac1006b0 0x7fcfac1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 -- 172.21.15.76:0/1813599550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfac1018a0 msgr2=0x7fcfac196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfac1018a0 0x7fcfac196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 -- 172.21.15.76:0/1813599550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfac1006b0 msgr2=0x7fcfac1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfac1006b0 0x7fcfac1965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 -- 172.21.15.76:0/1813599550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa4009900 con 0x7fcfac102260 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfaad9d640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac19db90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4037040 tx=0x7fcfa4034d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcf9b7fe640 1 -- 172.21.15.76:0/1813599550 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4003970 con 0x7fcfac102260 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcf9b7fe640 1 -- 172.21.15.76:0/1813599550 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa4004040 con 0x7fcfac102260 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcf9b7fe640 1 -- 172.21.15.76:0/1813599550 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40406b0 con 0x7fcfac102260 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac19e680 con 0x7fcfac102260 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.848+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac19eb30 con 0x7fcfac102260 2024-03-23T15:40:43.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.849+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfac107900 con 0x7fcfac102260 2024-03-23T15:40:43.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.850+0000 7fcf9b7fe640 1 -- 172.21.15.76:0/1813599550 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fcfa400a040 con 0x7fcfac102260 2024-03-23T15:40:43.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.851+0000 7fcf9b7fe640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf940a20c0 0x7fcf940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:43.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.851+0000 7fcfaa59c640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf940a20c0 0x7fcf940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:43.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.851+0000 7fcf9b7fe640 1 -- 172.21.15.76:0/1813599550 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcfa40f0a50 con 0x7fcfac102260 2024-03-23T15:40:43.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.851+0000 7fcfaa59c640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf940a20c0 0x7fcf940a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c005fd0 tx=0x7fcf9c005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:43.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:43.855+0000 7fcf9b7fe640 1 -- 172.21.15.76:0/1813599550 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fcfa40b9ec0 con 0x7fcfac102260 2024-03-23T15:40:44.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.076+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfac10c050 con 0x7fcfac102260 2024-03-23T15:40:44.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.076+0000 7fcf9b7fe640 1 -- 172.21.15.76:0/1813599550 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fcfa40041b0 con 0x7fcfac102260 2024-03-23T15:40:44.080 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:44.080 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T15:40:44.080 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 2 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf940a20c0 msgr2=0x7fcf940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf940a20c0 0x7fcf940a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c005fd0 tx=0x7fcf9c005ce0 comp rx=0 tx=0).stop 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 msgr2=0x7fcfac19db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac19db90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4037040 tx=0x7fcfa4034d20 comp rx=0 tx=0).stop 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 shutdown_connections 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac102260 0x7fcfac19db90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfac1018a0 0x7fcfac196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf940a20c0 0x7fcf940a4580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:44.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 --2- 172.21.15.76:0/1813599550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfac1006b0 0x7fcfac1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:44.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 >> 172.21.15.76:0/1813599550 conn(0x7fcfac0fbc40 msgr2=0x7fcfac104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:44.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 shutdown_connections 2024-03-23T15:40:44.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:44.079+0000 7fcfb0b3e640 1 -- 172.21.15.76:0/1813599550 wait complete. 2024-03-23T15:40:44.133 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 podman[66058]: 2024-03-23 15:40:43.77725351 +0000 UTC m=+0.411571687 container start db71d6822185d8474544eb54843b40ea26f8834303d12f1cd43d4684e5dcb20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-23T15:40:44.133 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 ceph-mds[66077]: -- 172.21.15.183:0/2823040312 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d590101a0 con 0x563d58fe6c00 2024-03-23T15:40:44.133 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k[66073]: starting mds.k at 2024-03-23T15:40:44.134 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 bash[66058]: db71d6822185d8474544eb54843b40ea26f8834303d12f1cd43d4684e5dcb20f 2024-03-23T15:40:44.134 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:40:43 smithi183 systemd[1]: Started Ceph mds.k for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:40:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:44 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1813599550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1813599550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:44.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:44.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:44 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1813599550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:45.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:45 smithi192 ceph-mon[36653]: Deploying daemon mds.j on smithi076 2024-03-23T15:40:45.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:45 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6832/2543151700,v1:172.21.15.183:6833/2543151700] up:boot 2024-03-23T15:40:45.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:45 smithi192 ceph-mon[36653]: fsmap 2 up:standby 2024-03-23T15:40:45.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:45 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T15:40:45.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:45 smithi076 ceph-mon[32176]: Deploying daemon mds.j on smithi076 2024-03-23T15:40:45.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:45 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6832/2543151700,v1:172.21.15.183:6833/2543151700] up:boot 2024-03-23T15:40:45.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:45 smithi076 ceph-mon[32176]: fsmap 2 up:standby 2024-03-23T15:40:45.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:45 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T15:40:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:45 smithi183 ceph-mon[36296]: Deploying daemon mds.j on smithi076 2024-03-23T15:40:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:45 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6832/2543151700,v1:172.21.15.183:6833/2543151700] up:boot 2024-03-23T15:40:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:45 smithi183 ceph-mon[36296]: fsmap 2 up:standby 2024-03-23T15:40:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:45 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T15:40:46.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:46.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 -- 172.21.15.76:0/2469755768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 msgr2=0x7fd474100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 --2- 172.21.15.76:0/2469755768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474100d50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd464009900 tx=0x7fd46402bc30 comp rx=0 tx=0).stop 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 -- 172.21.15.76:0/2469755768 shutdown_connections 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 --2- 172.21.15.76:0/2469755768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd474101290 0x7fd474108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 --2- 172.21.15.76:0/2469755768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474100d50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 --2- 172.21.15.76:0/2469755768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd474101c30 0x7fd474102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 -- 172.21.15.76:0/2469755768 >> 172.21.15.76:0/2469755768 conn(0x7fd4740fb340 msgr2=0x7fd4740fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 -- 172.21.15.76:0/2469755768 shutdown_connections 2024-03-23T15:40:46.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.205+0000 7fd47990c640 1 -- 172.21.15.76:0/2469755768 wait complete. 2024-03-23T15:40:46.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.206+0000 7fd47990c640 1 Processor -- start 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47990c640 1 -- start start 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47990c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47990c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd474101290 0x7fd474197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47990c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd474101c30 0x7fd47419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47990c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd47419f240 con 0x7fd4741008d0 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47990c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd47419f3b0 con 0x7fd474101c30 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47990c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd47419f570 con 0x7fd474101290 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36446/0 (socket says 172.21.15.76:36446) 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 -- 172.21.15.76:0/1622247523 learned_addr learned my addr 172.21.15.76:0/1622247523 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd4737fe640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd474101c30 0x7fd47419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:46.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd4727fc640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd474101290 0x7fd474197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 -- 172.21.15.76:0/1622247523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd474101290 msgr2=0x7fd474197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd474101290 0x7fd474197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 -- 172.21.15.76:0/1622247523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd474101c30 msgr2=0x7fd47419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd474101c30 0x7fd47419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 -- 172.21.15.76:0/1622247523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd464009560 con 0x7fd4741008d0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd4727fc640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd474101290 0x7fd474197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd472ffd640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474197750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd46800e9c0 tx=0x7fd46800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.207+0000 7fd47890a640 1 -- 172.21.15.76:0/1622247523 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468009700 con 0x7fd4741008d0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.208+0000 7fd47890a640 1 -- 172.21.15.76:0/1622247523 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46800cd80 con 0x7fd4741008d0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.208+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47419f850 con 0x7fd4741008d0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.208+0000 7fd47890a640 1 -- 172.21.15.76:0/1622247523 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468010430 con 0x7fd4741008d0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.208+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47419fdb0 con 0x7fd4741008d0 2024-03-23T15:40:46.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.208+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4741079b0 con 0x7fd4741008d0 2024-03-23T15:40:46.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.210+0000 7fd47890a640 1 -- 172.21.15.76:0/1622247523 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fd468003680 con 0x7fd4741008d0 2024-03-23T15:40:46.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.211+0000 7fd47890a640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd45c0a2140 0x7fd45c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:46.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.211+0000 7fd4727fc640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd45c0a2140 0x7fd45c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:46.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.211+0000 7fd47890a640 1 -- 172.21.15.76:0/1622247523 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd468014070 con 0x7fd4741008d0 2024-03-23T15:40:46.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.212+0000 7fd4727fc640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd45c0a2140 0x7fd45c0a4600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd464036040 tx=0x7fd464000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:46.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.215+0000 7fd47890a640 1 -- 172.21.15.76:0/1622247523 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd468097180 con 0x7fd4741008d0 2024-03-23T15:40:46.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.431+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd47410d190 con 0x7fd4741008d0 2024-03-23T15:40:46.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.431+0000 7fd47890a640 1 -- 172.21.15.76:0/1622247523 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fd46809a840 con 0x7fd4741008d0 2024-03-23T15:40:46.435 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:46.435 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T15:40:46.435 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 3 2024-03-23T15:40:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd45c0a2140 msgr2=0x7fd45c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd45c0a2140 0x7fd45c0a4600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd464036040 tx=0x7fd464000bc0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 msgr2=0x7fd474197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474197750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd46800e9c0 tx=0x7fd46800ee80 comp rx=0 tx=0).stop 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 shutdown_connections 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd45c0a2140 0x7fd45c0a4600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd474101c30 0x7fd47419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd474101290 0x7fd474197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 --2- 172.21.15.76:0/1622247523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4741008d0 0x7fd474197750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 >> 172.21.15.76:0/1622247523 conn(0x7fd4740fb340 msgr2=0x7fd474104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 shutdown_connections 2024-03-23T15:40:46.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:46.433+0000 7fd47990c640 1 -- 172.21.15.76:0/1622247523 wait complete. 2024-03-23T15:40:46.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:46 smithi076 ceph-mon[32176]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:46.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:46 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1622247523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:46 smithi192 ceph-mon[36653]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:46 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1622247523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:46 smithi183 ceph-mon[36296]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:46 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1622247523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:47.075 INFO:journalctl@ceph.mds.j.smithi076.stdout:Mar 23 15:40:46 smithi076 systemd[1]: Started Ceph mds.j for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:40:48.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: Deploying daemon mds.d on smithi076 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.76:6834/1996137428,v1:172.21.15.76:6835/1996137428] up:boot 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: fsmap 3 up:standby 2024-03-23T15:40:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:47 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T15:40:48.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: Deploying daemon mds.d on smithi076 2024-03-23T15:40:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.76:6834/1996137428,v1:172.21.15.76:6835/1996137428] up:boot 2024-03-23T15:40:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: fsmap 3 up:standby 2024-03-23T15:40:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:47 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T15:40:48.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:48.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:48.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:48.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:48.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: Deploying daemon mds.d on smithi076 2024-03-23T15:40:48.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:48.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.76:6834/1996137428,v1:172.21.15.76:6835/1996137428] up:boot 2024-03-23T15:40:48.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: fsmap 3 up:standby 2024-03-23T15:40:48.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:47 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T15:40:48.450 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.553+0000 7f9cb7d02640 1 -- 172.21.15.76:0/572521828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 msgr2=0x7f9cb0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/572521828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0108a60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009ca0 tx=0x7f9ca802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- 172.21.15.76:0/572521828 shutdown_connections 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/572521828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0108a60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/572521828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb0101810 0x7f9cb0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/572521828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb0100620 0x7f9cb0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- 172.21.15.76:0/572521828 >> 172.21.15.76:0/572521828 conn(0x7f9cb00fbbb0 msgr2=0x7f9cb00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- 172.21.15.76:0/572521828 shutdown_connections 2024-03-23T15:40:48.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- 172.21.15.76:0/572521828 wait complete. 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 Processor -- start 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- start start 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb0100620 0x7f9cb01045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb0101810 0x7f9cb0102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0103890 con 0x7f9cb01021d0 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0103a00 con 0x7f9cb0101810 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.554+0000 7f9cb7d02640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01a17e0 con 0x7f9cb0100620 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36486/0 (socket says 172.21.15.76:36486) 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 -- 172.21.15.76:0/2143126098 learned_addr learned my addr 172.21.15.76:0/2143126098 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb529d640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb0101810 0x7f9cb0102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb5a9e640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb0100620 0x7f9cb01045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:48.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 -- 172.21.15.76:0/2143126098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb0100620 msgr2=0x7f9cb01045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb0100620 0x7f9cb01045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 -- 172.21.15.76:0/2143126098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb0101810 msgr2=0x7f9cb0102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb0101810 0x7f9cb0102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 -- 172.21.15.76:0/2143126098 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca8009900 con 0x7f9cb01021d0 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb629f640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0103130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80098d0 tx=0x7f9ca8034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9ca6ffd640 1 -- 172.21.15.76:0/2143126098 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8032eb0 con 0x7f9cb01021d0 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9ca6ffd640 1 -- 172.21.15.76:0/2143126098 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ca8003e50 con 0x7f9cb01021d0 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb01a1a30 con 0x7f9cb01021d0 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9ca6ffd640 1 -- 172.21.15.76:0/2143126098 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80406b0 con 0x7f9cb01021d0 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.555+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb01a1f30 con 0x7f9cb01021d0 2024-03-23T15:40:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.556+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb010c030 con 0x7f9cb01021d0 2024-03-23T15:40:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.557+0000 7f9ca6ffd640 1 -- 172.21.15.76:0/2143126098 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f9ca800a040 con 0x7f9cb01021d0 2024-03-23T15:40:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.558+0000 7f9ca6ffd640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9c9c0a2140 0x7f9c9c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.558+0000 7f9cb5a9e640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9c9c0a2140 0x7f9c9c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.558+0000 7f9ca6ffd640 1 -- 172.21.15.76:0/2143126098 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9ca80f06d0 con 0x7f9cb01021d0 2024-03-23T15:40:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.558+0000 7f9cb5a9e640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9c9c0a2140 0x7f9c9c0a4600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9cac005fd0 tx=0x7f9cac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.562+0000 7f9ca6ffd640 1 -- 172.21.15.76:0/2143126098 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9ca80b9bd0 con 0x7f9cb01021d0 2024-03-23T15:40:48.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.792+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cb010c1c0 con 0x7f9cb01021d0 2024-03-23T15:40:48.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.792+0000 7f9ca6ffd640 1 -- 172.21.15.76:0/2143126098 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f9ca80bd290 con 0x7f9cb01021d0 2024-03-23T15:40:48.795 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:48.796 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T15:40:48.796 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 4 2024-03-23T15:40:48.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9c9c0a2140 msgr2=0x7f9c9c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9c9c0a2140 0x7f9c9c0a4600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9cac005fd0 tx=0x7f9cac005850 comp rx=0 tx=0).stop 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 msgr2=0x7f9cb0103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0103130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80098d0 tx=0x7f9ca8034d40 comp rx=0 tx=0).stop 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 shutdown_connections 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9cb01021d0 0x7f9cb0103130 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9c9c0a2140 0x7f9c9c0a4600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cb0101810 0x7f9cb0102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 --2- 172.21.15.76:0/2143126098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb0100620 0x7f9cb01045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 >> 172.21.15.76:0/2143126098 conn(0x7f9cb00fbbb0 msgr2=0x7f9cb00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 shutdown_connections 2024-03-23T15:40:48.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:48.794+0000 7f9cb7d02640 1 -- 172.21.15.76:0/2143126098 wait complete. 2024-03-23T15:40:49.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:48 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2143126098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:49.076 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:48 smithi076 systemd[1]: Starting Ceph mds.d for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:40:49.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:48 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2143126098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:49.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:48 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2143126098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:49.454 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 podman[70838]: 2024-03-23 15:40:49.083314469 +0000 UTC m=+0.017705750 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:40:49.455 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 podman[70838]: 2024-03-23 15:40:49.200628888 +0000 UTC m=+0.135020160 container create 8383b805fa59f8a3bf7a9bf7f363eb1cf258dea52fb6df429cd885cda0c95657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-23T15:40:49.455 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 podman[70838]: 2024-03-23 15:40:49.453148956 +0000 UTC m=+0.387540227 container init 8383b805fa59f8a3bf7a9bf7f363eb1cf258dea52fb6df429cd885cda0c95657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-23T15:40:49.826 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 podman[70838]: 2024-03-23 15:40:49.455736256 +0000 UTC m=+0.390127529 container start 8383b805fa59f8a3bf7a9bf7f363eb1cf258dea52fb6df429cd885cda0c95657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-23T15:40:49.826 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 ceph-mds[70857]: -- 172.21.15.76:0/2886116355 <== mon.0 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dcd0a941a0 con 0x55dcd0a6b000 2024-03-23T15:40:49.826 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d[70853]: starting mds.d at 2024-03-23T15:40:49.826 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 bash[70838]: 8383b805fa59f8a3bf7a9bf7f363eb1cf258dea52fb6df429cd885cda0c95657 2024-03-23T15:40:49.826 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:40:49 smithi076 systemd[1]: Started Ceph mds.d for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:40:50.809 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.76:6836/3380265538,v1:172.21.15.76:6837/3380265538] up:boot 2024-03-23T15:40:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: fsmap 4 up:standby 2024-03-23T15:40:50.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:50 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.923+0000 7f22f18bb640 1 -- 172.21.15.76:0/1024396441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec101720 msgr2=0x7f22ec101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.923+0000 7f22f18bb640 1 --2- 172.21.15.76:0/1024396441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec101720 0x7f22ec101b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009900 tx=0x7f22dc02bc30 comp rx=0 tx=0).stop 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.923+0000 7f22f18bb640 1 -- 172.21.15.76:0/1024396441 shutdown_connections 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.923+0000 7f22f18bb640 1 --2- 172.21.15.76:0/1024396441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec1032d0 0x7f22ec109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.923+0000 7f22f18bb640 1 --2- 172.21.15.76:0/1024396441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec102910 0x7f22ec102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.923+0000 7f22f18bb640 1 --2- 172.21.15.76:0/1024396441 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec101720 0x7f22ec101b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.923+0000 7f22f18bb640 1 -- 172.21.15.76:0/1024396441 >> 172.21.15.76:0/1024396441 conn(0x7f22ec0fccf0 msgr2=0x7f22ec0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 -- 172.21.15.76:0/1024396441 shutdown_connections 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 -- 172.21.15.76:0/1024396441 wait complete. 2024-03-23T15:40:50.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 Processor -- start 2024-03-23T15:40:50.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 -- start start 2024-03-23T15:40:50.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec101720 0x7f22ec0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec102910 0x7f22ec073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec1032d0 0x7f22ec074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec0747a0 con 0x7f22ec102910 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec074910 con 0x7f22ec1032d0 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.924+0000 7f22f18bb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1a18e0 con 0x7f22ec101720 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec102910 0x7f22ec073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec102910 0x7f22ec073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36520/0 (socket says 172.21.15.76:36520) 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 -- 172.21.15.76:0/3367282473 learned_addr learned my addr 172.21.15.76:0/3367282473 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22eb7fe640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec1032d0 0x7f22ec074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22eaffd640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec101720 0x7f22ec0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 -- 172.21.15.76:0/3367282473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec101720 msgr2=0x7f22ec0754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec101720 0x7f22ec0754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 -- 172.21.15.76:0/3367282473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec1032d0 msgr2=0x7f22ec074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:50.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec1032d0 0x7f22ec074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 -- 172.21.15.76:0/3367282473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc009560 con 0x7f22ec102910 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22ea7fc640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec102910 0x7f22ec073b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f22e000b4d0 tx=0x7f22e000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22f08b9640 1 -- 172.21.15.76:0/3367282473 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0010870 con 0x7f22ec102910 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22f08b9640 1 -- 172.21.15.76:0/3367282473 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e000f3c0 con 0x7f22ec102910 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22f08b9640 1 -- 172.21.15.76:0/3367282473 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00175c0 con 0x7f22ec102910 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec1a1b90 con 0x7f22ec102910 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.925+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec1a20f0 con 0x7f22ec102910 2024-03-23T15:40:50.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.926+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec108ae0 con 0x7f22ec102910 2024-03-23T15:40:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.927+0000 7f22f08b9640 1 -- 172.21.15.76:0/3367282473 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f22e000f8c0 con 0x7f22ec102910 2024-03-23T15:40:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.928+0000 7f22f08b9640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f22d40a2140 0x7f22d40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.928+0000 7f22eaffd640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f22d40a2140 0x7f22d40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.928+0000 7f22f08b9640 1 -- 172.21.15.76:0/3367282473 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f22e00cd090 con 0x7f22ec102910 2024-03-23T15:40:50.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.928+0000 7f22eaffd640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f22d40a2140 0x7f22d40a4600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005ab0 tx=0x7f22dc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:50.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:50.932+0000 7f22f08b9640 1 -- 172.21.15.76:0/3367282473 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f22e00965b0 con 0x7f22ec102910 2024-03-23T15:40:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.76:6836/3380265538,v1:172.21.15.76:6837/3380265538] up:boot 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: fsmap 4 up:standby 2024-03-23T15:40:51.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:50 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T15:40:51.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:51.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.76:6836/3380265538,v1:172.21.15.76:6837/3380265538] up:boot 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: fsmap 4 up:standby 2024-03-23T15:40:51.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:50 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T15:40:51.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.145+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22ec10d380 con 0x7f22ec102910 2024-03-23T15:40:51.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.145+0000 7f22f08b9640 1 -- 172.21.15.76:0/3367282473 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f22e00092b0 con 0x7f22ec102910 2024-03-23T15:40:51.148 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:51.148 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T15:40:51.148 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 5 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f22d40a2140 msgr2=0x7f22d40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f22d40a2140 0x7f22d40a4600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005ab0 tx=0x7f22dc005fd0 comp rx=0 tx=0).stop 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec102910 msgr2=0x7f22ec073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec102910 0x7f22ec073b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f22e000b4d0 tx=0x7f22e000b990 comp rx=0 tx=0).stop 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 shutdown_connections 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f22d40a2140 0x7f22d40a4600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec1032d0 0x7f22ec074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22ec102910 0x7f22ec073b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 --2- 172.21.15.76:0/3367282473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec101720 0x7f22ec0754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:51.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 >> 172.21.15.76:0/3367282473 conn(0x7f22ec0fccf0 msgr2=0x7f22ec105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:51.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 shutdown_connections 2024-03-23T15:40:51.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:51.148+0000 7f22f18bb640 1 -- 172.21.15.76:0/3367282473 wait complete. 2024-03-23T15:40:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:51 smithi183 ceph-mon[36296]: Deploying daemon mds.e on smithi183 2024-03-23T15:40:51.969 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:51 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3367282473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:52.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:51 smithi192 ceph-mon[36653]: Deploying daemon mds.e on smithi183 2024-03-23T15:40:52.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:51 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3367282473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:52.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:51 smithi076 ceph-mon[32176]: Deploying daemon mds.e on smithi183 2024-03-23T15:40:52.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:51 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3367282473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:52.622 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:40:52 smithi183 systemd[1]: Started Ceph mds.e for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:52 smithi183 ceph-mon[36296]: Deploying daemon mds.f on smithi192 2024-03-23T15:40:53.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:53.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:53.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:53.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:53.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:53.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:53.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:53.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:52 smithi192 ceph-mon[36653]: Deploying daemon mds.f on smithi192 2024-03-23T15:40:53.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:53.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:53.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:53.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:53.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:53.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:53.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:53.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:52 smithi076 ceph-mon[32176]: Deploying daemon mds.f on smithi192 2024-03-23T15:40:53.161 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:53.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 -- 172.21.15.76:0/3845530743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760101820 msgr2=0x7fd760101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:53.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 --2- 172.21.15.76:0/3845530743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760101820 0x7fd760101ca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009900 tx=0x7fd75c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:40:53.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 -- 172.21.15.76:0/3845530743 shutdown_connections 2024-03-23T15:40:53.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 --2- 172.21.15.76:0/3845530743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7601021e0 0x7fd760108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 --2- 172.21.15.76:0/3845530743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760101820 0x7fd760101ca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 --2- 172.21.15.76:0/3845530743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd760100630 0x7fd760100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 -- 172.21.15.76:0/3845530743 >> 172.21.15.76:0/3845530743 conn(0x7fd7600fbba0 msgr2=0x7fd7600fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:53.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 -- 172.21.15.76:0/3845530743 shutdown_connections 2024-03-23T15:40:53.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.279+0000 7fd767807640 1 -- 172.21.15.76:0/3845530743 wait complete. 2024-03-23T15:40:53.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 Processor -- start 2024-03-23T15:40:53.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 -- start start 2024-03-23T15:40:53.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760100630 0x7fd760199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:53.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd760101820 0x7fd760199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7601021e0 0x7fd7601a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7601a1500 con 0x7fd760100630 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7601a1670 con 0x7fd760101820 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd767807640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd7601a1830 con 0x7fd7601021e0 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd7655a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760100630 0x7fd760199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd7655a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760100630 0x7fd760199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36534/0 (socket says 172.21.15.76:36534) 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd7655a3640 1 -- 172.21.15.76:0/2213149287 learned_addr learned my addr 172.21.15.76:0/2213149287 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd765da4640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7601021e0 0x7fd7601a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:53.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.280+0000 7fd764da2640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd760101820 0x7fd760199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7655a3640 1 -- 172.21.15.76:0/2213149287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7601021e0 msgr2=0x7fd7601a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7655a3640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7601021e0 0x7fd7601a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7655a3640 1 -- 172.21.15.76:0/2213149287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd760101820 msgr2=0x7fd760199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7655a3640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd760101820 0x7fd760199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7655a3640 1 -- 172.21.15.76:0/2213149287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd758009840 con 0x7fd760100630 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd765da4640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7601021e0 0x7fd7601a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7655a3640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760100630 0x7fd760199a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd75000e9c0 tx=0x7fd75000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:53.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7567fc640 1 -- 172.21.15.76:0/2213149287 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500096e0 con 0x7fd760100630 2024-03-23T15:40:53.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7567fc640 1 -- 172.21.15.76:0/2213149287 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75000ccc0 con 0x7fd760100630 2024-03-23T15:40:53.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c009560 con 0x7fd760100630 2024-03-23T15:40:53.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd7567fc640 1 -- 172.21.15.76:0/2213149287 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750010430 con 0x7fd760100630 2024-03-23T15:40:53.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.281+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7601a1e10 con 0x7fd760100630 2024-03-23T15:40:53.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.282+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76010c040 con 0x7fd760100630 2024-03-23T15:40:53.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.283+0000 7fd7567fc640 1 -- 172.21.15.76:0/2213149287 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fd750003f60 con 0x7fd760100630 2024-03-23T15:40:53.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.284+0000 7fd7567fc640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd74c0a20c0 0x7fd74c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:53.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.284+0000 7fd764da2640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd74c0a20c0 0x7fd74c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:53.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.284+0000 7fd7567fc640 1 -- 172.21.15.76:0/2213149287 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd750014070 con 0x7fd760100630 2024-03-23T15:40:53.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.284+0000 7fd764da2640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd74c0a20c0 0x7fd74c0a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009230 tx=0x7fd75c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:53.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.288+0000 7fd7567fc640 1 -- 172.21.15.76:0/2213149287 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd750097110 con 0x7fd760100630 2024-03-23T15:40:53.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.499+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7601a2030 con 0x7fd760100630 2024-03-23T15:40:53.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.499+0000 7fd7567fc640 1 -- 172.21.15.76:0/2213149287 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fd75009a7d0 con 0x7fd760100630 2024-03-23T15:40:53.502 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:53.502 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-23T15:40:53.502 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 6 2024-03-23T15:40:53.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd74c0a20c0 msgr2=0x7fd74c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:53.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd74c0a20c0 0x7fd74c0a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009230 tx=0x7fd75c005fd0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760100630 msgr2=0x7fd760199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760100630 0x7fd760199a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd75000e9c0 tx=0x7fd75000edc0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 shutdown_connections 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7601021e0 0x7fd7601a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd74c0a20c0 0x7fd74c0a4580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd760101820 0x7fd760199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 --2- 172.21.15.76:0/2213149287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd760100630 0x7fd760199a10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 >> 172.21.15.76:0/2213149287 conn(0x7fd7600fbba0 msgr2=0x7fd760068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 shutdown_connections 2024-03-23T15:40:53.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:53.501+0000 7fd767807640 1 -- 172.21.15.76:0/2213149287 wait complete. 2024-03-23T15:40:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:53 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6834/1781188743,v1:172.21.15.183:6835/1781188743] up:boot 2024-03-23T15:40:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:53 smithi192 ceph-mon[36653]: fsmap 5 up:standby 2024-03-23T15:40:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:53 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:40:53.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:53 smithi192 ceph-mon[36653]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:53.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:53 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2213149287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:53.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:53 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6834/1781188743,v1:172.21.15.183:6835/1781188743] up:boot 2024-03-23T15:40:53.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:53 smithi076 ceph-mon[32176]: fsmap 5 up:standby 2024-03-23T15:40:53.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:53 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:40:53.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:53 smithi076 ceph-mon[32176]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:53.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:53 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2213149287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:53 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6834/1781188743,v1:172.21.15.183:6835/1781188743] up:boot 2024-03-23T15:40:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:53 smithi183 ceph-mon[36296]: fsmap 5 up:standby 2024-03-23T15:40:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:53 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:40:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:53 smithi183 ceph-mon[36296]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:53 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2213149287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:54.819 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:54 smithi192 systemd[1]: Starting Ceph mds.f for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:40:55.166 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:54 smithi192 podman[67301]: 2024-03-23 15:40:54.768839289 +0000 UTC m=+0.017709319 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:40:55.166 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:54 smithi192 podman[67301]: 2024-03-23 15:40:54.894551196 +0000 UTC m=+0.143421211 container create f049503cec02a660a40df821b6126ee6d5896530baa99e37eb11c6e455164bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-23T15:40:55.167 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:55 smithi192 podman[67301]: 2024-03-23 15:40:55.164467199 +0000 UTC m=+0.413337214 container init f049503cec02a660a40df821b6126ee6d5896530baa99e37eb11c6e455164bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:40:55.515 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:55.569 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:55 smithi192 podman[67301]: 2024-03-23 15:40:55.167324803 +0000 UTC m=+0.416194810 container start f049503cec02a660a40df821b6126ee6d5896530baa99e37eb11c6e455164bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:40:55.569 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:55 smithi192 ceph-mds[67320]: -- 172.21.15.192:0/675018819 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dc2c781a0 con 0x558dc2c4ec00 2024-03-23T15:40:55.569 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:55 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f[67316]: starting mds.f at 2024-03-23T15:40:55.569 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:55 smithi192 bash[67301]: f049503cec02a660a40df821b6126ee6d5896530baa99e37eb11c6e455164bab 2024-03-23T15:40:55.570 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:40:55 smithi192 systemd[1]: Started Ceph mds.f for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 -- 172.21.15.76:0/218377233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4101740 msgr2=0x7fefa4101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 --2- 172.21.15.76:0/218377233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4101740 0x7fefa4101b60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef94009900 tx=0x7fef9402bc30 comp rx=0 tx=0).stop 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 -- 172.21.15.76:0/218377233 shutdown_connections 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 --2- 172.21.15.76:0/218377233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa41032f0 0x7fefa4109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 --2- 172.21.15.76:0/218377233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa4102930 0x7fefa4102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 --2- 172.21.15.76:0/218377233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4101740 0x7fefa4101b60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 -- 172.21.15.76:0/218377233 >> 172.21.15.76:0/218377233 conn(0x7fefa40fccf0 msgr2=0x7fefa40ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 -- 172.21.15.76:0/218377233 shutdown_connections 2024-03-23T15:40:55.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 -- 172.21.15.76:0/218377233 wait complete. 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 Processor -- start 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.625+0000 7fefab4b5640 1 -- start start 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa4101740 0x7fefa4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4102930 0x7fefa4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa41032f0 0x7fefa419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefa419f310 con 0x7fefa4102930 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefa419f480 con 0x7fefa41032f0 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fefa419f640 con 0x7fefa4101740 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4102930 0x7fefa4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4102930 0x7fefa4197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53648/0 (socket says 172.21.15.76:53648) 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 -- 172.21.15.76:0/3700110538 learned_addr learned my addr 172.21.15.76:0/3700110538 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa9251640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa4101740 0x7fefa4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:55.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa9a52640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa41032f0 0x7fefa419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 -- 172.21.15.76:0/3700110538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa4101740 msgr2=0x7fefa4197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa4101740 0x7fefa4197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 -- 172.21.15.76:0/3700110538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa41032f0 msgr2=0x7fefa419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa41032f0 0x7fefa419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 -- 172.21.15.76:0/3700110538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef94009560 con 0x7fefa4102930 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa9a52640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa41032f0 0x7fefa419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefa8a50640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4102930 0x7fefa4197d60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefa000d950 tx=0x7fefa000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fef9a7fc640 1 -- 172.21.15.76:0/3700110538 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0009a30 con 0x7fefa4102930 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa419f920 con 0x7fefa4102930 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fef9a7fc640 1 -- 172.21.15.76:0/3700110538 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0009b90 con 0x7fefa4102930 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fef9a7fc640 1 -- 172.21.15.76:0/3700110538 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0004a00 con 0x7fefa4102930 2024-03-23T15:40:55.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.626+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa419fe30 con 0x7fefa4102930 2024-03-23T15:40:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.627+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa4108b50 con 0x7fefa4102930 2024-03-23T15:40:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.628+0000 7fef9a7fc640 1 -- 172.21.15.76:0/3700110538 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fefa0009d00 con 0x7fefa4102930 2024-03-23T15:40:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.629+0000 7fef9a7fc640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fef900a20c0 0x7fef900a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.629+0000 7fefa9251640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fef900a20c0 0x7fef900a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:55.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.629+0000 7fef9a7fc640 1 -- 172.21.15.76:0/3700110538 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fefa00cd290 con 0x7fefa4102930 2024-03-23T15:40:55.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.629+0000 7fefa9251640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fef900a20c0 0x7fef900a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fef94005ab0 tx=0x7fef94005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:55.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.633+0000 7fef9a7fc640 1 -- 172.21.15.76:0/3700110538 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fefa0096700 con 0x7fefa4102930 2024-03-23T15:40:55.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.844+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefa410d130 con 0x7fefa4102930 2024-03-23T15:40:55.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.845+0000 7fef9a7fc640 1 -- 172.21.15.76:0/3700110538 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7fefa0099dc0 con 0x7fefa4102930 2024-03-23T15:40:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:55.847 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-23T15:40:55.847 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 7 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fef900a20c0 msgr2=0x7fef900a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fef900a20c0 0x7fef900a4580 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fef94005ab0 tx=0x7fef94005fd0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4102930 msgr2=0x7fefa4197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4102930 0x7fefa4197d60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefa000d950 tx=0x7fefa000de10 comp rx=0 tx=0).stop 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 shutdown_connections 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fef900a20c0 0x7fef900a4580 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa41032f0 0x7fefa419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefa4102930 0x7fefa4197d60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 --2- 172.21.15.76:0/3700110538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa4101740 0x7fefa4197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 >> 172.21.15.76:0/3700110538 conn(0x7fefa40fccf0 msgr2=0x7fefa4105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:55.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 shutdown_connections 2024-03-23T15:40:55.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:55.847+0000 7fefab4b5640 1 -- 172.21.15.76:0/3700110538 wait complete. 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6834/1421961871,v1:172.21.15.192:6835/1421961871] up:boot 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: fsmap 6 up:standby 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: Deploying daemon mds.c on smithi192 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:56.455 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:56 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3700110538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:56.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6834/1421961871,v1:172.21.15.192:6835/1421961871] up:boot 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: fsmap 6 up:standby 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: Deploying daemon mds.c on smithi192 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:56.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:56 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3700110538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:56.586 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6834/1421961871,v1:172.21.15.192:6835/1421961871] up:boot 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: fsmap 6 up:standby 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: Deploying daemon mds.c on smithi192 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:56.587 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:56 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3700110538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:57.571 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:57 smithi192 systemd[1]: Starting Ceph mds.c for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:40:57.861 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:40:57.940 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:57 smithi192 podman[67626]: 2024-03-23 15:40:57.568992145 +0000 UTC m=+0.017949003 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:40:57.941 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:57 smithi192 podman[67626]: 2024-03-23 15:40:57.685945176 +0000 UTC m=+0.134902010 container create 982978143faa59a4208ff6d61bc090a2a3f13acb6fff350ad3888bd8698201e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-23T15:40:57.941 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:57 smithi192 podman[67626]: 2024-03-23 15:40:57.938686669 +0000 UTC m=+0.387643528 container init 982978143faa59a4208ff6d61bc090a2a3f13acb6fff350ad3888bd8698201e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-23T15:40:57.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 -- 172.21.15.76:0/614477076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 msgr2=0x7f0a4c100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:57.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 --2- 172.21.15.76:0/614477076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c100a70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009900 tx=0x7f0a4802bc30 comp rx=0 tx=0).stop 2024-03-23T15:40:57.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 -- 172.21.15.76:0/614477076 shutdown_connections 2024-03-23T15:40:57.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 --2- 172.21.15.76:0/614477076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a4c102200 0x7f0a4c108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:57.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 --2- 172.21.15.76:0/614477076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c101840 0x7f0a4c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:57.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 --2- 172.21.15.76:0/614477076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c100a70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:57.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 -- 172.21.15.76:0/614477076 >> 172.21.15.76:0/614477076 conn(0x7f0a4c0fbbc0 msgr2=0x7f0a4c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:57.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 -- 172.21.15.76:0/614477076 shutdown_connections 2024-03-23T15:40:57.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 -- 172.21.15.76:0/614477076 wait complete. 2024-03-23T15:40:57.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 Processor -- start 2024-03-23T15:40:57.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.974+0000 7f0a5475e640 1 -- start start 2024-03-23T15:40:57.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a5475e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a5475e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c101840 0x7f0a4c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a5475e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a4c102200 0x7f0a4c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a5475e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c19f380 con 0x7f0a4c100650 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a5475e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c19f4f0 con 0x7f0a4c102200 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a5475e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c19f6b0 con 0x7f0a4c101840 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53664/0 (socket says 172.21.15.76:53664) 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 -- 172.21.15.76:0/3180919943 learned_addr learned my addr 172.21.15.76:0/3180919943 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a52cfb640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a4c102200 0x7f0a4c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a51cf9640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c101840 0x7f0a4c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 -- 172.21.15.76:0/3180919943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c101840 msgr2=0x7f0a4c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c101840 0x7f0a4c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 -- 172.21.15.76:0/3180919943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a4c102200 msgr2=0x7f0a4c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:57.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a4c102200 0x7f0a4c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 -- 172.21.15.76:0/3180919943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a48009560 con 0x7f0a4c100650 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a524fa640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c197890 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a48036040 tx=0x7f0a48005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a437fe640 1 -- 172.21.15.76:0/3180919943 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48004410 con 0x7f0a4c100650 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a437fe640 1 -- 172.21.15.76:0/3180919943 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a480052d0 con 0x7f0a4c100650 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a437fe640 1 -- 172.21.15.76:0/3180919943 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48003d70 con 0x7f0a4c100650 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.975+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c19f930 con 0x7f0a4c100650 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.976+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c19fe90 con 0x7f0a4c100650 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.977+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c10c030 con 0x7f0a4c100650 2024-03-23T15:40:57.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.978+0000 7f0a437fe640 1 -- 172.21.15.76:0/3180919943 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f0a48005440 con 0x7f0a4c100650 2024-03-23T15:40:57.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.978+0000 7f0a437fe640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a380a2140 0x7f0a380a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:40:57.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.978+0000 7f0a51cf9640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a380a2140 0x7f0a380a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:40:57.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.978+0000 7f0a437fe640 1 -- 172.21.15.76:0/3180919943 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0a480ef820 con 0x7f0a4c100650 2024-03-23T15:40:57.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.979+0000 7f0a51cf9640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a380a2140 0x7f0a380a4600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c198da0 tx=0x7f0a3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:40:57.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:57.983+0000 7f0a437fe640 1 -- 172.21.15.76:0/3180919943 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0a480b8c90 con 0x7f0a4c100650 2024-03-23T15:40:58.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.192+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a4c10c1f0 con 0x7f0a4c100650 2024-03-23T15:40:58.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.193+0000 7f0a437fe640 1 -- 172.21.15.76:0/3180919943 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f0a480bc350 con 0x7f0a4c100650 2024-03-23T15:40:58.196 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:40:58.196 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-23T15:40:58.196 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 7 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a380a2140 msgr2=0x7f0a380a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a380a2140 0x7f0a380a4600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c198da0 tx=0x7f0a3c005f90 comp rx=0 tx=0).stop 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 msgr2=0x7f0a4c197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c197890 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a48036040 tx=0x7f0a48005dd0 comp rx=0 tx=0).stop 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 shutdown_connections 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a380a2140 0x7f0a380a4600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a4c102200 0x7f0a4c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c101840 0x7f0a4c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 --2- 172.21.15.76:0/3180919943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a4c100650 0x7f0a4c197890 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 >> 172.21.15.76:0/3180919943 conn(0x7f0a4c0fbbc0 msgr2=0x7f0a4c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 shutdown_connections 2024-03-23T15:40:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:40:58.195+0000 7f0a5475e640 1 -- 172.21.15.76:0/3180919943 wait complete. 2024-03-23T15:40:58.319 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:57 smithi192 podman[67626]: 2024-03-23 15:40:57.941457373 +0000 UTC m=+0.390414224 container start 982978143faa59a4208ff6d61bc090a2a3f13acb6fff350ad3888bd8698201e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-23T15:40:58.320 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:57 smithi192 ceph-mds[67645]: -- 172.21.15.192:0/2046326008 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b253ca81a0 con 0x55b253c7ec00 2024-03-23T15:40:58.320 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:57 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c[67641]: starting mds.c at 2024-03-23T15:40:58.320 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:58 smithi192 bash[67626]: 982978143faa59a4208ff6d61bc090a2a3f13acb6fff350ad3888bd8698201e9 2024-03-23T15:40:58.320 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:40:58 smithi192 systemd[1]: Started Ceph mds.c for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3180919943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:58 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3180919943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:58.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:58.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:58 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:58.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:40:58.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3180919943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:40:58.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:40:58.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:40:58.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:40:58.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:58 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:40:59.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:59 smithi192 ceph-mon[36653]: Deploying daemon mds.h on smithi183 2024-03-23T15:40:59.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:59 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6836/26718199,v1:172.21.15.192:6837/26718199] up:boot 2024-03-23T15:40:59.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:59 smithi192 ceph-mon[36653]: fsmap 7 up:standby 2024-03-23T15:40:59.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:40:59 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:40:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:59 smithi076 ceph-mon[32176]: Deploying daemon mds.h on smithi183 2024-03-23T15:40:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:59 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6836/26718199,v1:172.21.15.192:6837/26718199] up:boot 2024-03-23T15:40:59.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:59 smithi076 ceph-mon[32176]: fsmap 7 up:standby 2024-03-23T15:40:59.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:40:59 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:40:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:59 smithi183 ceph-mon[36296]: Deploying daemon mds.h on smithi183 2024-03-23T15:40:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:59 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6836/26718199,v1:172.21.15.192:6837/26718199] up:boot 2024-03-23T15:40:59.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:59 smithi183 ceph-mon[36296]: fsmap 7 up:standby 2024-03-23T15:40:59.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:40:59 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:41:00.209 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:00.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 -- 172.21.15.76:0/159557380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894104c00 msgr2=0x7f5894106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 --2- 172.21.15.76:0/159557380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894104c00 0x7f5894106ff0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5888009900 tx=0x7f588802bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 -- 172.21.15.76:0/159557380 shutdown_connections 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 --2- 172.21.15.76:0/159557380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5894107530 0x7f5894109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 --2- 172.21.15.76:0/159557380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5894104c00 0x7f5894106ff0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 --2- 172.21.15.76:0/159557380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58941022d0 0x7f58941046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 -- 172.21.15.76:0/159557380 >> 172.21.15.76:0/159557380 conn(0x7f58940fbc40 msgr2=0x7f58940fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 -- 172.21.15.76:0/159557380 shutdown_connections 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.319+0000 7f589a1d7640 1 -- 172.21.15.76:0/159557380 wait complete. 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 Processor -- start 2024-03-23T15:41:00.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 -- start start 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58941022d0 0x7f5894197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5894104c00 0x7f5894197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5894107530 0x7f589419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f589419f330 con 0x7f58941022d0 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f589419f4a0 con 0x7f5894104c00 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f589a1d7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f589419f660 con 0x7f5894107530 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f58937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58941022d0 0x7f5894197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f58937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58941022d0 0x7f5894197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53672/0 (socket says 172.21.15.76:53672) 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f58937fe640 1 -- 172.21.15.76:0/1600881935 learned_addr learned my addr 172.21.15.76:0/1600881935 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f5892ffd640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5894104c00 0x7f5894197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.320+0000 7f5893fff640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5894107530 0x7f589419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:00.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f58937fe640 1 -- 172.21.15.76:0/1600881935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5894107530 msgr2=0x7f589419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f58937fe640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5894107530 0x7f589419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f58937fe640 1 -- 172.21.15.76:0/1600881935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5894104c00 msgr2=0x7f5894197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f58937fe640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5894104c00 0x7f5894197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f58937fe640 1 -- 172.21.15.76:0/1600881935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5888009560 con 0x7f58941022d0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f5893fff640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5894107530 0x7f589419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f58937fe640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58941022d0 0x7f5894197840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f588400e9f0 tx=0x7f588400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f5890ff9640 1 -- 172.21.15.76:0/1600881935 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884009720 con 0x7f58941022d0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589419f940 con 0x7f58941022d0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f5890ff9640 1 -- 172.21.15.76:0/1600881935 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f588400cd80 con 0x7f58941022d0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f5890ff9640 1 -- 172.21.15.76:0/1600881935 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884010690 con 0x7f58941022d0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.321+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589419fe20 con 0x7f58941022d0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.322+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58940fd730 con 0x7f58941022d0 2024-03-23T15:41:00.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.323+0000 7f5890ff9640 1 -- 172.21.15.76:0/1600881935 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f58840046c0 con 0x7f58941022d0 2024-03-23T15:41:00.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.323+0000 7f5890ff9640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f587c0a20c0 0x7f587c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:00.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.324+0000 7f5892ffd640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f587c0a20c0 0x7f587c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:00.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.324+0000 7f5890ff9640 1 -- 172.21.15.76:0/1600881935 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5884014070 con 0x7f58941022d0 2024-03-23T15:41:00.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.324+0000 7f5892ffd640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f587c0a20c0 0x7f587c0a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5888005ab0 tx=0x7f5888005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:00.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.328+0000 7f5890ff9640 1 -- 172.21.15.76:0/1600881935 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f5884097090 con 0x7f58941022d0 2024-03-23T15:41:00.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.539+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f589410d140 con 0x7f58941022d0 2024-03-23T15:41:00.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.539+0000 7f5890ff9640 1 -- 172.21.15.76:0/1600881935 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f588409a750 con 0x7f58941022d0 2024-03-23T15:41:00.542 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:00.542 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-23T15:41:00.542 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-03-23T15:41:00.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f587c0a20c0 msgr2=0x7f587c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:00.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f587c0a20c0 0x7f587c0a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5888005ab0 tx=0x7f5888005fd0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58941022d0 msgr2=0x7f5894197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:00.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58941022d0 0x7f5894197840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f588400e9f0 tx=0x7f588400edf0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 shutdown_connections 2024-03-23T15:41:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5894107530 0x7f589419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f587c0a20c0 0x7f587c0a4580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5894104c00 0x7f5894197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 --2- 172.21.15.76:0/1600881935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58941022d0 0x7f5894197840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 >> 172.21.15.76:0/1600881935 conn(0x7f58940fbc40 msgr2=0x7f5894102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:00.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 shutdown_connections 2024-03-23T15:41:00.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:00.541+0000 7f589a1d7640 1 -- 172.21.15.76:0/1600881935 wait complete. 2024-03-23T15:41:00.546 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 systemd[1]: Starting Ceph mds.h for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:41:00.796 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:00 smithi183 ceph-mon[36296]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:00.796 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:00 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1600881935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:00.797 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 podman[66684]: 2024-03-23 15:41:00.573380659 +0000 UTC m=+0.018312537 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:41:00.797 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 podman[66684]: 2024-03-23 15:41:00.682939844 +0000 UTC m=+0.127871701 container create be9a99f2bdc163e76bc13b55d979ec33c44974cd3ce3ad94ae848a8886c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:41:00.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:00 smithi192 ceph-mon[36653]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:00.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:00 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1600881935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:00.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:00 smithi076 ceph-mon[32176]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:00.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:00 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1600881935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:01.133 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 podman[66684]: 2024-03-23 15:41:00.902296492 +0000 UTC m=+0.347228361 container init be9a99f2bdc163e76bc13b55d979ec33c44974cd3ce3ad94ae848a8886c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-23T15:41:01.133 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 podman[66684]: 2024-03-23 15:41:00.904928324 +0000 UTC m=+0.349860197 container start be9a99f2bdc163e76bc13b55d979ec33c44974cd3ce3ad94ae848a8886c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5) 2024-03-23T15:41:01.133 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 ceph-mds[66702]: -- 172.21.15.183:0/2676313252 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599840b81a0 con 0x55998408e800 2024-03-23T15:41:01.133 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: starting mds.h at 2024-03-23T15:41:01.133 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 bash[66684]: be9a99f2bdc163e76bc13b55d979ec33c44974cd3ce3ad94ae848a8886c511d7 2024-03-23T15:41:01.134 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:41:00 smithi183 systemd[1]: Started Ceph mds.h for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:41:02.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: Deploying daemon mds.i on smithi192 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:boot 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: fsmap 8 up:standby 2024-03-23T15:41:02.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:02 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:41:02.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: Deploying daemon mds.i on smithi192 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:boot 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: fsmap 8 up:standby 2024-03-23T15:41:02.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:02 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:41:02.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: Deploying daemon mds.i on smithi192 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:boot 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: fsmap 8 up:standby 2024-03-23T15:41:02.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:02 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:41:02.557 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:02.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 -- 172.21.15.76:0/46587119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65341052e0 msgr2=0x7f6534105760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:02.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 --2- 172.21.15.76:0/46587119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65341052e0 0x7f6534105760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f652c009ca0 tx=0x7f652c02c0f0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 -- 172.21.15.76:0/46587119 shutdown_connections 2024-03-23T15:41:02.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 --2- 172.21.15.76:0/46587119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65341052e0 0x7f6534105760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 --2- 172.21.15.76:0/46587119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6534102930 0x7f6534104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 --2- 172.21.15.76:0/46587119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65340fffc0 0x7f65341023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 -- 172.21.15.76:0/46587119 >> 172.21.15.76:0/46587119 conn(0x7f65340fbbc0 msgr2=0x7f65340fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 -- 172.21.15.76:0/46587119 shutdown_connections 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.672+0000 7f653b356640 1 -- 172.21.15.76:0/46587119 wait complete. 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 Processor -- start 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 -- start start 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65340fffc0 0x7f653419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6534102930 0x7f653419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65341052e0 0x7f65341a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65341a5830 con 0x7f6534102930 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65341a59a0 con 0x7f65341052e0 2024-03-23T15:41:02.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f653b356640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65341a5b60 con 0x7f65340fffc0 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6534102930 0x7f653419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6534102930 0x7f653419e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53690/0 (socket says 172.21.15.76:53690) 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 -- 172.21.15.76:0/2661299544 learned_addr learned my addr 172.21.15.76:0/2661299544 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65390f2640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65340fffc0 0x7f653419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 -- 172.21.15.76:0/2661299544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65340fffc0 msgr2=0x7f653419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65340fffc0 0x7f653419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65398f3640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65341052e0 0x7f65341a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 -- 172.21.15.76:0/2661299544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65341052e0 msgr2=0x7f65341a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65341052e0 0x7f65341a52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.673+0000 7f65388f1640 1 -- 172.21.15.76:0/2661299544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c009900 con 0x7f6534102930 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.674+0000 7f65388f1640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6534102930 0x7f653419e280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f653000c890 tx=0x7f653000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.674+0000 7f652a7fc640 1 -- 172.21.15.76:0/2661299544 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530004220 con 0x7f6534102930 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.674+0000 7f652a7fc640 1 -- 172.21.15.76:0/2661299544 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6530007e90 con 0x7f6534102930 2024-03-23T15:41:02.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.674+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65341a5e40 con 0x7f6534102930 2024-03-23T15:41:02.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.674+0000 7f652a7fc640 1 -- 172.21.15.76:0/2661299544 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530004a00 con 0x7f6534102930 2024-03-23T15:41:02.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.674+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65341a6310 con 0x7f6534102930 2024-03-23T15:41:02.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.675+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65340fd6b0 con 0x7f6534102930 2024-03-23T15:41:02.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.676+0000 7f652a7fc640 1 -- 172.21.15.76:0/2661299544 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f65300079b0 con 0x7f6534102930 2024-03-23T15:41:02.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.676+0000 7f652a7fc640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f65200a2140 0x7f65200a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:02.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.677+0000 7f65390f2640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f65200a2140 0x7f65200a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:02.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.677+0000 7f652a7fc640 1 -- 172.21.15.76:0/2661299544 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f65300cd880 con 0x7f6534102930 2024-03-23T15:41:02.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.677+0000 7f65390f2640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f65200a2140 0x7f65200a4600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6524005d00 tx=0x7f6524005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:02.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.680+0000 7f652a7fc640 1 -- 172.21.15.76:0/2661299544 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6530096d70 con 0x7f6534102930 2024-03-23T15:41:02.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.892+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6534113720 con 0x7f6534102930 2024-03-23T15:41:02.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.893+0000 7f652a7fc640 1 -- 172.21.15.76:0/2661299544 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f653001d090 con 0x7f6534102930 2024-03-23T15:41:02.895 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:02.895 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-23T15:41:02.896 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2024-03-23T15:41:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f65200a2140 msgr2=0x7f65200a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f65200a2140 0x7f65200a4600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6524005d00 tx=0x7f6524005c50 comp rx=0 tx=0).stop 2024-03-23T15:41:02.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6534102930 msgr2=0x7f653419e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6534102930 0x7f653419e280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f653000c890 tx=0x7f653000cd50 comp rx=0 tx=0).stop 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 shutdown_connections 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f65200a2140 0x7f65200a4600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65341052e0 0x7f65341a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6534102930 0x7f653419e280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 --2- 172.21.15.76:0/2661299544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65340fffc0 0x7f653419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 >> 172.21.15.76:0/2661299544 conn(0x7f65340fbbc0 msgr2=0x7f6534100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 shutdown_connections 2024-03-23T15:41:02.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:02.895+0000 7f653b356640 1 -- 172.21.15.76:0/2661299544 wait complete. 2024-03-23T15:41:03.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:03 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2661299544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:03.320 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 systemd[1]: Starting Ceph mds.i for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:41:03.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:03 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2661299544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:03.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:03 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2661299544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:03.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 podman[67930]: 2024-03-23 15:41:03.362911629 +0000 UTC m=+0.017641057 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:41:03.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 podman[67930]: 2024-03-23 15:41:03.468067264 +0000 UTC m=+0.122796708 container create 0bc7bf21b48ab05ace1937d2f5ec0b5e7147d0909390ee3f9232b639f00b62ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-23T15:41:03.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 podman[67930]: 2024-03-23 15:41:03.812736246 +0000 UTC m=+0.467465671 container init 0bc7bf21b48ab05ace1937d2f5ec0b5e7147d0909390ee3f9232b639f00b62ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-23T15:41:04.069 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 podman[67930]: 2024-03-23 15:41:03.815803838 +0000 UTC m=+0.470533272 container start 0bc7bf21b48ab05ace1937d2f5ec0b5e7147d0909390ee3f9232b639f00b62ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-03-23T15:41:04.069 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 ceph-mds[67949]: -- 172.21.15.192:0/893218814 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ff6c741a0 con 0x562ff6c4b000 2024-03-23T15:41:04.070 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: starting mds.i at 2024-03-23T15:41:04.070 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 bash[67930]: 0bc7bf21b48ab05ace1937d2f5ec0b5e7147d0909390ee3f9232b639f00b62ce 2024-03-23T15:41:04.070 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:41:03 smithi192 systemd[1]: Started Ceph mds.i for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:41:04.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:04 smithi183 ceph-mon[36296]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:04.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:04.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:04.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:04 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:04.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:04 smithi192 ceph-mon[36653]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:04.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:04.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:04.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:04 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:04.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:04 smithi076 ceph-mon[32176]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:04.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:04.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:04.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:04.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:04 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:04.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 -- 172.21.15.76:0/1812105739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7181032f0 msgr2=0x7fa718109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 --2- 172.21.15.76:0/1812105739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7181032f0 0x7fa718109b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa710009ca0 tx=0x7fa71002c0f0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 -- 172.21.15.76:0/1812105739 shutdown_connections 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 --2- 172.21.15.76:0/1812105739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7181032f0 0x7fa718109b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 --2- 172.21.15.76:0/1812105739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa718102930 0x7fa718102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 --2- 172.21.15.76:0/1812105739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa718101740 0x7fa718101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 -- 172.21.15.76:0/1812105739 >> 172.21.15.76:0/1812105739 conn(0x7fa7180fccf0 msgr2=0x7fa7180ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 -- 172.21.15.76:0/1812105739 shutdown_connections 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.014+0000 7fa72021c640 1 -- 172.21.15.76:0/1812105739 wait complete. 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 Processor -- start 2024-03-23T15:41:05.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 -- start start 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa718101740 0x7fa7181977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa718102930 0x7fa718197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7181032f0 0x7fa71819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa71819f2e0 con 0x7fa718102930 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa71819f450 con 0x7fa718101740 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa72021c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa71819f610 con 0x7fa7181032f0 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa718102930 0x7fa718197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa718102930 0x7fa718197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55136/0 (socket says 172.21.15.76:55136) 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 -- 172.21.15.76:0/3224978904 learned_addr learned my addr 172.21.15.76:0/3224978904 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71e7b9640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7181032f0 0x7fa71819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 -- 172.21.15.76:0/3224978904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7181032f0 msgr2=0x7fa71819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7181032f0 0x7fa71819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 -- 172.21.15.76:0/3224978904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa718101740 msgr2=0x7fa7181977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:05.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa718101740 0x7fa7181977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.015+0000 7fa71d7b7640 1 -- 172.21.15.76:0/3224978904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa710009900 con 0x7fa718102930 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.016+0000 7fa71d7b7640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa718102930 0x7fa718197d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa70800d770 tx=0x7fa70800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.016+0000 7fa70effd640 1 -- 172.21.15.76:0/3224978904 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708009c30 con 0x7fa718102930 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.016+0000 7fa70effd640 1 -- 172.21.15.76:0/3224978904 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70800de80 con 0x7fa718102930 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.016+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71819f8f0 con 0x7fa718102930 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.016+0000 7fa70effd640 1 -- 172.21.15.76:0/3224978904 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708004e80 con 0x7fa718102930 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.016+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71819fdb0 con 0x7fa718102930 2024-03-23T15:41:05.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.017+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718108a60 con 0x7fa718102930 2024-03-23T15:41:05.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.018+0000 7fa70effd640 1 -- 172.21.15.76:0/3224978904 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fa708005000 con 0x7fa718102930 2024-03-23T15:41:05.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.019+0000 7fa70effd640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7040a20c0 0x7fa7040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:05.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.019+0000 7fa71dfb8640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7040a20c0 0x7fa7040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:05.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.019+0000 7fa70effd640 1 -- 172.21.15.76:0/3224978904 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa7080cd660 con 0x7fa718102930 2024-03-23T15:41:05.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.019+0000 7fa71dfb8640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7040a20c0 0x7fa7040a4580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa718102790 tx=0x7fa714006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:05.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.023+0000 7fa70effd640 1 -- 172.21.15.76:0/3224978904 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa708096ad0 con 0x7fa718102930 2024-03-23T15:41:05.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.238+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa71810d3a0 con 0x7fa718102930 2024-03-23T15:41:05.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.239+0000 7fa70effd640 1 -- 172.21.15.76:0/3224978904 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fa70809a190 con 0x7fa718102930 2024-03-23T15:41:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:05.242 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-23T15:41:05.242 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 10 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7040a20c0 msgr2=0x7fa7040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7040a20c0 0x7fa7040a4580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa718102790 tx=0x7fa714006c90 comp rx=0 tx=0).stop 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa718102930 msgr2=0x7fa718197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa718102930 0x7fa718197d30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa70800d770 tx=0x7fa70800dc30 comp rx=0 tx=0).stop 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 shutdown_connections 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7181032f0 0x7fa71819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa718102930 0x7fa718197d30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7040a20c0 0x7fa7040a4580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 --2- 172.21.15.76:0/3224978904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa718101740 0x7fa7181977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:05.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 >> 172.21.15.76:0/3224978904 conn(0x7fa7180fccf0 msgr2=0x7fa718105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:05.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 shutdown_connections 2024-03-23T15:41:05.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:05.241+0000 7fa72021c640 1 -- 172.21.15.76:0/3224978904 wait complete. 2024-03-23T15:41:05.253 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:05 smithi076 ceph-mon[32176]: Deploying daemon mds.g on smithi076 2024-03-23T15:41:05.253 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:05 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:boot 2024-03-23T15:41:05.253 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:05 smithi076 ceph-mon[32176]: fsmap 9 up:standby 2024-03-23T15:41:05.253 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:05 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:41:05.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:05 smithi183 ceph-mon[36296]: Deploying daemon mds.g on smithi076 2024-03-23T15:41:05.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:05 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:boot 2024-03-23T15:41:05.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:05 smithi183 ceph-mon[36296]: fsmap 9 up:standby 2024-03-23T15:41:05.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:05 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:41:05.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:05 smithi192 ceph-mon[36653]: Deploying daemon mds.g on smithi076 2024-03-23T15:41:05.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:05 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:boot 2024-03-23T15:41:05.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:05 smithi192 ceph-mon[36653]: fsmap 9 up:standby 2024-03-23T15:41:05.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:05 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:41:06.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:06 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3224978904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:06.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:06 smithi076 ceph-mon[32176]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:06.326 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 systemd[1]: Starting Ceph mds.g for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:41:06.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:06 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3224978904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:06.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:06 smithi183 ceph-mon[36296]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:06.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:06 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3224978904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:06.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:06 smithi192 ceph-mon[36653]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:06.735 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 podman[71434]: 2024-03-23 15:41:06.333020722 +0000 UTC m=+0.017889972 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:41:06.736 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 podman[71434]: 2024-03-23 15:41:06.447693099 +0000 UTC m=+0.132562402 container create 4776c633b3457427a871e89c3e71b553d2ce639bb3eaecfd7c234ca076e9a90f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-23T15:41:06.736 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 podman[71434]: 2024-03-23 15:41:06.733625835 +0000 UTC m=+0.418495081 container init 4776c633b3457427a871e89c3e71b553d2ce639bb3eaecfd7c234ca076e9a90f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:41:07.075 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 podman[71434]: 2024-03-23 15:41:06.736744721 +0000 UTC m=+0.421613959 container start 4776c633b3457427a871e89c3e71b553d2ce639bb3eaecfd7c234ca076e9a90f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-23T15:41:07.076 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 ceph-mds[71453]: -- 172.21.15.76:0/911096001 <== mon.0 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8b180c1a0 con 0x55b8b17e3000 2024-03-23T15:41:07.076 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g[71449]: starting mds.g at 2024-03-23T15:41:07.076 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 bash[71434]: 4776c633b3457427a871e89c3e71b553d2ce639bb3eaecfd7c234ca076e9a90f 2024-03-23T15:41:07.076 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:41:06 smithi076 systemd[1]: Started Ceph mds.g for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:41:07.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:07.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 -- 172.21.15.76:0/3273980261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 msgr2=0x7f84c0108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:07.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/3273980261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 0x7f84c0108a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009ca0 tx=0x7f84b802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 -- 172.21.15.76:0/3273980261 shutdown_connections 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/3273980261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 0x7f84c0108a70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/3273980261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c0101820 0x7f84c0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/3273980261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 0x7f84c0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 -- 172.21.15.76:0/3273980261 >> 172.21.15.76:0/3273980261 conn(0x7f84c00fbba0 msgr2=0x7f84c00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 -- 172.21.15.76:0/3273980261 shutdown_connections 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.361+0000 7f84c6a6b640 1 -- 172.21.15.76:0/3273980261 wait complete. 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 Processor -- start 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 -- start start 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 0x7f84c0069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c0101820 0x7f84c019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 0x7f84c019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84c019f290 con 0x7f84c01021e0 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84c019f400 con 0x7f84c0100630 2024-03-23T15:41:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c6a6b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f84c019f5c0 con 0x7f84c0101820 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c4807640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 0x7f84c0069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c5008640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 0x7f84c019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c4807640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 0x7f84c0069400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:51668/0 (socket says 172.21.15.76:51668) 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c4807640 1 -- 172.21.15.76:0/4189016233 learned_addr learned my addr 172.21.15.76:0/4189016233 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84bffff640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c0101820 0x7f84c019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c4807640 1 -- 172.21.15.76:0/4189016233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c0101820 msgr2=0x7f84c019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.362+0000 7f84c4807640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c0101820 0x7f84c019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84c4807640 1 -- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 msgr2=0x7f84c019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84c4807640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 0x7f84c019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84c4807640 1 -- 172.21.15.76:0/4189016233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b8009900 con 0x7f84c0100630 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84c4807640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 0x7f84c0069400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d950 tx=0x7f84b400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84bdffb640 1 -- 172.21.15.76:0/4189016233 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4009c30 con 0x7f84c0100630 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84bdffb640 1 -- 172.21.15.76:0/4189016233 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b4009d90 con 0x7f84c0100630 2024-03-23T15:41:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c019f8a0 con 0x7f84c0100630 2024-03-23T15:41:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84bdffb640 1 -- 172.21.15.76:0/4189016233 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400f850 con 0x7f84c0100630 2024-03-23T15:41:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.363+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c019fdb0 con 0x7f84c0100630 2024-03-23T15:41:07.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.364+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c0107950 con 0x7f84c0100630 2024-03-23T15:41:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.365+0000 7f84bdffb640 1 -- 172.21.15.76:0/4189016233 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f84b400f9b0 con 0x7f84c0100630 2024-03-23T15:41:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.366+0000 7f84bdffb640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f84b00a20c0 0x7f84b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.366+0000 7f84bffff640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f84b00a20c0 0x7f84b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.366+0000 7f84bdffb640 1 -- 172.21.15.76:0/4189016233 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f84b40cd430 con 0x7f84c0100630 2024-03-23T15:41:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.366+0000 7f84bffff640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f84b00a20c0 0x7f84b00a4580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84c0069170 tx=0x7f84ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.370+0000 7f84bdffb640 1 -- 172.21.15.76:0/4189016233 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f84b40968a0 con 0x7f84c0100630 2024-03-23T15:41:07.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.618+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84c010c290 con 0x7f84c0100630 2024-03-23T15:41:07.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.619+0000 7f84bdffb640 1 -- 172.21.15.76:0/4189016233 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f84b4096070 con 0x7f84c0100630 2024-03-23T15:41:07.622 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:07.622 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-23T15:41:07.622 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 11 2024-03-23T15:41:07.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f84b00a20c0 msgr2=0x7f84b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:07.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f84b00a20c0 0x7f84b00a4580 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84c0069170 tx=0x7f84ac00b390 comp rx=0 tx=0).stop 2024-03-23T15:41:07.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 msgr2=0x7f84c0069400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:07.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 0x7f84c0069400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d950 tx=0x7f84b400de10 comp rx=0 tx=0).stop 2024-03-23T15:41:07.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 shutdown_connections 2024-03-23T15:41:07.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84c01021e0 0x7f84c019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c0101820 0x7f84c019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f84b00a20c0 0x7f84b00a4580 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 --2- 172.21.15.76:0/4189016233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0100630 0x7f84c0069400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:07.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 >> 172.21.15.76:0/4189016233 conn(0x7f84c00fbba0 msgr2=0x7f84c0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:07.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 shutdown_connections 2024-03-23T15:41:07.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:07.621+0000 7f84c6a6b640 1 -- 172.21.15.76:0/4189016233 wait complete. 2024-03-23T15:41:08.277 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.277 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.277 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.277 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:08.277 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:08.278 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:08.278 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: Deploying daemon mds.a on smithi076 2024-03-23T15:41:08.278 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.76:6838/2542851051,v1:172.21.15.76:6839/2542851051] up:boot 2024-03-23T15:41:08.278 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: fsmap 10 up:standby 2024-03-23T15:41:08.278 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T15:41:08.278 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:08.278 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:07 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/4189016233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:08.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: Deploying daemon mds.a on smithi076 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.76:6838/2542851051,v1:172.21.15.76:6839/2542851051] up:boot 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: fsmap 10 up:standby 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:08.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:07 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/4189016233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:08.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:08.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:08.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:08.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:08.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: Deploying daemon mds.a on smithi076 2024-03-23T15:41:08.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.76:6838/2542851051,v1:172.21.15.76:6839/2542851051] up:boot 2024-03-23T15:41:08.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: fsmap 10 up:standby 2024-03-23T15:41:08.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T15:41:08.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:08.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:07 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/4189016233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:09.326 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 systemd[1]: Starting Ceph mds.a for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:41:09.592 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 podman[71793]: 2024-03-23 15:41:09.228172404 +0000 UTC m=+0.024209263 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:41:09.592 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 podman[71793]: 2024-03-23 15:41:09.327198839 +0000 UTC m=+0.123235693 container create c4dd412540ede56784f888a68f71239b7787b3ac0b11f05ab24933945b4f876e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:41:09.592 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 podman[71793]: 2024-03-23 15:41:09.56337088 +0000 UTC m=+0.359407742 container init c4dd412540ede56784f888a68f71239b7787b3ac0b11f05ab24933945b4f876e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-23T15:41:09.592 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 podman[71793]: 2024-03-23 15:41:09.566186612 +0000 UTC m=+0.362223474 container start c4dd412540ede56784f888a68f71239b7787b3ac0b11f05ab24933945b4f876e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-23T15:41:09.636 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:09.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 -- 172.21.15.76:0/1253725150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 msgr2=0x7fd19c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:09.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/1253725150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c106ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009900 tx=0x7fd18c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:09.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 -- 172.21.15.76:0/1253725150 shutdown_connections 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/1253725150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd19c107410 0x7fd19c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/1253725150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c106ed0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/1253725150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd19c1021b0 0x7fd19c1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 -- 172.21.15.76:0/1253725150 >> 172.21.15.76:0/1253725150 conn(0x7fd19c0fbba0 msgr2=0x7fd19c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 -- 172.21.15.76:0/1253725150 shutdown_connections 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.759+0000 7fd1a08eb640 1 -- 172.21.15.76:0/1253725150 wait complete. 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 Processor -- start 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 -- start start 2024-03-23T15:41:09.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd19c1021b0 0x7fd19c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd19c107410 0x7fd19c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c19f2b0 con 0x7fd19c104ae0 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c19f420 con 0x7fd19c107410 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd1a08eb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c19f5e0 con 0x7fd19c1021b0 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55202/0 (socket says 172.21.15.76:55202) 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 -- 172.21.15.76:0/3003375537 learned_addr learned my addr 172.21.15.76:0/3003375537 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd19a59c640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd19c1021b0 0x7fd19c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 -- 172.21.15.76:0/3003375537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd19c1021b0 msgr2=0x7fd19c1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:09.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd19c1021b0 0x7fd19c1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 -- 172.21.15.76:0/3003375537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd19c107410 msgr2=0x7fd19c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd19ad9d640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd19c107410 0x7fd19c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd19c107410 0x7fd19c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.760+0000 7fd199d9b640 1 -- 172.21.15.76:0/3003375537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c009560 con 0x7fd19c104ae0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.761+0000 7fd199d9b640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c197d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000c00 tx=0x7fd18c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.761+0000 7fd18b7fe640 1 -- 172.21.15.76:0/3003375537 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c004df0 con 0x7fd19c104ae0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.761+0000 7fd18b7fe640 1 -- 172.21.15.76:0/3003375537 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c005430 con 0x7fd19c104ae0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.761+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c19f860 con 0x7fd19c104ae0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.761+0000 7fd18b7fe640 1 -- 172.21.15.76:0/3003375537 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c03e450 con 0x7fd19c104ae0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.761+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c19fd70 con 0x7fd19c104ae0 2024-03-23T15:41:09.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.762+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c0fd690 con 0x7fd19c104ae0 2024-03-23T15:41:09.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.763+0000 7fd18b7fe640 1 -- 172.21.15.76:0/3003375537 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fd18c003680 con 0x7fd19c104ae0 2024-03-23T15:41:09.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.763+0000 7fd18b7fe640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1840a20c0 0x7fd1840a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:09.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.763+0000 7fd19a59c640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1840a20c0 0x7fd1840a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:09.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.763+0000 7fd18b7fe640 1 -- 172.21.15.76:0/3003375537 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd18c0efd20 con 0x7fd19c104ae0 2024-03-23T15:41:09.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.764+0000 7fd19a59c640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1840a20c0 0x7fd1840a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd190005d00 tx=0x7fd19000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:09.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.768+0000 7fd18b7fe640 1 -- 172.21.15.76:0/3003375537 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd18c0b9190 con 0x7fd19c104ae0 2024-03-23T15:41:09.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.992+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd19c10d2c0 con 0x7fd19c104ae0 2024-03-23T15:41:09.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.993+0000 7fd18b7fe640 1 -- 172.21.15.76:0/3003375537 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fd18c0bc850 con 0x7fd19c104ae0 2024-03-23T15:41:09.996 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:09.996 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-23T15:41:09.996 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 11 2024-03-23T15:41:09.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1840a20c0 msgr2=0x7fd1840a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:09.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1840a20c0 0x7fd1840a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd190005d00 tx=0x7fd19000b410 comp rx=0 tx=0).stop 2024-03-23T15:41:09.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 msgr2=0x7fd19c197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:09.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c197d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000c00 tx=0x7fd18c004270 comp rx=0 tx=0).stop 2024-03-23T15:41:09.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 shutdown_connections 2024-03-23T15:41:09.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1840a20c0 0x7fd1840a4580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd19c107410 0x7fd19c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd19c104ae0 0x7fd19c197d00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 --2- 172.21.15.76:0/3003375537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd19c1021b0 0x7fd19c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:09.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 >> 172.21.15.76:0/3003375537 conn(0x7fd19c0fbba0 msgr2=0x7fd19c102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:09.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 shutdown_connections 2024-03-23T15:41:09.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:09.995+0000 7fd1a08eb640 1 -- 172.21.15.76:0/3003375537 wait complete. 2024-03-23T15:41:10.008 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 ceph-mds[71813]: -- 172.21.15.76:0/1382144714 <== mon.0 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fcff6c41a0 con 0x55fcff69ac00 2024-03-23T15:41:10.008 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a[71809]: starting mds.a at 2024-03-23T15:41:10.008 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 bash[71793]: c4dd412540ede56784f888a68f71239b7787b3ac0b11f05ab24933945b4f876e 2024-03-23T15:41:10.008 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:41:09 smithi076 systemd[1]: Started Ceph mds.a for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:41:10.732 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:10 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3003375537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:10.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:10.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:10.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:10 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3003375537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T15:41:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T15:41:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:10 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3003375537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:11.796 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:11 smithi183 ceph-mon[36296]: Deploying daemon mds.b on smithi183 2024-03-23T15:41:11.797 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:11 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.76:6840/925396691,v1:172.21.15.76:6841/925396691] up:boot 2024-03-23T15:41:11.797 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:11 smithi183 ceph-mon[36296]: fsmap 11 up:standby 2024-03-23T15:41:11.797 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:11 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T15:41:11.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:11 smithi192 ceph-mon[36653]: Deploying daemon mds.b on smithi183 2024-03-23T15:41:11.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:11 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.76:6840/925396691,v1:172.21.15.76:6841/925396691] up:boot 2024-03-23T15:41:11.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:11 smithi192 ceph-mon[36653]: fsmap 11 up:standby 2024-03-23T15:41:11.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:11 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T15:41:11.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:11 smithi076 ceph-mon[32176]: Deploying daemon mds.b on smithi183 2024-03-23T15:41:11.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:11 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.76:6840/925396691,v1:172.21.15.76:6841/925396691] up:boot 2024-03-23T15:41:11.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:11 smithi076 ceph-mon[32176]: fsmap 11 up:standby 2024-03-23T15:41:11.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:11 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T15:41:12.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 -- 172.21.15.76:0/108756363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014075750 msgr2=0x7f3014073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 --2- 172.21.15.76:0/108756363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014075750 0x7f3014073ba0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3010009930 tx=0x7f301002bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 -- 172.21.15.76:0/108756363 shutdown_connections 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 --2- 172.21.15.76:0/108756363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f301410dd00 0x7f30141100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 --2- 172.21.15.76:0/108756363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3014074170 0x7f30140745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 --2- 172.21.15.76:0/108756363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014075750 0x7f3014073ba0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 -- 172.21.15.76:0/108756363 >> 172.21.15.76:0/108756363 conn(0x7f30140fccc0 msgr2=0x7f30140ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 -- 172.21.15.76:0/108756363 shutdown_connections 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.123+0000 7f301b59a640 1 -- 172.21.15.76:0/108756363 wait complete. 2024-03-23T15:41:12.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 Processor -- start 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 -- start start 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014074170 0x7f301419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3014075750 0x7f301419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f301410dd00 0x7f30141a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30141a5900 con 0x7f3014074170 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30141a5a70 con 0x7f3014075750 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f301b59a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f30141a5c30 con 0x7f301410dd00 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014074170 0x7f301419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014074170 0x7f301419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55206/0 (socket says 172.21.15.76:55206) 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 -- 172.21.15.76:0/1586631109 learned_addr learned my addr 172.21.15.76:0/1586631109 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3018b35640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3014075750 0x7f301419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 -- 172.21.15.76:0/1586631109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f301410dd00 msgr2=0x7f30141a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f301410dd00 0x7f30141a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 -- 172.21.15.76:0/1586631109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3014075750 msgr2=0x7f301419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3014075750 0x7f301419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 -- 172.21.15.76:0/1586631109 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3010009560 con 0x7f3014074170 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.124+0000 7f3019336640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014074170 0x7f301419de10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3010005ab0 tx=0x7f30100042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.125+0000 7f300a7fc640 1 -- 172.21.15.76:0/1586631109 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100044c0 con 0x7f3014074170 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.125+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30141a5eb0 con 0x7f3014074170 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.125+0000 7f300a7fc640 1 -- 172.21.15.76:0/1586631109 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30100054e0 con 0x7f3014074170 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.125+0000 7f300a7fc640 1 -- 172.21.15.76:0/1586631109 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301003e450 con 0x7f3014074170 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.125+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30141a62e0 con 0x7f3014074170 2024-03-23T15:41:12.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.126+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3014103ed0 con 0x7f3014074170 2024-03-23T15:41:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.127+0000 7f300a7fc640 1 -- 172.21.15.76:0/1586631109 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f3010003780 con 0x7f3014074170 2024-03-23T15:41:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.127+0000 7f300a7fc640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f30000a20c0 0x7f30000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.128+0000 7f3018b35640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f30000a20c0 0x7f30000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.128+0000 7f300a7fc640 1 -- 172.21.15.76:0/1586631109 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f30100f0700 con 0x7f3014074170 2024-03-23T15:41:12.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.128+0000 7f3018b35640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f30000a20c0 0x7f30000a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f301419f320 tx=0x7f300400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:12.133 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:11 smithi183 systemd[1]: Starting Ceph mds.b for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:41:12.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.131+0000 7f300a7fc640 1 -- 172.21.15.76:0/1586631109 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f30100b9c90 con 0x7f3014074170 2024-03-23T15:41:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.343+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3014113890 con 0x7f3014074170 2024-03-23T15:41:12.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.344+0000 7f300a7fc640 1 -- 172.21.15.76:0/1586631109 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f3010043090 con 0x7f3014074170 2024-03-23T15:41:12.347 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:12.347 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-23T15:41:12.347 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 12 2024-03-23T15:41:12.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f30000a20c0 msgr2=0x7f30000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:12.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f30000a20c0 0x7f30000a4580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f301419f320 tx=0x7f300400a4c0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014074170 msgr2=0x7f301419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:12.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014074170 0x7f301419de10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3010005ab0 tx=0x7f30100042d0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 shutdown_connections 2024-03-23T15:41:12.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f301410dd00 0x7f30141a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f30000a20c0 0x7f30000a4580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3014075750 0x7f301419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 --2- 172.21.15.76:0/1586631109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3014074170 0x7f301419de10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:12.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 >> 172.21.15.76:0/1586631109 conn(0x7f30140fccc0 msgr2=0x7f30141010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:12.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 shutdown_connections 2024-03-23T15:41:12.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:12.346+0000 7f301b59a640 1 -- 172.21.15.76:0/1586631109 wait complete. 2024-03-23T15:41:12.467 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 podman[66992]: 2024-03-23 15:41:12.097028833 +0000 UTC m=+0.017676483 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:41:12.467 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 podman[66992]: 2024-03-23 15:41:12.212755836 +0000 UTC m=+0.133403464 container create cf5580cbf4f26d629fb0d661b5a211b8b9b47175633dd6ff7c99dfca57c0adee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-23T15:41:12.467 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 podman[66992]: 2024-03-23 15:41:12.44050999 +0000 UTC m=+0.361157625 container init cf5580cbf4f26d629fb0d661b5a211b8b9b47175633dd6ff7c99dfca57c0adee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:41:12.468 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 podman[66992]: 2024-03-23 15:41:12.443137924 +0000 UTC m=+0.363785567 container start cf5580cbf4f26d629fb0d661b5a211b8b9b47175633dd6ff7c99dfca57c0adee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-03-23T15:41:12.468 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 ceph-mds[67011]: -- 172.21.15.183:0/61480451 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b6cd381a0 con 0x559b6cd0f000 2024-03-23T15:41:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:12 smithi192 ceph-mon[36653]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:12.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:12 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1586631109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:12 smithi076 ceph-mon[32176]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:12.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:12 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1586631109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:12 smithi183 ceph-mon[36296]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:12 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1586631109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:12.883 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: starting mds.b at 2024-03-23T15:41:12.883 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 bash[66992]: cf5580cbf4f26d629fb0d661b5a211b8b9b47175633dd6ff7c99dfca57c0adee 2024-03-23T15:41:12.883 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:12 smithi183 systemd[1]: Started Ceph mds.b for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:41:13.818 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] up:boot 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: fsmap 12 up:standby 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.819 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:13 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] up:boot 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: fsmap 12 up:standby 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:13 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:41:13.823 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] up:boot 2024-03-23T15:41:13.823 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: fsmap 12 up:standby 2024-03-23T15:41:13.823 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:41:13.823 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.823 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.823 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:13.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:13 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:41:14.361 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.465+0000 7fd440113640 1 -- 172.21.15.76:0/1365585788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438100630 msgr2=0x7fd438100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.465+0000 7fd440113640 1 --2- 172.21.15.76:0/1365585788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438100630 0x7fd438100a50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd434009900 tx=0x7fd43402bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.465+0000 7fd440113640 1 -- 172.21.15.76:0/1365585788 shutdown_connections 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.465+0000 7fd440113640 1 --2- 172.21.15.76:0/1365585788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4381021e0 0x7fd438108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.465+0000 7fd440113640 1 --2- 172.21.15.76:0/1365585788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd438101820 0x7fd438101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.465+0000 7fd440113640 1 --2- 172.21.15.76:0/1365585788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438100630 0x7fd438100a50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.465+0000 7fd440113640 1 -- 172.21.15.76:0/1365585788 >> 172.21.15.76:0/1365585788 conn(0x7fd4380fbba0 msgr2=0x7fd4380fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 -- 172.21.15.76:0/1365585788 shutdown_connections 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 -- 172.21.15.76:0/1365585788 wait complete. 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 Processor -- start 2024-03-23T15:41:14.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 -- start start 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd438100630 0x7fd4380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438101820 0x7fd43819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4381021e0 0x7fd43819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd43819f380 con 0x7fd438101820 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd43819f4f0 con 0x7fd438100630 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd440113640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd43819f6b0 con 0x7fd4381021e0 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd43d6ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438101820 0x7fd43819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd43deaf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd438100630 0x7fd4380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd43deaf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd438100630 0x7fd4380693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:46664/0 (socket says 172.21.15.76:46664) 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd43d6ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438101820 0x7fd43819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36954/0 (socket says 172.21.15.76:36954) 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd43deaf640 1 -- 172.21.15.76:0/565515914 learned_addr learned my addr 172.21.15.76:0/565515914 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.466+0000 7fd43e6b0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4381021e0 0x7fd43819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43deaf640 1 -- 172.21.15.76:0/565515914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4381021e0 msgr2=0x7fd43819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43deaf640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4381021e0 0x7fd43819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43deaf640 1 -- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438101820 msgr2=0x7fd43819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43deaf640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438101820 0x7fd43819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43deaf640 1 -- 172.21.15.76:0/565515914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd434009560 con 0x7fd438100630 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43d6ae640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438101820 0x7fd43819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43e6b0640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4381021e0 0x7fd43819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.467+0000 7fd43deaf640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd438100630 0x7fd4380693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.468+0000 7fd42effd640 1 -- 172.21.15.76:0/565515914 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434004df0 con 0x7fd438100630 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.468+0000 7fd42effd640 1 -- 172.21.15.76:0/565515914 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd434005430 con 0x7fd438100630 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.468+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43819f8d0 con 0x7fd438100630 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.468+0000 7fd42effd640 1 -- 172.21.15.76:0/565515914 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43403e450 con 0x7fd438100630 2024-03-23T15:41:14.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.468+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43819fde0 con 0x7fd438100630 2024-03-23T15:41:14.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.469+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4381079d0 con 0x7fd438100630 2024-03-23T15:41:14.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.469+0000 7fd42effd640 1 -- 172.21.15.76:0/565515914 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fd434004f50 con 0x7fd438100630 2024-03-23T15:41:14.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.470+0000 7fd42effd640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4240a20c0 0x7fd4240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.470+0000 7fd43d6ae640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4240a20c0 0x7fd4240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.470+0000 7fd42effd640 1 -- 172.21.15.76:0/565515914 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd4340efcb0 con 0x7fd438100630 2024-03-23T15:41:14.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.470+0000 7fd43d6ae640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4240a20c0 0x7fd4240a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd438069150 tx=0x7fd42800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:14.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.475+0000 7fd42effd640 1 -- 172.21.15.76:0/565515914 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd4340b9120 con 0x7fd438100630 2024-03-23T15:41:14.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.697+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd43810c240 con 0x7fd438100630 2024-03-23T15:41:14.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.698+0000 7fd42effd640 1 -- 172.21.15.76:0/565515914 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7fd4340bc7e0 con 0x7fd438100630 2024-03-23T15:41:14.701 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:14.702 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24463,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-23T15:41:14.702 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 13 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.700+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4240a20c0 msgr2=0x7fd4240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.700+0000 7fd440113640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4240a20c0 0x7fd4240a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd438069150 tx=0x7fd42800b390 comp rx=0 tx=0).stop 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd438100630 msgr2=0x7fd4380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd438100630 0x7fd4380693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004270 comp rx=0 tx=0).stop 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 shutdown_connections 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4381021e0 0x7fd43819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd438101820 0x7fd43819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4240a20c0 0x7fd4240a4580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 --2- 172.21.15.76:0/565515914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd438100630 0x7fd4380693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 >> 172.21.15.76:0/565515914 conn(0x7fd4380fbba0 msgr2=0x7fd438104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 shutdown_connections 2024-03-23T15:41:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.701+0000 7fd440113640 1 -- 172.21.15.76:0/565515914 wait complete. 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.714 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:14 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:41:14.716 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:41:14.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/2489186363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb01021e0 msgr2=0x7fcfb0108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/2489186363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb01021e0 0x7fcfb0108a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009ca0 tx=0x7fcfa802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/2489186363 shutdown_connections 2024-03-23T15:41:14.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/2489186363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb01021e0 0x7fcfb0108a70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/2489186363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfb0101820 0x7fcfb0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/2489186363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfb0100630 0x7fcfb0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/2489186363 >> 172.21.15.76:0/2489186363 conn(0x7fcfb00fbba0 msgr2=0x7fcfb00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:14.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/2489186363 shutdown_connections 2024-03-23T15:41:14.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.817+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/2489186363 wait complete. 2024-03-23T15:41:14.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 Processor -- start 2024-03-23T15:41:14.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 -- start start 2024-03-23T15:41:14.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb0100630 0x7fcfb01966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfb0101820 0x7fcfb0196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfb01021e0 0x7fcfb019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb019e190 con 0x7fcfb0100630 2024-03-23T15:41:14.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb019e300 con 0x7fcfb0101820 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfb4a9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb019e4c0 con 0x7fcfb01021e0 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb0100630 0x7fcfb01966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb0100630 0x7fcfb01966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36978/0 (socket says 172.21.15.76:36978) 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 -- 172.21.15.76:0/3066603010 learned_addr learned my addr 172.21.15.76:0/3066603010 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfaed9d640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfb01021e0 0x7fcfb019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfadd9b640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfb0101820 0x7fcfb0196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 -- 172.21.15.76:0/3066603010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfb01021e0 msgr2=0x7fcfb019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfb01021e0 0x7fcfb019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 -- 172.21.15.76:0/3066603010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfb0101820 msgr2=0x7fcfb0196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfb0101820 0x7fcfb0196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 -- 172.21.15.76:0/3066603010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa8009900 con 0x7fcfb0100630 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.818+0000 7fcfae59c640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb0100630 0x7fcfb01966a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:14.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.819+0000 7fcf9f7fe640 1 -- 172.21.15.76:0/3066603010 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00096e0 con 0x7fcfb0100630 2024-03-23T15:41:14.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.819+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb019e7a0 con 0x7fcfb0100630 2024-03-23T15:41:14.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.819+0000 7fcf9f7fe640 1 -- 172.21.15.76:0/3066603010 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa000cd80 con 0x7fcfb0100630 2024-03-23T15:41:14.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.819+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb019ecb0 con 0x7fcfb0100630 2024-03-23T15:41:14.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.819+0000 7fcf9f7fe640 1 -- 172.21.15.76:0/3066603010 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0010430 con 0x7fcfb0100630 2024-03-23T15:41:14.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.820+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb0107950 con 0x7fcfb0100630 2024-03-23T15:41:14.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.821+0000 7fcf9f7fe640 1 -- 172.21.15.76:0/3066603010 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fcfa0003680 con 0x7fcfb0100630 2024-03-23T15:41:14.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.822+0000 7fcf9f7fe640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf980a20c0 0x7fcf980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:14.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.822+0000 7fcfadd9b640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf980a20c0 0x7fcf980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:14.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.822+0000 7fcf9f7fe640 1 -- 172.21.15.76:0/3066603010 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcfa0014070 con 0x7fcfb0100630 2024-03-23T15:41:14.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.823+0000 7fcfadd9b640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf980a20c0 0x7fcf980a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0197bb0 tx=0x7fcfa4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:14.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:14.826+0000 7fcf9f7fe640 1 -- 172.21.15.76:0/3066603010 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fcfa0097080 con 0x7fcfb0100630 2024-03-23T15:41:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:14.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:41:14.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:14.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:14 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:41:15.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.004+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfb010c040 con 0x7fcfb0100630 2024-03-23T15:41:15.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.005+0000 7fcf9f7fe640 1 -- 172.21.15.76:0/3066603010 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18440 (secure 0 0 0) 0x7fcfa009a740 con 0x7fcfb0100630 2024-03-23T15:41:15.008 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:15.008 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":81,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:40:29.276449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:41:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf980a20c0 msgr2=0x7fcf980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf980a20c0 0x7fcf980a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0197bb0 tx=0x7fcfa4005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb0100630 msgr2=0x7fcfb01966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb0100630 0x7fcfb01966a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9c0 tx=0x7fcfa000ee80 comp rx=0 tx=0).stop 2024-03-23T15:41:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 shutdown_connections 2024-03-23T15:41:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfb01021e0 0x7fcfb019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcf980a20c0 0x7fcf980a4580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfb0101820 0x7fcfb0196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 --2- 172.21.15.76:0/3066603010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfb0100630 0x7fcfb01966a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 >> 172.21.15.76:0/3066603010 conn(0x7fcfb00fbba0 msgr2=0x7fcfb0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 shutdown_connections 2024-03-23T15:41:15.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.007+0000 7fcfb4a9d640 1 -- 172.21.15.76:0/3066603010 wait complete. 2024-03-23T15:41:15.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:41:15.021 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:41:15.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:15.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:14 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.133+0000 7f1e039c2640 1 -- 172.21.15.76:0/478176062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101c80 msgr2=0x7f1dfc1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.133+0000 7f1e039c2640 1 --2- 172.21.15.76:0/478176062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101c80 0x7f1dfc1020a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009900 tx=0x7f1df802bc80 comp rx=0 tx=0).stop 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.133+0000 7f1e039c2640 1 -- 172.21.15.76:0/478176062 shutdown_connections 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.133+0000 7f1e039c2640 1 --2- 172.21.15.76:0/478176062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dfc101200 0x7f1dfc108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.133+0000 7f1e039c2640 1 --2- 172.21.15.76:0/478176062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc100840 0x7f1dfc100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.133+0000 7f1e039c2640 1 --2- 172.21.15.76:0/478176062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101c80 0x7f1dfc1020a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.134+0000 7f1e039c2640 1 -- 172.21.15.76:0/478176062 >> 172.21.15.76:0/478176062 conn(0x7f1dfc0fb2f0 msgr2=0x7f1dfc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.134+0000 7f1e039c2640 1 -- 172.21.15.76:0/478176062 shutdown_connections 2024-03-23T15:41:15.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.134+0000 7f1e039c2640 1 -- 172.21.15.76:0/478176062 wait complete. 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.134+0000 7f1e039c2640 1 Processor -- start 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.134+0000 7f1e039c2640 1 -- start start 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e039c2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc100840 0x7f1dfc196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e039c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101200 0x7f1dfc196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e039c2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dfc101c80 0x7f1dfc19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e039c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc19e180 con 0x7f1dfc101200 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e039c2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc19e2f0 con 0x7f1dfc100840 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e039c2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc19e4b0 con 0x7f1dfc101c80 2024-03-23T15:41:15.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e00f5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101200 0x7f1dfc196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e0175e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc100840 0x7f1dfc196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e00f5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101200 0x7f1dfc196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37006/0 (socket says 172.21.15.76:37006) 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e00f5d640 1 -- 172.21.15.76:0/2572527767 learned_addr learned my addr 172.21.15.76:0/2572527767 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e01f5f640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dfc101c80 0x7f1dfc19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e0175e640 1 -- 172.21.15.76:0/2572527767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dfc101c80 msgr2=0x7f1dfc19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e0175e640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dfc101c80 0x7f1dfc19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e0175e640 1 -- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101200 msgr2=0x7f1dfc196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e0175e640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101200 0x7f1dfc196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e0175e640 1 -- 172.21.15.76:0/2572527767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df8009560 con 0x7f1dfc100840 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e00f5d640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101200 0x7f1dfc196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.135+0000 7f1e0175e640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc100840 0x7f1dfc196690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.136+0000 7f1df27fc640 1 -- 172.21.15.76:0/2572527767 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8004aa0 con 0x7f1dfc100840 2024-03-23T15:41:15.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.136+0000 7f1df27fc640 1 -- 172.21.15.76:0/2572527767 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df80052d0 con 0x7f1dfc100840 2024-03-23T15:41:15.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.136+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc19e730 con 0x7f1dfc100840 2024-03-23T15:41:15.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.136+0000 7f1df27fc640 1 -- 172.21.15.76:0/2572527767 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8003e80 con 0x7f1dfc100840 2024-03-23T15:41:15.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.136+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc19ec10 con 0x7f1dfc100840 2024-03-23T15:41:15.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.137+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc107a00 con 0x7f1dfc100840 2024-03-23T15:41:15.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.138+0000 7f1df27fc640 1 -- 172.21.15.76:0/2572527767 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f1df8005440 con 0x7f1dfc100840 2024-03-23T15:41:15.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.138+0000 7f1df27fc640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1de80a20c0 0x7f1de80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.138+0000 7f1e00f5d640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1de80a20c0 0x7f1de80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.139+0000 7f1df27fc640 1 -- 172.21.15.76:0/2572527767 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1df80efef0 con 0x7f1dfc100840 2024-03-23T15:41:15.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.139+0000 7f1e00f5d640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1de80a20c0 0x7f1de80a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc197ba0 tx=0x7f1dec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:15.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.142+0000 7f1df27fc640 1 -- 172.21.15.76:0/2572527767 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1df80b92c0 con 0x7f1dfc100840 2024-03-23T15:41:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.337+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1dfc10c180 con 0x7f1dfc100840 2024-03-23T15:41:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.338+0000 7f1df27fc640 1 -- 172.21.15.76:0/2572527767 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f1dfc10c180 con 0x7f1dfc100840 2024-03-23T15:41:15.341 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:41:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1de80a20c0 msgr2=0x7f1de80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1de80a20c0 0x7f1de80a4580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc197ba0 tx=0x7f1dec005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc100840 msgr2=0x7f1dfc196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc100840 0x7f1dfc196690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df8004270 comp rx=0 tx=0).stop 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 shutdown_connections 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dfc101c80 0x7f1dfc19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dfc101200 0x7f1dfc196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1de80a20c0 0x7f1de80a4580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 --2- 172.21.15.76:0/2572527767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc100840 0x7f1dfc196690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 >> 172.21.15.76:0/2572527767 conn(0x7f1dfc0fb2f0 msgr2=0x7f1dfc104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 shutdown_connections 2024-03-23T15:41:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.340+0000 7f1e039c2640 1 -- 172.21.15.76:0/2572527767 wait complete. 2024-03-23T15:41:15.354 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 -- 172.21.15.76:0/1499444296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde081017f0 msgr2=0x7fde08101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 --2- 172.21.15.76:0/1499444296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde081017f0 0x7fde08101c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009900 tx=0x7fddf402bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 -- 172.21.15.76:0/1499444296 shutdown_connections 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 --2- 172.21.15.76:0/1499444296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde081021b0 0x7fde08108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 --2- 172.21.15.76:0/1499444296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde081017f0 0x7fde08101c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 --2- 172.21.15.76:0/1499444296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde08100600 0x7fde08100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 -- 172.21.15.76:0/1499444296 >> 172.21.15.76:0/1499444296 conn(0x7fde080fbb70 msgr2=0x7fde080fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 -- 172.21.15.76:0/1499444296 shutdown_connections 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 -- 172.21.15.76:0/1499444296 wait complete. 2024-03-23T15:41:15.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 Processor -- start 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 -- start start 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde08100600 0x7fde08197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde081017f0 0x7fde08197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.457+0000 7fde0f088640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde081021b0 0x7fde0819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0f088640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde0819f2b0 con 0x7fde08100600 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0f088640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fde0819f420 con 0x7fde081017f0 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0f088640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fde0819f5e0 con 0x7fde081021b0 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde08100600 0x7fde08197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde08100600 0x7fde08197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37024/0 (socket says 172.21.15.76:37024) 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 -- 172.21.15.76:0/3895772128 learned_addr learned my addr 172.21.15.76:0/3895772128 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0d625640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde081021b0 0x7fde0819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde07fff640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde081017f0 0x7fde08197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 -- 172.21.15.76:0/3895772128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde081021b0 msgr2=0x7fde0819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde081021b0 0x7fde0819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 -- 172.21.15.76:0/3895772128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde081017f0 msgr2=0x7fde08197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde081017f0 0x7fde08197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 -- 172.21.15.76:0/3895772128 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf4009560 con 0x7fde08100600 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0ce24640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde08100600 0x7fde08197810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d770 tx=0x7fddfc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde05ffb640 1 -- 172.21.15.76:0/3895772128 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc009c30 con 0x7fde08100600 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde05ffb640 1 -- 172.21.15.76:0/3895772128 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddfc00de80 con 0x7fde08100600 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0819f8c0 con 0x7fde08100600 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.458+0000 7fde05ffb640 1 -- 172.21.15.76:0/3895772128 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc010460 con 0x7fde08100600 2024-03-23T15:41:15.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.459+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0819fdf0 con 0x7fde08100600 2024-03-23T15:41:15.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.459+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde081078c0 con 0x7fde08100600 2024-03-23T15:41:15.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.461+0000 7fde05ffb640 1 -- 172.21.15.76:0/3895772128 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fddfc003f60 con 0x7fde08100600 2024-03-23T15:41:15.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.461+0000 7fde05ffb640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fddf80a2140 0x7fddf80a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.461+0000 7fde07fff640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fddf80a2140 0x7fddf80a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.461+0000 7fde05ffb640 1 -- 172.21.15.76:0/3895772128 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fddfc0cd400 con 0x7fde08100600 2024-03-23T15:41:15.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.462+0000 7fde07fff640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fddf80a2140 0x7fddf80a4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fddf4000c00 tx=0x7fddf402c740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:15.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.465+0000 7fde05ffb640 1 -- 172.21.15.76:0/3895772128 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fddfc0969a0 con 0x7fde08100600 2024-03-23T15:41:15.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.654+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fde0810d160 con 0x7fde08100600 2024-03-23T15:41:15.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.654+0000 7fde05ffb640 1 -- 172.21.15.76:0/3895772128 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fddfc017090 con 0x7fde08100600 2024-03-23T15:41:15.658 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-03-23T15:41:15.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:41:15.658 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-03-23T15:41:15.658 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-03-23T15:41:15.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:41:15.658 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fddf80a2140 msgr2=0x7fddf80a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fddf80a2140 0x7fddf80a4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fddf4000c00 tx=0x7fddf402c740 comp rx=0 tx=0).stop 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde08100600 msgr2=0x7fde08197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde08100600 0x7fde08197810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d770 tx=0x7fddfc00dc30 comp rx=0 tx=0).stop 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 shutdown_connections 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde081021b0 0x7fde0819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fddf80a2140 0x7fddf80a4600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde081017f0 0x7fde08197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 --2- 172.21.15.76:0/3895772128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde08100600 0x7fde08197810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 >> 172.21.15.76:0/3895772128 conn(0x7fde080fbb70 msgr2=0x7fde08104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:15.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 shutdown_connections 2024-03-23T15:41:15.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.657+0000 7fde0f088640 1 -- 172.21.15.76:0/3895772128 wait complete. 2024-03-23T15:41:15.671 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T15:41:15.671 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T15:41:15.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.774+0000 7fabdb53a640 1 -- 172.21.15.76:0/1039824038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd4101800 msgr2=0x7fabd4101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.774+0000 7fabdb53a640 1 --2- 172.21.15.76:0/1039824038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd4101800 0x7fabd4101c80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009900 tx=0x7fabc402bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:15.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.774+0000 7fabdb53a640 1 -- 172.21.15.76:0/1039824038 shutdown_connections 2024-03-23T15:41:15.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.774+0000 7fabdb53a640 1 --2- 172.21.15.76:0/1039824038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd41021c0 0x7fabd4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.774+0000 7fabdb53a640 1 --2- 172.21.15.76:0/1039824038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd4101800 0x7fabd4101c80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.774+0000 7fabdb53a640 1 --2- 172.21.15.76:0/1039824038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabd4100610 0x7fabd4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.774+0000 7fabdb53a640 1 -- 172.21.15.76:0/1039824038 >> 172.21.15.76:0/1039824038 conn(0x7fabd40fbba0 msgr2=0x7fabd40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 -- 172.21.15.76:0/1039824038 shutdown_connections 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 -- 172.21.15.76:0/1039824038 wait complete. 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 Processor -- start 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 -- start start 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabd4100610 0x7fabd4196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd4101800 0x7fabd4196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd41021c0 0x7fabd419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fabd419e210 con 0x7fabd41021c0 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabd419e380 con 0x7fabd4100610 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabdb53a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fabd419e540 con 0x7fabd4101800 2024-03-23T15:41:15.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabd9ad7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd41021c0 0x7fabd419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabd9ad7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd41021c0 0x7fabd419dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37032/0 (socket says 172.21.15.76:37032) 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabd9ad7640 1 -- 172.21.15.76:0/936863986 learned_addr learned my addr 172.21.15.76:0/936863986 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.775+0000 7fabd92d6640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabd4100610 0x7fabd4196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd8ad5640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd4101800 0x7fabd4196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd9ad7640 1 -- 172.21.15.76:0/936863986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd4101800 msgr2=0x7fabd4196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd9ad7640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd4101800 0x7fabd4196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd9ad7640 1 -- 172.21.15.76:0/936863986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabd4100610 msgr2=0x7fabd4196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd9ad7640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabd4100610 0x7fabd4196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd9ad7640 1 -- 172.21.15.76:0/936863986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc4009560 con 0x7fabd41021c0 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd8ad5640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd4101800 0x7fabd4196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabd9ad7640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd41021c0 0x7fabd419dcd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00d950 tx=0x7fabcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabca7fc640 1 -- 172.21.15.76:0/936863986 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc009a30 con 0x7fabd41021c0 2024-03-23T15:41:15.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd419e820 con 0x7fabd41021c0 2024-03-23T15:41:15.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabca7fc640 1 -- 172.21.15.76:0/936863986 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabcc009b90 con 0x7fabd41021c0 2024-03-23T15:41:15.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabca7fc640 1 -- 172.21.15.76:0/936863986 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc003d60 con 0x7fabd41021c0 2024-03-23T15:41:15.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.776+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd419ece0 con 0x7fabd41021c0 2024-03-23T15:41:15.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.777+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd4107a00 con 0x7fabd41021c0 2024-03-23T15:41:15.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.779+0000 7fabca7fc640 1 -- 172.21.15.76:0/936863986 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fabcc009d00 con 0x7fabd41021c0 2024-03-23T15:41:15.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.779+0000 7fabca7fc640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc00a20c0 0x7fabc00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:15.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.779+0000 7fabd92d6640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc00a20c0 0x7fabc00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:15.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.779+0000 7fabca7fc640 1 -- 172.21.15.76:0/936863986 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fabcc0cd5f0 con 0x7fabd41021c0 2024-03-23T15:41:15.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.780+0000 7fabd92d6640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc00a20c0 0x7fabc00a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fabd4101660 tx=0x7fabd0009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:15.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.783+0000 7fabca7fc640 1 -- 172.21.15.76:0/936863986 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fabcc096a60 con 0x7fabd41021c0 2024-03-23T15:41:15.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:15 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/565515914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:15.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:15 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3066603010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:15.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:15 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2572527767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:41:15.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:15 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/565515914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:15 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3066603010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:15 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2572527767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:41:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:15.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:15.971+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fabd410c020 con 0x7fabd41021c0 2024-03-23T15:41:16.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:15 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/565515914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:16.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:15 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3066603010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:16.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:15 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2572527767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:41:16.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.630+0000 7fabca7fc640 1 -- 172.21.15.76:0/936863986 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fabcc09a120 con 0x7fabd41021c0 2024-03-23T15:41:16.634 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-03-23T15:41:16.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc00a20c0 msgr2=0x7fabc00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:16.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc00a20c0 0x7fabc00a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fabd4101660 tx=0x7fabd0009670 comp rx=0 tx=0).stop 2024-03-23T15:41:16.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd41021c0 msgr2=0x7fabd419dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:16.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd41021c0 0x7fabd419dcd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00d950 tx=0x7fabcc00de10 comp rx=0 tx=0).stop 2024-03-23T15:41:16.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 shutdown_connections 2024-03-23T15:41:16.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabd41021c0 0x7fabd419dcd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd4101800 0x7fabd4196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc00a20c0 0x7fabc00a4580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 --2- 172.21.15.76:0/936863986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabd4100610 0x7fabd4196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 >> 172.21.15.76:0/936863986 conn(0x7fabd40fbba0 msgr2=0x7fabd4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:16.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 shutdown_connections 2024-03-23T15:41:16.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.632+0000 7fabdb53a640 1 -- 172.21.15.76:0/936863986 wait complete. 2024-03-23T15:41:16.649 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T15:41:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 -- 172.21.15.76:0/3525645716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8104b30 msgr2=0x7fa7b8106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/3525645716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8104b30 0x7fa7b8106f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009900 tx=0x7fa7ac02bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 -- 172.21.15.76:0/3525645716 shutdown_connections 2024-03-23T15:41:16.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/3525645716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b8107460 0x7fa7b8109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/3525645716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8104b30 0x7fa7b8106f20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/3525645716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b8102200 0x7fa7b81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 -- 172.21.15.76:0/3525645716 >> 172.21.15.76:0/3525645716 conn(0x7fa7b80fbbb0 msgr2=0x7fa7b80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 -- 172.21.15.76:0/3525645716 shutdown_connections 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.755+0000 7fa7bdd25640 1 -- 172.21.15.76:0/3525645716 wait complete. 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 Processor -- start 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 -- start start 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8102200 0x7fa7b8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b8104b30 0x7fa7b8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b8107460 0x7fa7b819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b819f300 con 0x7fa7b8102200 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b819f470 con 0x7fa7b8104b30 2024-03-23T15:41:16.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7bdd25640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b819f630 con 0x7fa7b8107460 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8102200 0x7fa7b8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8102200 0x7fa7b8197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37048/0 (socket says 172.21.15.76:37048) 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7b77fe640 1 -- 172.21.15.76:0/298234023 learned_addr learned my addr 172.21.15.76:0/298234023 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.756+0000 7fa7b7fff640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b8107460 0x7fa7b819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b6ffd640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b8104b30 0x7fa7b8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b77fe640 1 -- 172.21.15.76:0/298234023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b8107460 msgr2=0x7fa7b819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b77fe640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b8107460 0x7fa7b819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b77fe640 1 -- 172.21.15.76:0/298234023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b8104b30 msgr2=0x7fa7b8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b77fe640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b8104b30 0x7fa7b8197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b77fe640 1 -- 172.21.15.76:0/298234023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac009560 con 0x7fa7b8102200 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b77fe640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8102200 0x7fa7b8197810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800d950 tx=0x7fa7a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b4ff9640 1 -- 172.21.15.76:0/298234023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8009c30 con 0x7fa7b8102200 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b819f910 con 0x7fa7b8102200 2024-03-23T15:41:16.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b4ff9640 1 -- 172.21.15.76:0/298234023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a8009d90 con 0x7fa7b8102200 2024-03-23T15:41:16.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7b4ff9640 1 -- 172.21.15.76:0/298234023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800f7e0 con 0x7fa7b8102200 2024-03-23T15:41:16.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.757+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b819fe20 con 0x7fa7b8102200 2024-03-23T15:41:16.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.758+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b80fd6a0 con 0x7fa7b8102200 2024-03-23T15:41:16.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.759+0000 7fa7b4ff9640 1 -- 172.21.15.76:0/298234023 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fa7a8010ad0 con 0x7fa7b8102200 2024-03-23T15:41:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.759+0000 7fa7b4ff9640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7a00a20c0 0x7fa7a00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.760+0000 7fa7b6ffd640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7a00a20c0 0x7fa7a00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.760+0000 7fa7b4ff9640 1 -- 172.21.15.76:0/298234023 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fa7a80cd610 con 0x7fa7b8102200 2024-03-23T15:41:16.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.760+0000 7fa7b6ffd640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7a00a20c0 0x7fa7a00a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac036040 tx=0x7fa7ac005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:16.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.764+0000 7fa7b4ff9640 1 -- 172.21.15.76:0/298234023 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa7a8096910 con 0x7fa7b8102200 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3895772128' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/936863986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:41:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:16.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:16.942+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa7b810d110 con 0x7fa7b8102200 2024-03-23T15:41:17.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:17.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3895772128' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T15:41:17.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/936863986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T15:41:17.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:41:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:17.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3895772128' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-23T15:41:17.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/936863986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:41:17.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:41:17.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.636+0000 7fa7b4ff9640 1 -- 172.21.15.76:0/298234023 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7fa7a8099fd0 con 0x7fa7b8102200 2024-03-23T15:41:17.639 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-03-23T15:41:17.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7a00a20c0 msgr2=0x7fa7a00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7a00a20c0 0x7fa7a00a4580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac036040 tx=0x7fa7ac005fd0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8102200 msgr2=0x7fa7b8197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8102200 0x7fa7b8197810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800d950 tx=0x7fa7a800de10 comp rx=0 tx=0).stop 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 shutdown_connections 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b8107460 0x7fa7b819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa7a00a20c0 0x7fa7a00a4580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b8104b30 0x7fa7b8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 --2- 172.21.15.76:0/298234023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7b8102200 0x7fa7b8197810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 >> 172.21.15.76:0/298234023 conn(0x7fa7b80fbbb0 msgr2=0x7fa7b8102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 shutdown_connections 2024-03-23T15:41:17.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.639+0000 7fa7bdd25640 1 -- 172.21.15.76:0/298234023 wait complete. 2024-03-23T15:41:17.661 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T15:41:17.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 -- 172.21.15.76:0/805136858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b0101820 msgr2=0x7f44b0101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:17.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 --2- 172.21.15.76:0/805136858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b0101820 0x7f44b0101ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009900 tx=0x7f44a402bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 -- 172.21.15.76:0/805136858 shutdown_connections 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 --2- 172.21.15.76:0/805136858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b01021e0 0x7f44b0108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 --2- 172.21.15.76:0/805136858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b0101820 0x7f44b0101ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 --2- 172.21.15.76:0/805136858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b0100630 0x7f44b0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 -- 172.21.15.76:0/805136858 >> 172.21.15.76:0/805136858 conn(0x7f44b00fbbc0 msgr2=0x7f44b00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 -- 172.21.15.76:0/805136858 shutdown_connections 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 -- 172.21.15.76:0/805136858 wait complete. 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 Processor -- start 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.765+0000 7f44b78b5640 1 -- start start 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b0100630 0x7f44b0197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b0101820 0x7f44b0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b01021e0 0x7f44b019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:17.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f44b019f220 con 0x7f44b01021e0 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44b019f390 con 0x7f44b0100630 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f44b019f550 con 0x7f44b0101820 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b01021e0 0x7f44b019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b01021e0 0x7f44b019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37076/0 (socket says 172.21.15.76:37076) 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 -- 172.21.15.76:0/1050068124 learned_addr learned my addr 172.21.15.76:0/1050068124 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b4e50640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b0101820 0x7f44b0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5651640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b0100630 0x7f44b0197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 -- 172.21.15.76:0/1050068124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b0101820 msgr2=0x7f44b0197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b0101820 0x7f44b0197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 -- 172.21.15.76:0/1050068124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b0100630 msgr2=0x7f44b0197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b0100630 0x7f44b0197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 -- 172.21.15.76:0/1050068124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a4009560 con 0x7f44b01021e0 2024-03-23T15:41:17.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b5e52640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b01021e0 0x7f44b019ece0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44a800cce0 tx=0x7f44a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:17.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44a27fc640 1 -- 172.21.15.76:0/1050068124 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a800f450 con 0x7f44b01021e0 2024-03-23T15:41:17.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44a27fc640 1 -- 172.21.15.76:0/1050068124 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a800fa90 con 0x7f44b01021e0 2024-03-23T15:41:17.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b019f830 con 0x7f44b01021e0 2024-03-23T15:41:17.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44a27fc640 1 -- 172.21.15.76:0/1050068124 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8018b90 con 0x7f44b01021e0 2024-03-23T15:41:17.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.766+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b019fd10 con 0x7f44b01021e0 2024-03-23T15:41:17.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.767+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b01078f0 con 0x7f44b01021e0 2024-03-23T15:41:17.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.769+0000 7f44a27fc640 1 -- 172.21.15.76:0/1050068124 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f44a800fc00 con 0x7f44b01021e0 2024-03-23T15:41:17.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.769+0000 7f44a27fc640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f449c0a2140 0x7f449c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:17.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.769+0000 7f44b5651640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f449c0a2140 0x7f449c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:17.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.770+0000 7f44a27fc640 1 -- 172.21.15.76:0/1050068124 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f44a80ce390 con 0x7f44b01021e0 2024-03-23T15:41:17.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.770+0000 7f44b5651640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f449c0a2140 0x7f449c0a4600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005fd0 tx=0x7f44ac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:17.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.773+0000 7f44a27fc640 1 -- 172.21.15.76:0/1050068124 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f44a8097580 con 0x7f44b01021e0 2024-03-23T15:41:17.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:17.988+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f44b010d140 con 0x7f44b01021e0 2024-03-23T15:41:18.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:17 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/936863986' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T15:41:18.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:17 smithi192 ceph-mon[36653]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T15:41:18.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:17 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/298234023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T15:41:18.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:17 smithi192 ceph-mon[36653]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:18.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:17 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/936863986' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T15:41:18.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:17 smithi076 ceph-mon[32176]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T15:41:18.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:17 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/298234023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T15:41:18.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:17 smithi076 ceph-mon[32176]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:18.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:17 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/936863986' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-23T15:41:18.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:17 smithi183 ceph-mon[36296]: osdmap e82: 12 total, 12 up, 12 in 2024-03-23T15:41:18.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:17 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/298234023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-23T15:41:18.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:17 smithi183 ceph-mon[36296]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:41:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.648+0000 7f44a27fc640 1 -- 172.21.15.76:0/1050068124 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f44a809ac40 con 0x7f44b01021e0 2024-03-23T15:41:18.652 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 2 and data pool 3 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f449c0a2140 msgr2=0x7f449c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f449c0a2140 0x7f449c0a4600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005fd0 tx=0x7f44ac005850 comp rx=0 tx=0).stop 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b01021e0 msgr2=0x7f44b019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b01021e0 0x7f44b019ece0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44a800cce0 tx=0x7f44a800a620 comp rx=0 tx=0).stop 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 shutdown_connections 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44b01021e0 0x7f44b019ece0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b0101820 0x7f44b0197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f449c0a2140 0x7f449c0a4600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 --2- 172.21.15.76:0/1050068124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44b0100630 0x7f44b0197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 >> 172.21.15.76:0/1050068124 conn(0x7f44b00fbbc0 msgr2=0x7f44b0104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 shutdown_connections 2024-03-23T15:41:18.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.652+0000 7f44b78b5640 1 -- 172.21.15.76:0/1050068124 wait complete. 2024-03-23T15:41:18.673 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:41:18.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/793342665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b0102920 msgr2=0x7fc6b0102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/793342665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b0102920 0x7fc6b0102da0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc698009900 tx=0x7fc69802bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/793342665 shutdown_connections 2024-03-23T15:41:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/793342665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b01032e0 0x7fc6b0109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/793342665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b0102920 0x7fc6b0102da0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/793342665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0101730 0x7fc6b0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/793342665 >> 172.21.15.76:0/793342665 conn(0x7fc6b00fccc0 msgr2=0x7fc6b00ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:18.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/793342665 shutdown_connections 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.772+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/793342665 wait complete. 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 Processor -- start 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 -- start start 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b0101730 0x7fc6b0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0102920 0x7fc6b0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b01032e0 0x7fc6b019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b019f340 con 0x7fc6b01032e0 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b019f4b0 con 0x7fc6b0101730 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6b4b3e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b019f670 con 0x7fc6b0102920 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b01032e0 0x7fc6b019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b01032e0 0x7fc6b019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37094/0 (socket says 172.21.15.76:37094) 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 -- 172.21.15.76:0/3193696168 learned_addr learned my addr 172.21.15.76:0/3193696168 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6a7fff640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0102920 0x7fc6b0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6ae59c640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b0101730 0x7fc6b0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:18.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 -- 172.21.15.76:0/3193696168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0102920 msgr2=0x7fc6b0197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0102920 0x7fc6b0197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 -- 172.21.15.76:0/3193696168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b0101730 msgr2=0x7fc6b0197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b0101730 0x7fc6b0197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 -- 172.21.15.76:0/3193696168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc698009560 con 0x7fc6b01032e0 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6ae59c640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b0101730 0x7fc6b0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.773+0000 7fc6aed9d640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b01032e0 0x7fc6b019ee00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b970 tx=0x7fc6a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.774+0000 7fc6a77fe640 1 -- 172.21.15.76:0/3193696168 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a800cb40 con 0x7fc6b01032e0 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.774+0000 7fc6a77fe640 1 -- 172.21.15.76:0/3193696168 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a800cca0 con 0x7fc6b01032e0 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.774+0000 7fc6a77fe640 1 -- 172.21.15.76:0/3193696168 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80077d0 con 0x7fc6b01032e0 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.774+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b019f950 con 0x7fc6b01032e0 2024-03-23T15:41:18.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.774+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b019feb0 con 0x7fc6b01032e0 2024-03-23T15:41:18.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.775+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b0108a50 con 0x7fc6b01032e0 2024-03-23T15:41:18.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.776+0000 7fc6a77fe640 1 -- 172.21.15.76:0/3193696168 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fc6a8014440 con 0x7fc6b01032e0 2024-03-23T15:41:18.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.776+0000 7fc6a77fe640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc69c0a2140 0x7fc69c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:18.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.776+0000 7fc6ae59c640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc69c0a2140 0x7fc69c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:18.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.776+0000 7fc6a77fe640 1 -- 172.21.15.76:0/3193696168 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fc6a80cd360 con 0x7fc6b01032e0 2024-03-23T15:41:18.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.777+0000 7fc6ae59c640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc69c0a2140 0x7fc69c0a4600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00058c0 tx=0x7fc6a0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:18.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.781+0000 7fc6a77fe640 1 -- 172.21.15.76:0/3193696168 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc6a8096530 con 0x7fc6b01032e0 2024-03-23T15:41:18.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.955+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6b010d400 con 0x7fc6b01032e0 2024-03-23T15:41:18.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.957+0000 7fc6a77fe640 1 -- 172.21.15.76:0/3193696168 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22216 (secure 0 0 0) 0x7fc6a8099bf0 con 0x7fc6b01032e0 2024-03-23T15:41:18.960 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:41:17.990525+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.958+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc69c0a2140 msgr2=0x7fc69c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.958+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc69c0a2140 0x7fc69c0a4600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00058c0 tx=0x7fc6a0005ec0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.958+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b01032e0 msgr2=0x7fc6b019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.958+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b01032e0 0x7fc6b019ee00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b970 tx=0x7fc6a800be30 comp rx=0 tx=0).stop 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 shutdown_connections 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b01032e0 0x7fc6b019ee00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0102920 0x7fc6b0197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc69c0a2140 0x7fc69c0a4600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 --2- 172.21.15.76:0/3193696168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6b0101730 0x7fc6b0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:18.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 >> 172.21.15.76:0/3193696168 conn(0x7fc6b00fccc0 msgr2=0x7fc6b0105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:18.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 shutdown_connections 2024-03-23T15:41:18.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:18.959+0000 7fc6b4b3e640 1 -- 172.21.15.76:0/3193696168 wait complete. 2024-03-23T15:41:18.972 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:18 smithi076 ceph-mon[32176]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T15:41:18.972 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:18 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/298234023' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T15:41:18.972 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:18 smithi076 ceph-mon[32176]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T15:41:18.972 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:18 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1050068124' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T15:41:18.972 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:18 smithi076 ceph-mon[32176]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T15:41:18.972 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:18 smithi076 ceph-mon[32176]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T15:41:18.974 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:41:19.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:18 smithi192 ceph-mon[36653]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T15:41:19.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:18 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/298234023' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T15:41:19.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:18 smithi192 ceph-mon[36653]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T15:41:19.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:18 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1050068124' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T15:41:19.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:18 smithi192 ceph-mon[36653]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T15:41:19.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:18 smithi192 ceph-mon[36653]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T15:41:19.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 -- 172.21.15.76:0/3090876297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8102950 msgr2=0x7f72a8102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 --2- 172.21.15.76:0/3090876297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8102950 0x7f72a8102dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7290009900 tx=0x7f729002bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:19.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 -- 172.21.15.76:0/3090876297 shutdown_connections 2024-03-23T15:41:19.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 --2- 172.21.15.76:0/3090876297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8103310 0x7f72a8109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 --2- 172.21.15.76:0/3090876297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8102950 0x7f72a8102dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 --2- 172.21.15.76:0/3090876297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a8101760 0x7f72a8101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 -- 172.21.15.76:0/3090876297 >> 172.21.15.76:0/3090876297 conn(0x7f72a80fccb0 msgr2=0x7f72a80ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:19.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.072+0000 7f72adfa1640 1 -- 172.21.15.76:0/3090876297 shutdown_connections 2024-03-23T15:41:19.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 -- 172.21.15.76:0/3090876297 wait complete. 2024-03-23T15:41:19.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 Processor -- start 2024-03-23T15:41:19.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 -- start start 2024-03-23T15:41:19.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101760 0x7f72a80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a8102950 0x7f72a8073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8103310 0x7f72a8074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72a80747a0 con 0x7f72a8103310 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8074910 con 0x7f72a8102950 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.073+0000 7f72adfa1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f72a81a1860 con 0x7f72a8101760 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8103310 0x7f72a8074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8103310 0x7f72a8074040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37110/0 (socket says 172.21.15.76:37110) 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 -- 172.21.15.76:0/4279634793 learned_addr learned my addr 172.21.15.76:0/4279634793 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f729effd640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a8102950 0x7f72a8073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 -- 172.21.15.76:0/4279634793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101760 msgr2=0x7f72a80754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101760 0x7f72a80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 -- 172.21.15.76:0/4279634793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a8102950 msgr2=0x7f72a8073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a8102950 0x7f72a8073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 -- 172.21.15.76:0/4279634793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7290009560 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f729effd640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a8102950 0x7f72a8073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a7fff640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8103310 0x7f72a8074040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72a000db20 tx=0x7f72a000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a57fa640 1 -- 172.21.15.76:0/4279634793 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a000bea0 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a57fa640 1 -- 172.21.15.76:0/4279634793 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a0009940 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a81a1b10 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72a57fa640 1 -- 172.21.15.76:0/4279634793 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00106c0 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.074+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a81a1fd0 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.075+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a8108b00 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.076+0000 7f72a57fa640 1 -- 172.21.15.76:0/4279634793 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f72a0004240 con 0x7f72a8103310 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.077+0000 7f72a57fa640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f72940a20c0 0x7f72940a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.077+0000 7f72a77fe640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f72940a20c0 0x7f72940a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.077+0000 7f72a57fa640 1 -- 172.21.15.76:0/4279634793 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f72a00cd940 con 0x7f72a8103310 2024-03-23T15:41:19.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.077+0000 7f72a77fe640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f72940a20c0 0x7f72940a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7298005fd0 tx=0x7f729800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:19.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.081+0000 7f72a57fa640 1 -- 172.21.15.76:0/4279634793 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f72a0096a60 con 0x7f72a8103310 2024-03-23T15:41:19.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:18 smithi183 ceph-mon[36296]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-23T15:41:19.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:18 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/298234023' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-23T15:41:19.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:18 smithi183 ceph-mon[36296]: osdmap e83: 12 total, 12 up, 12 in 2024-03-23T15:41:19.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:18 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1050068124' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-23T15:41:19.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:18 smithi183 ceph-mon[36296]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-23T15:41:19.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:18 smithi183 ceph-mon[36296]: osdmap e84: 12 total, 12 up, 12 in 2024-03-23T15:41:19.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.261+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72a810d150 con 0x7f72a8103310 2024-03-23T15:41:19.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.263+0000 7f72a57fa640 1 -- 172.21.15.76:0/4279634793 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22216 (secure 0 0 0) 0x7f72a009a120 con 0x7f72a8103310 2024-03-23T15:41:19.266 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:19.266 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:41:17.990525+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:41:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f72940a20c0 msgr2=0x7f72940a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f72940a20c0 0x7f72940a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7298005fd0 tx=0x7f729800b410 comp rx=0 tx=0).stop 2024-03-23T15:41:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8103310 msgr2=0x7f72a8074040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8103310 0x7f72a8074040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f72a000db20 tx=0x7f72a000b260 comp rx=0 tx=0).stop 2024-03-23T15:41:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 shutdown_connections 2024-03-23T15:41:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a8103310 0x7f72a8074040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f72940a20c0 0x7f72940a4580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a8102950 0x7f72a8073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 --2- 172.21.15.76:0/4279634793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101760 0x7f72a80754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 >> 172.21.15.76:0/4279634793 conn(0x7f72a80fccb0 msgr2=0x7f72a8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:19.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 shutdown_connections 2024-03-23T15:41:19.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.265+0000 7f72adfa1640 1 -- 172.21.15.76:0/4279634793 wait complete. 2024-03-23T15:41:19.280 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 -- 172.21.15.76:0/81482189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c101820 msgr2=0x7fad4c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 --2- 172.21.15.76:0/81482189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c101820 0x7fad4c101ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fad34009900 tx=0x7fad3402bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 -- 172.21.15.76:0/81482189 shutdown_connections 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 --2- 172.21.15.76:0/81482189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad4c1021e0 0x7fad4c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 --2- 172.21.15.76:0/81482189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c101820 0x7fad4c101ca0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 --2- 172.21.15.76:0/81482189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad4c100630 0x7fad4c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 -- 172.21.15.76:0/81482189 >> 172.21.15.76:0/81482189 conn(0x7fad4c0fbbc0 msgr2=0x7fad4c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 -- 172.21.15.76:0/81482189 shutdown_connections 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.382+0000 7fad50f77640 1 -- 172.21.15.76:0/81482189 wait complete. 2024-03-23T15:41:19.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 Processor -- start 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 -- start start 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad4c100630 0x7fad4c10aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad4c101820 0x7fad4c109530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c1021e0 0x7fad4c109a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10a1d0 con 0x7fad4c1021e0 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10a340 con 0x7fad4c100630 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad50f77640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c1a17d0 con 0x7fad4c101820 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad4ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c1021e0 0x7fad4c109a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad4ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c1021e0 0x7fad4c109a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37124/0 (socket says 172.21.15.76:37124) 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad4ad9d640 1 -- 172.21.15.76:0/2397554633 learned_addr learned my addr 172.21.15.76:0/2397554633 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad4a59c640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad4c100630 0x7fad4c10aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad41d9b640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad4c101820 0x7fad4c109530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad4ad9d640 1 -- 172.21.15.76:0/2397554633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad4c101820 msgr2=0x7fad4c109530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.383+0000 7fad4ad9d640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad4c101820 0x7fad4c109530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad4ad9d640 1 -- 172.21.15.76:0/2397554633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad4c100630 msgr2=0x7fad4c10aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad4ad9d640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad4c100630 0x7fad4c10aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad4ad9d640 1 -- 172.21.15.76:0/2397554633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad34009560 con 0x7fad4c1021e0 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad4ad9d640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c1021e0 0x7fad4c109a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad4400cd10 tx=0x7fad4400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad43fff640 1 -- 172.21.15.76:0/2397554633 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4400f450 con 0x7fad4c1021e0 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad43fff640 1 -- 172.21.15.76:0/2397554633 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4400fa90 con 0x7fad4c1021e0 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c1a1a80 con 0x7fad4c1021e0 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad43fff640 1 -- 172.21.15.76:0/2397554633 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44018b70 con 0x7fad4c1021e0 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.384+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c1a1f10 con 0x7fad4c1021e0 2024-03-23T15:41:19.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.385+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c107950 con 0x7fad4c1021e0 2024-03-23T15:41:19.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.386+0000 7fad43fff640 1 -- 172.21.15.76:0/2397554633 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fad4400fc00 con 0x7fad4c1021e0 2024-03-23T15:41:19.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.386+0000 7fad43fff640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fad380a20c0 0x7fad380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.386+0000 7fad4a59c640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fad380a20c0 0x7fad380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.386+0000 7fad43fff640 1 -- 172.21.15.76:0/2397554633 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fad440cde70 con 0x7fad4c1021e0 2024-03-23T15:41:19.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.387+0000 7fad4a59c640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fad380a20c0 0x7fad380a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad4c101680 tx=0x7fad3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:19.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.391+0000 7fad43fff640 1 -- 172.21.15.76:0/2397554633 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fad44096f90 con 0x7fad4c1021e0 2024-03-23T15:41:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.602+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fad4c02d190 con 0x7fad4c1021e0 2024-03-23T15:41:19.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.667+0000 7fad43fff640 1 -- 172.21.15.76:0/2397554633 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fad4409a650 con 0x7fad4c1021e0 2024-03-23T15:41:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fad380a20c0 msgr2=0x7fad380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fad380a20c0 0x7fad380a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad4c101680 tx=0x7fad3c00a2d0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c1021e0 msgr2=0x7fad4c109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c1021e0 0x7fad4c109a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad4400cd10 tx=0x7fad4400a620 comp rx=0 tx=0).stop 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 shutdown_connections 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad4c1021e0 0x7fad4c109a70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad4c101820 0x7fad4c109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fad380a20c0 0x7fad380a4580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 --2- 172.21.15.76:0/2397554633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad4c100630 0x7fad4c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 >> 172.21.15.76:0/2397554633 conn(0x7fad4c0fbbc0 msgr2=0x7fad4c104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 shutdown_connections 2024-03-23T15:41:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.669+0000 7fad50f77640 1 -- 172.21.15.76:0/2397554633 wait complete. 2024-03-23T15:41:19.683 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-23T15:41:19.684 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-23T15:41:19.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 -- 172.21.15.76:0/3297839739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 msgr2=0x7f9024101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 --2- 172.21.15.76:0/3297839739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024101ca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9018009900 tx=0x7f901802bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 -- 172.21.15.76:0/3297839739 shutdown_connections 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 --2- 172.21.15.76:0/3297839739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90241021e0 0x7f9024108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 --2- 172.21.15.76:0/3297839739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024101ca0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 --2- 172.21.15.76:0/3297839739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9024100630 0x7f9024100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 -- 172.21.15.76:0/3297839739 >> 172.21.15.76:0/3297839739 conn(0x7f90240fbbc0 msgr2=0x7f90240fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 -- 172.21.15.76:0/3297839739 shutdown_connections 2024-03-23T15:41:19.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.790+0000 7f902907c640 1 -- 172.21.15.76:0/3297839739 wait complete. 2024-03-23T15:41:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 Processor -- start 2024-03-23T15:41:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 -- start start 2024-03-23T15:41:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9024100630 0x7f9024104550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90241021e0 0x7f9024103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9024103830 con 0x7f9024101820 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90241039a0 con 0x7f90241021e0 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902907c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90241a1900 con 0x7f9024100630 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37142/0 (socket says 172.21.15.76:37142) 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.791+0000 7f902259c640 1 -- 172.21.15.76:0/2149732829 learned_addr learned my addr 172.21.15.76:0/2149732829 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902359e640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90241021e0 0x7f9024103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f9022d9d640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9024100630 0x7f9024104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902259c640 1 -- 172.21.15.76:0/2149732829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9024100630 msgr2=0x7f9024104550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902259c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9024100630 0x7f9024104550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902259c640 1 -- 172.21.15.76:0/2149732829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90241021e0 msgr2=0x7f9024103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902259c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90241021e0 0x7f9024103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902259c640 1 -- 172.21.15.76:0/2149732829 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9018009560 con 0x7f9024101820 2024-03-23T15:41:19.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902259c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024102c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9018006010 tx=0x7f90180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:19.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f900bfff640 1 -- 172.21.15.76:0/2149732829 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90180044c0 con 0x7f9024101820 2024-03-23T15:41:19.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f900bfff640 1 -- 172.21.15.76:0/2149732829 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90180052d0 con 0x7f9024101820 2024-03-23T15:41:19.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90241a1b00 con 0x7f9024101820 2024-03-23T15:41:19.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f900bfff640 1 -- 172.21.15.76:0/2149732829 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018003e10 con 0x7f9024101820 2024-03-23T15:41:19.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.792+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90241a2060 con 0x7f9024101820 2024-03-23T15:41:19.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.793+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902410c080 con 0x7f9024101820 2024-03-23T15:41:19.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.794+0000 7f900bfff640 1 -- 172.21.15.76:0/2149732829 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f9018005440 con 0x7f9024101820 2024-03-23T15:41:19.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.795+0000 7f900bfff640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f900c0a20c0 0x7f900c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:19.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.795+0000 7f9022d9d640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f900c0a20c0 0x7f900c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:19.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.795+0000 7f900bfff640 1 -- 172.21.15.76:0/2149732829 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f90180efe60 con 0x7f9024101820 2024-03-23T15:41:19.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.795+0000 7f9022d9d640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f900c0a20c0 0x7f900c0a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9024101680 tx=0x7f9014005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:19.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:19.799+0000 7f900bfff640 1 -- 172.21.15.76:0/2149732829 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f90180b8f80 con 0x7f9024101820 2024-03-23T15:41:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.013+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f90241a2250 con 0x7f9024101820 2024-03-23T15:41:20.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1050068124' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T15:41:20.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T15:41:20.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T15:41:20.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3193696168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:20.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/4279634793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:20.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: pgmap v286: 97 pgs: 7 creating+peering, 36 active+clean, 54 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-03-23T15:41:20.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2397554633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T15:41:20.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:19 smithi192 ceph-mon[36653]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T15:41:20.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1050068124' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T15:41:20.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T15:41:20.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T15:41:20.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3193696168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:20.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/4279634793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:20.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: pgmap v286: 97 pgs: 7 creating+peering, 36 active+clean, 54 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-03-23T15:41:20.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2397554633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T15:41:20.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:19 smithi076 ceph-mon[32176]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1050068124' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3193696168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/4279634793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: pgmap v286: 97 pgs: 7 creating+peering, 36 active+clean, 54 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2397554633' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-23T15:41:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:19 smithi183 ceph-mon[36296]: osdmap e85: 12 total, 12 up, 12 in 2024-03-23T15:41:20.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.669+0000 7f900bfff640 1 -- 172.21.15.76:0/2149732829 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f90180bc640 con 0x7f9024101820 2024-03-23T15:41:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f900c0a20c0 msgr2=0x7f900c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:20.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f900c0a20c0 0x7f900c0a4580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9024101680 tx=0x7f9014005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 msgr2=0x7f9024102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024102c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9018006010 tx=0x7f90180040c0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 shutdown_connections 2024-03-23T15:41:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f900c0a20c0 0x7f900c0a4580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90241021e0 0x7f9024103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9024101820 0x7f9024102c00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 --2- 172.21.15.76:0/2149732829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9024100630 0x7f9024104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 >> 172.21.15.76:0/2149732829 conn(0x7f90240fbbc0 msgr2=0x7f90240fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:20.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 shutdown_connections 2024-03-23T15:41:20.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.671+0000 7f902907c640 1 -- 172.21.15.76:0/2149732829 wait complete. 2024-03-23T15:41:20.689 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-23T15:41:20.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 -- 172.21.15.76:0/4037820284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 msgr2=0x7fed04108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 --2- 172.21.15.76:0/4037820284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed04108a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009ca0 tx=0x7fecf402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 -- 172.21.15.76:0/4037820284 shutdown_connections 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 --2- 172.21.15.76:0/4037820284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed04108a50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 --2- 172.21.15.76:0/4037820284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04101800 0x7fed04101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 --2- 172.21.15.76:0/4037820284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed04100610 0x7fed04100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 -- 172.21.15.76:0/4037820284 >> 172.21.15.76:0/4037820284 conn(0x7fed040fbba0 msgr2=0x7fed040fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 -- 172.21.15.76:0/4037820284 shutdown_connections 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.793+0000 7fed0b95e640 1 -- 172.21.15.76:0/4037820284 wait complete. 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 Processor -- start 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 -- start start 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04100610 0x7fed040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed04101800 0x7fed0419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:20.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed0419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fed0419f340 con 0x7fed041021c0 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed0419f4b0 con 0x7fed04100610 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed0b95e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed0419f670 con 0x7fed04101800 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed09efb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed0419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed09efb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed0419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37172/0 (socket says 172.21.15.76:37172) 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed09efb640 1 -- 172.21.15.76:0/838902905 learned_addr learned my addr 172.21.15.76:0/838902905 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed096fa640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04100610 0x7fed040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed08ef9640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed04101800 0x7fed0419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.794+0000 7fed09efb640 1 -- 172.21.15.76:0/838902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed04101800 msgr2=0x7fed0419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed09efb640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed04101800 0x7fed0419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed09efb640 1 -- 172.21.15.76:0/838902905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04100610 msgr2=0x7fed040693e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed09efb640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04100610 0x7fed040693e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:20.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed09efb640 1 -- 172.21.15.76:0/838902905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf4009900 con 0x7fed041021c0 2024-03-23T15:41:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed09efb640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed0419eb60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecf402c630 tx=0x7fecf4034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed027fc640 1 -- 172.21.15.76:0/838902905 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4032eb0 con 0x7fed041021c0 2024-03-23T15:41:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed027fc640 1 -- 172.21.15.76:0/838902905 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4003e50 con 0x7fed041021c0 2024-03-23T15:41:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0419f890 con 0x7fed041021c0 2024-03-23T15:41:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed027fc640 1 -- 172.21.15.76:0/838902905 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4040450 con 0x7fed041021c0 2024-03-23T15:41:20.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.795+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0419fd40 con 0x7fed041021c0 2024-03-23T15:41:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.796+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed04068500 con 0x7fed041021c0 2024-03-23T15:41:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.797+0000 7fed027fc640 1 -- 172.21.15.76:0/838902905 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fecf400a040 con 0x7fed041021c0 2024-03-23T15:41:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.797+0000 7fed027fc640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecec0a20c0 0x7fecec0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.797+0000 7fed096fa640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecec0a20c0 0x7fecec0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.797+0000 7fed027fc640 1 -- 172.21.15.76:0/838902905 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fecf40f0b30 con 0x7fed041021c0 2024-03-23T15:41:20.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.798+0000 7fed096fa640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecec0a20c0 0x7fecec0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed04101660 tx=0x7fecfc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:20.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:20.803+0000 7fed027fc640 1 -- 172.21.15.76:0/838902905 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fecf40b9c50 con 0x7fed041021c0 2024-03-23T15:41:21.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.015+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fed04107930 con 0x7fed041021c0 2024-03-23T15:41:21.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:20 smithi192 ceph-mon[36653]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T15:41:21.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:20 smithi192 ceph-mon[36653]: Cluster is now healthy 2024-03-23T15:41:21.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:20 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2397554633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T15:41:21.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:20 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] up:active 2024-03-23T15:41:21.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:20 smithi192 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T15:41:21.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:20 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2149732829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-23T15:41:21.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:20 smithi076 ceph-mon[32176]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T15:41:21.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:20 smithi076 ceph-mon[32176]: Cluster is now healthy 2024-03-23T15:41:21.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:20 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2397554633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T15:41:21.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:20 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] up:active 2024-03-23T15:41:21.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:20 smithi076 ceph-mon[32176]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T15:41:21.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:20 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2149732829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-23T15:41:21.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:20 smithi183 ceph-mon[36296]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-23T15:41:21.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:20 smithi183 ceph-mon[36296]: Cluster is now healthy 2024-03-23T15:41:21.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:20 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2397554633' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-23T15:41:21.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:20 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] up:active 2024-03-23T15:41:21.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:20 smithi183 ceph-mon[36296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T15:41:21.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:20 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2149732829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-23T15:41:21.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.681+0000 7fed027fc640 1 -- 172.21.15.76:0/838902905 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fecf4004040 con 0x7fed041021c0 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecec0a20c0 msgr2=0x7fecec0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecec0a20c0 0x7fecec0a4580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed04101660 tx=0x7fecfc009110 comp rx=0 tx=0).stop 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 msgr2=0x7fed0419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed0419eb60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecf402c630 tx=0x7fecf4034cc0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 shutdown_connections 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed041021c0 0x7fed0419eb60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed04101800 0x7fed0419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecec0a20c0 0x7fecec0a4580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 --2- 172.21.15.76:0/838902905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04100610 0x7fed040693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 >> 172.21.15.76:0/838902905 conn(0x7fed040fbba0 msgr2=0x7fed04104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 shutdown_connections 2024-03-23T15:41:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.684+0000 7fed0b95e640 1 -- 172.21.15.76:0/838902905 wait complete. 2024-03-23T15:41:21.702 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-23T15:41:21.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 -- 172.21.15.76:0/1945292212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10340ff9f0 msgr2=0x7f10340ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 --2- 172.21.15.76:0/1945292212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10340ff9f0 0x7f10340ffe10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1030009900 tx=0x7f103002bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 -- 172.21.15.76:0/1945292212 shutdown_connections 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 --2- 172.21.15.76:0/1945292212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1034103380 0x7f1034109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 --2- 172.21.15.76:0/1945292212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10341029c0 0x7f1034102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 --2- 172.21.15.76:0/1945292212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10340ff9f0 0x7f10340ffe10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 -- 172.21.15.76:0/1945292212 >> 172.21.15.76:0/1945292212 conn(0x7f10340fb320 msgr2=0x7f10340fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 -- 172.21.15.76:0/1945292212 shutdown_connections 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.809+0000 7f103b93e640 1 -- 172.21.15.76:0/1945292212 wait complete. 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 Processor -- start 2024-03-23T15:41:21.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 -- start start 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10340ff9f0 0x7f10341977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10341029c0 0x7f1034197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1034103380 0x7f103419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f103419f2b0 con 0x7f10341029c0 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f103419f420 con 0x7f1034103380 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f103419f5e0 con 0x7f10340ff9f0 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10341029c0 0x7f1034197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10341029c0 0x7f1034197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37194/0 (socket says 172.21.15.76:37194) 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 -- 172.21.15.76:0/644439095 learned_addr learned my addr 172.21.15.76:0/644439095 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1039edb640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1034103380 0x7f103419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f10396da640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10340ff9f0 0x7f10341977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 -- 172.21.15.76:0/644439095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10340ff9f0 msgr2=0x7f10341977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10340ff9f0 0x7f10341977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 -- 172.21.15.76:0/644439095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1034103380 msgr2=0x7f103419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:21.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1034103380 0x7f103419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 -- 172.21.15.76:0/644439095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1030009560 con 0x7f10341029c0 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1039edb640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1034103380 0x7f103419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f1038ed9640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10341029c0 0x7f1034197d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9f0 tx=0x7f102400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f102a7fc640 1 -- 172.21.15.76:0/644439095 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024009700 con 0x7f10341029c0 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103419f8c0 con 0x7f10341029c0 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f102a7fc640 1 -- 172.21.15.76:0/644439095 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102400cd80 con 0x7f10341029c0 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.810+0000 7f102a7fc640 1 -- 172.21.15.76:0/644439095 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024018920 con 0x7f10341029c0 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.811+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103419fdd0 con 0x7f10341029c0 2024-03-23T15:41:21.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.811+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1034108b40 con 0x7f10341029c0 2024-03-23T15:41:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.813+0000 7f102a7fc640 1 -- 172.21.15.76:0/644439095 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f1024018a80 con 0x7f10341029c0 2024-03-23T15:41:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.813+0000 7f102a7fc640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10200a20c0 0x7f10200a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.813+0000 7f102a7fc640 1 -- 172.21.15.76:0/644439095 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1024014070 con 0x7f10341029c0 2024-03-23T15:41:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.813+0000 7f10396da640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10200a20c0 0x7f10200a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.813+0000 7f10396da640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10200a20c0 0x7f10200a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1030005ab0 tx=0x7f1030005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:21.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:21.817+0000 7f102a7fc640 1 -- 172.21.15.76:0/644439095 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1024097530 con 0x7f10341029c0 2024-03-23T15:41:22.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.029+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f103410d390 con 0x7f10341029c0 2024-03-23T15:41:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T15:41:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2149732829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-23T15:41:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T15:41:22.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T15:41:22.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T15:41:22.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T15:41:22.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/838902905' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T15:41:22.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:21 smithi192 ceph-mon[36653]: pgmap v288: 97 pgs: 7 creating+peering, 78 active+clean, 12 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-03-23T15:41:22.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T15:41:22.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2149732829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-23T15:41:22.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T15:41:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T15:41:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T15:41:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T15:41:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/838902905' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T15:41:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:21 smithi076 ceph-mon[32176]: pgmap v288: 97 pgs: 7 creating+peering, 78 active+clean, 12 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2149732829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/838902905' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T15:41:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:21 smithi183 ceph-mon[36296]: pgmap v288: 97 pgs: 7 creating+peering, 78 active+clean, 12 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-03-23T15:41:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.690+0000 7f102a7fc640 1 -- 172.21.15.76:0/644439095 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f102409abf0 con 0x7f10341029c0 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.692+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10200a20c0 msgr2=0x7f10200a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.692+0000 7f103b93e640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10200a20c0 0x7f10200a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1030005ab0 tx=0x7f1030005fd0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.692+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10341029c0 msgr2=0x7f1034197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.692+0000 7f103b93e640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10341029c0 0x7f1034197d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9f0 tx=0x7f102400eeb0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.692+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 shutdown_connections 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.692+0000 7f103b93e640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10200a20c0 0x7f10200a4580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.692+0000 7f103b93e640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1034103380 0x7f103419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.693+0000 7f103b93e640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10341029c0 0x7f1034197d00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.693+0000 7f103b93e640 1 --2- 172.21.15.76:0/644439095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10340ff9f0 0x7f10341977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.693+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 >> 172.21.15.76:0/644439095 conn(0x7f10340fb320 msgr2=0x7f1034105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.693+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 shutdown_connections 2024-03-23T15:41:22.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.693+0000 7f103b93e640 1 -- 172.21.15.76:0/644439095 wait complete. 2024-03-23T15:41:22.707 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-23T15:41:22.708 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-23T15:41:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 -- 172.21.15.76:0/235092545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 msgr2=0x7f4ab00ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 --2- 172.21.15.76:0/235092545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab00ffe50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009900 tx=0x7f4aa002bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 -- 172.21.15.76:0/235092545 shutdown_connections 2024-03-23T15:41:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 --2- 172.21.15.76:0/235092545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0103460 0x7f4ab0109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 --2- 172.21.15.76:0/235092545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ab0102aa0 0x7f4ab0102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 --2- 172.21.15.76:0/235092545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab00ffe50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 -- 172.21.15.76:0/235092545 >> 172.21.15.76:0/235092545 conn(0x7f4ab00fb320 msgr2=0x7f4ab00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 -- 172.21.15.76:0/235092545 shutdown_connections 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 -- 172.21.15.76:0/235092545 wait complete. 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 Processor -- start 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.812+0000 7f4ab826d640 1 -- start start 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab0197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0102aa0 0x7f4ab0197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ab0103460 0x7f4ab019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab019f320 con 0x7f4ab00ffa30 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab019f490 con 0x7f4ab0102aa0 2024-03-23T15:41:22.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab019f650 con 0x7f4ab0103460 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab0197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab0197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37216/0 (socket says 172.21.15.76:37216) 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 -- 172.21.15.76:0/3552629719 learned_addr learned my addr 172.21.15.76:0/3552629719 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab5808640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0102aa0 0x7f4ab0197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab680a640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ab0103460 0x7f4ab019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 -- 172.21.15.76:0/3552629719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ab0103460 msgr2=0x7f4ab019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ab0103460 0x7f4ab019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 -- 172.21.15.76:0/3552629719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0102aa0 msgr2=0x7f4ab0197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0102aa0 0x7f4ab0197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 -- 172.21.15.76:0/3552629719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa0009560 con 0x7f4ab00ffa30 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab680a640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ab0103460 0x7f4ab019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:41:22.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab6009640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab0197830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0005ab0 tx=0x7f4aa0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:22.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4aa77fe640 1 -- 172.21.15.76:0/3552629719 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa00043d0 con 0x7f4ab00ffa30 2024-03-23T15:41:22.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4aa77fe640 1 -- 172.21.15.76:0/3552629719 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa00052d0 con 0x7f4ab00ffa30 2024-03-23T15:41:22.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab019f8d0 con 0x7f4ab00ffa30 2024-03-23T15:41:22.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4aa77fe640 1 -- 172.21.15.76:0/3552629719 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0003d40 con 0x7f4ab00ffa30 2024-03-23T15:41:22.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.813+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab019fde0 con 0x7f4ab00ffa30 2024-03-23T15:41:22.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.814+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab0108bd0 con 0x7f4ab00ffa30 2024-03-23T15:41:22.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.815+0000 7f4aa77fe640 1 -- 172.21.15.76:0/3552629719 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f4aa0005440 con 0x7f4ab00ffa30 2024-03-23T15:41:22.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.816+0000 7f4aa77fe640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4a9c0a20c0 0x7f4a9c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:22.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.816+0000 7f4ab5808640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4a9c0a20c0 0x7f4a9c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:22.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.816+0000 7f4aa77fe640 1 -- 172.21.15.76:0/3552629719 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4aa00efd20 con 0x7f4ab00ffa30 2024-03-23T15:41:22.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.816+0000 7f4ab5808640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4a9c0a20c0 0x7f4a9c0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0198d40 tx=0x7f4aac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:22.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:22.820+0000 7f4aa77fe640 1 -- 172.21.15.76:0/3552629719 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4aa00b8e40 con 0x7f4ab00ffa30 2024-03-23T15:41:22.883 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: 2024-03-23T15:41:22.552+0000 7f6529a47700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T15:41:22.883 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: 2024-03-23T15:41:22.552+0000 7f6529a47700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-23T15:41:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/838902905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T15:41:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:active 2024-03-23T15:41:22.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T15:41:22.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T15:41:22.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: Cluster is now healthy 2024-03-23T15:41:22.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-23T15:41:22.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-23T15:41:22.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-23T15:41:22.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:22 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/644439095' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T15:41:23.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.021+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4ab002d1d0 con 0x7f4a9c0a20c0 2024-03-23T15:41:23.028 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/838902905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T15:41:23.028 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:active 2024-03-23T15:41:23.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T15:41:23.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T15:41:23.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: Cluster is now healthy 2024-03-23T15:41:23.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-23T15:41:23.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-23T15:41:23.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-23T15:41:23.029 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:22 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/644439095' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T15:41:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.043+0000 7f4aa77fe640 1 -- 172.21.15.76:0/3552629719 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ab002d1d0 con 0x7f4a9c0a20c0 2024-03-23T15:41:23.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4a9c0a20c0 msgr2=0x7f4a9c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4a9c0a20c0 0x7f4a9c0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0198d40 tx=0x7f4aac005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:23.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 msgr2=0x7f4ab0197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab0197830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0005ab0 tx=0x7f4aa0005e50 comp rx=0 tx=0).stop 2024-03-23T15:41:23.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 shutdown_connections 2024-03-23T15:41:23.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ab0103460 0x7f4ab019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f4a9c0a20c0 0x7f4a9c0a4580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0102aa0 0x7f4ab0197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 --2- 172.21.15.76:0/3552629719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ab00ffa30 0x7f4ab0197830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 >> 172.21.15.76:0/3552629719 conn(0x7f4ab00fb320 msgr2=0x7f4ab0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:23.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 shutdown_connections 2024-03-23T15:41:23.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.046+0000 7f4ab826d640 1 -- 172.21.15.76:0/3552629719 wait complete. 2024-03-23T15:41:23.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-23T15:41:23.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/838902905' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T15:41:23.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:active 2024-03-23T15:41:23.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-23T15:41:23.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T15:41:23.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: Cluster is now healthy 2024-03-23T15:41:23.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-23T15:41:23.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-23T15:41:23.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-23T15:41:23.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:22 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/644439095' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-23T15:41:23.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 -- 172.21.15.76:0/1416646878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db0102910 msgr2=0x7f7db0102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 --2- 172.21.15.76:0/1416646878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db0102910 0x7f7db0102d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009900 tx=0x7f7da402bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:23.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 -- 172.21.15.76:0/1416646878 shutdown_connections 2024-03-23T15:41:23.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 --2- 172.21.15.76:0/1416646878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7db01032d0 0x7f7db0109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 --2- 172.21.15.76:0/1416646878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db0102910 0x7f7db0102d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 --2- 172.21.15.76:0/1416646878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db0101720 0x7f7db0101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 -- 172.21.15.76:0/1416646878 >> 172.21.15.76:0/1416646878 conn(0x7f7db00fccb0 msgr2=0x7f7db00ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:23.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 -- 172.21.15.76:0/1416646878 shutdown_connections 2024-03-23T15:41:23.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 -- 172.21.15.76:0/1416646878 wait complete. 2024-03-23T15:41:23.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.164+0000 7f7db5215640 1 Processor -- start 2024-03-23T15:41:23.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 -- start start 2024-03-23T15:41:23.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db0101720 0x7f7db0197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7db0102910 0x7f7db0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db01032d0 0x7f7db019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7db019f2f0 con 0x7f7db01032d0 2024-03-23T15:41:23.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7db019f460 con 0x7f7db0102910 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7db019f620 con 0x7f7db0101720 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db01032d0 0x7f7db019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db01032d0 0x7f7db019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37254/0 (socket says 172.21.15.76:37254) 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 -- 172.21.15.76:0/3051492579 learned_addr learned my addr 172.21.15.76:0/3051492579 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7dae59c640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7db0102910 0x7f7db0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daed9d640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db0101720 0x7f7db0197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 -- 172.21.15.76:0/3051492579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db0101720 msgr2=0x7f7db0197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db0101720 0x7f7db0197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 -- 172.21.15.76:0/3051492579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7db0102910 msgr2=0x7f7db0197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7db0102910 0x7f7db0197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 -- 172.21.15.76:0/3051492579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da4009560 con 0x7f7db01032d0 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7daf59e640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db01032d0 0x7f7db019edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009fd0 tx=0x7f7da800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:23.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7d97fff640 1 -- 172.21.15.76:0/3051492579 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da800bea0 con 0x7f7db01032d0 2024-03-23T15:41:23.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7d97fff640 1 -- 172.21.15.76:0/3051492579 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da8009940 con 0x7f7db01032d0 2024-03-23T15:41:23.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.165+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db019f900 con 0x7f7db01032d0 2024-03-23T15:41:23.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.166+0000 7f7d97fff640 1 -- 172.21.15.76:0/3051492579 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80106c0 con 0x7f7db01032d0 2024-03-23T15:41:23.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.166+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db019fd90 con 0x7f7db01032d0 2024-03-23T15:41:23.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.166+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db0108b60 con 0x7f7db01032d0 2024-03-23T15:41:23.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.167+0000 7f7d97fff640 1 -- 172.21.15.76:0/3051492579 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f7da8004240 con 0x7f7db01032d0 2024-03-23T15:41:23.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.168+0000 7f7d97fff640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7d980a20c0 0x7f7d980a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.168+0000 7f7daed9d640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7d980a20c0 0x7f7d980a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.168+0000 7f7d97fff640 1 -- 172.21.15.76:0/3051492579 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7da80cd7c0 con 0x7f7db01032d0 2024-03-23T15:41:23.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.169+0000 7f7daed9d640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7d980a20c0 0x7f7d980a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7db0102770 tx=0x7f7da0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:23.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.173+0000 7f7d97fff640 1 -- 172.21.15.76:0/3051492579 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7da8096990 con 0x7f7db01032d0 2024-03-23T15:41:23.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:41:23.025+0000 7f6240cae700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:41:23.326 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:41:23.025+0000 7f6240cae700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:41:23.326 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:41:23.025+0000 7f6240cae700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:41:23.326 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:41:23.025+0000 7f6240cae700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:41:23.326 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:41:23.025+0000 7f6240cae700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:41:23.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.374+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7db002d1d0 con 0x7f7d980a20c0 2024-03-23T15:41:23.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.399+0000 7f7d97fff640 1 -- 172.21.15.76:0/3051492579 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7db002d1d0 con 0x7f7d980a20c0 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7d980a20c0 msgr2=0x7f7d980a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7d980a20c0 0x7f7d980a4580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7db0102770 tx=0x7f7da0005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db01032d0 msgr2=0x7f7db019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db01032d0 0x7f7db019edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009fd0 tx=0x7f7da800b260 comp rx=0 tx=0).stop 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 shutdown_connections 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db01032d0 0x7f7db019edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7d980a20c0 0x7f7d980a4580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7db0102910 0x7f7db0197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 --2- 172.21.15.76:0/3051492579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db0101720 0x7f7db0197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 >> 172.21.15.76:0/3051492579 conn(0x7f7db00fccb0 msgr2=0x7f7db0105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:23.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 shutdown_connections 2024-03-23T15:41:23.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.401+0000 7f7db5215640 1 -- 172.21.15.76:0/3051492579 wait complete. 2024-03-23T15:41:23.414 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 -- 172.21.15.76:0/1946282685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc100630 msgr2=0x7f29dc100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 --2- 172.21.15.76:0/1946282685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc100630 0x7f29dc100a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009900 tx=0x7f29d002bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 -- 172.21.15.76:0/1946282685 shutdown_connections 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 --2- 172.21.15.76:0/1946282685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29dc1021e0 0x7f29dc108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 --2- 172.21.15.76:0/1946282685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29dc101820 0x7f29dc101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 --2- 172.21.15.76:0/1946282685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc100630 0x7f29dc100a50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 -- 172.21.15.76:0/1946282685 >> 172.21.15.76:0/1946282685 conn(0x7f29dc0fbbc0 msgr2=0x7f29dc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 -- 172.21.15.76:0/1946282685 shutdown_connections 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.519+0000 7f29e2be2640 1 -- 172.21.15.76:0/1946282685 wait complete. 2024-03-23T15:41:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 Processor -- start 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 -- start start 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29dc100630 0x7f29dc197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc101820 0x7f29dc197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29dc1021e0 0x7f29dc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc19f2d0 con 0x7f29dc101820 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc19f440 con 0x7f29dc1021e0 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e2be2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc19f600 con 0x7f29dc100630 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc101820 0x7f29dc197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc101820 0x7f29dc197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37266/0 (socket says 172.21.15.76:37266) 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 -- 172.21.15.76:0/2132026306 learned_addr learned my addr 172.21.15.76:0/2132026306 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e097e640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29dc100630 0x7f29dc197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29e117f640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29dc1021e0 0x7f29dc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 -- 172.21.15.76:0/2132026306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29dc100630 msgr2=0x7f29dc197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29dc100630 0x7f29dc197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 -- 172.21.15.76:0/2132026306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29dc1021e0 msgr2=0x7f29dc19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29dc1021e0 0x7f29dc19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 -- 172.21.15.76:0/2132026306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d0009560 con 0x7f29dc101820 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.520+0000 7f29dbfff640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc101820 0x7f29dc197d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9f0 tx=0x7f29c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.521+0000 7f29d9ffb640 1 -- 172.21.15.76:0/2132026306 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8009720 con 0x7f29dc101820 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.521+0000 7f29d9ffb640 1 -- 172.21.15.76:0/2132026306 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c800cd80 con 0x7f29dc101820 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.521+0000 7f29d9ffb640 1 -- 172.21.15.76:0/2132026306 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8010690 con 0x7f29dc101820 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.521+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc19f8e0 con 0x7f29dc101820 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.521+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc19fda0 con 0x7f29dc101820 2024-03-23T15:41:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.522+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc10c040 con 0x7f29dc101820 2024-03-23T15:41:23.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.523+0000 7f29d9ffb640 1 -- 172.21.15.76:0/2132026306 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f29c80046c0 con 0x7f29dc101820 2024-03-23T15:41:23.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.523+0000 7f29d9ffb640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f29cc0a20c0 0x7f29cc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.523+0000 7f29e097e640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f29cc0a20c0 0x7f29cc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.523+0000 7f29d9ffb640 1 -- 172.21.15.76:0/2132026306 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f29c8014070 con 0x7f29dc101820 2024-03-23T15:41:23.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.524+0000 7f29e097e640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f29cc0a20c0 0x7f29cc0a4580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005ab0 tx=0x7f29d0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:23.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.528+0000 7f29d9ffb640 1 -- 172.21.15.76:0/2132026306 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f29c80d2070 con 0x7f29dc101820 2024-03-23T15:41:23.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.736+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f29dc06a3d0 con 0x7f29cc0a20c0 2024-03-23T15:41:23.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.741+0000 7f29d9ffb640 1 -- 172.21.15.76:0/2132026306 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f29dc06a3d0 con 0x7f29cc0a20c0 2024-03-23T15:41:23.743 INFO:teuthology.orchestra.run.smithi076.stdout:/volumes/qa/sv_0/fb6357bf-5d2d-407b-aab3-4303adad8a7d 2024-03-23T15:41:23.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f29cc0a20c0 msgr2=0x7f29cc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f29cc0a20c0 0x7f29cc0a4580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005ab0 tx=0x7f29d0005fd0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc101820 msgr2=0x7f29dc197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc101820 0x7f29dc197d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9f0 tx=0x7f29c800edf0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 shutdown_connections 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f29cc0a20c0 0x7f29cc0a4580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29dc1021e0 0x7f29dc19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29dc101820 0x7f29dc197d70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.743+0000 7f29e2be2640 1 --2- 172.21.15.76:0/2132026306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29dc100630 0x7f29dc197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.744+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 >> 172.21.15.76:0/2132026306 conn(0x7f29dc0fbbc0 msgr2=0x7f29dc069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:23.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.744+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 shutdown_connections 2024-03-23T15:41:23.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.744+0000 7f29e2be2640 1 -- 172.21.15.76:0/2132026306 wait complete. 2024-03-23T15:41:23.757 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-23T15:41:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 -- 172.21.15.76:0/1991225388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c81021c0 msgr2=0x7f50c8108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 --2- 172.21.15.76:0/1991225388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c81021c0 0x7f50c8108a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009900 tx=0x7f50c402bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 -- 172.21.15.76:0/1991225388 shutdown_connections 2024-03-23T15:41:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 --2- 172.21.15.76:0/1991225388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c81021c0 0x7f50c8108a50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 --2- 172.21.15.76:0/1991225388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c8101800 0x7f50c8101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 --2- 172.21.15.76:0/1991225388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c8100610 0x7f50c8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 -- 172.21.15.76:0/1991225388 >> 172.21.15.76:0/1991225388 conn(0x7f50c80fbba0 msgr2=0x7f50c80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 -- 172.21.15.76:0/1991225388 shutdown_connections 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.863+0000 7f50cf807640 1 -- 172.21.15.76:0/1991225388 wait complete. 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 Processor -- start 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 -- start start 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c8100610 0x7f50c8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c8101800 0x7f50c8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c81021c0 0x7f50c819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50c819f270 con 0x7f50c8100610 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50c819f3e0 con 0x7f50c81021c0 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cf807640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f50c819f5a0 con 0x7f50c8101800 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c8100610 0x7f50c8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c8100610 0x7f50c8197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37276/0 (socket says 172.21.15.76:37276) 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 -- 172.21.15.76:0/610305279 learned_addr learned my addr 172.21.15.76:0/610305279 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50ccda2640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c8101800 0x7f50c8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cdda4640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c81021c0 0x7f50c819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 -- 172.21.15.76:0/610305279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c8101800 msgr2=0x7f50c8197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c8101800 0x7f50c8197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 -- 172.21.15.76:0/610305279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c81021c0 msgr2=0x7f50c819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c81021c0 0x7f50c819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cd5a3640 1 -- 172.21.15.76:0/610305279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c4009560 con 0x7f50c8100610 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50ccda2640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c8101800 0x7f50c8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.864+0000 7f50cdda4640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c81021c0 0x7f50c819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.865+0000 7f50cd5a3640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c8100610 0x7f50c8197780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ebc0 tx=0x7f50c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.865+0000 7f50be7fc640 1 -- 172.21.15.76:0/610305279 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0003b70 con 0x7f50c8100610 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.865+0000 7f50be7fc640 1 -- 172.21.15.76:0/610305279 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c000eec0 con 0x7f50c8100610 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.865+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c819f880 con 0x7f50c8100610 2024-03-23T15:41:23.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.865+0000 7f50be7fc640 1 -- 172.21.15.76:0/610305279 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0019800 con 0x7f50c8100610 2024-03-23T15:41:23.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.865+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c819fd90 con 0x7f50c8100610 2024-03-23T15:41:23.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.866+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c810c020 con 0x7f50c8100610 2024-03-23T15:41:23.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.867+0000 7f50be7fc640 1 -- 172.21.15.76:0/610305279 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f50c0019960 con 0x7f50c8100610 2024-03-23T15:41:23.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.867+0000 7f50be7fc640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50b40a20c0 0x7f50b40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.867+0000 7f50ccda2640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50b40a20c0 0x7f50b40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.867+0000 7f50be7fc640 1 -- 172.21.15.76:0/610305279 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f50c0014070 con 0x7f50c8100610 2024-03-23T15:41:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.868+0000 7f50ccda2640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50b40a20c0 0x7f50b40a4580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f50c8198c90 tx=0x7f50b800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:23.872+0000 7f50be7fc640 1 -- 172.21.15.76:0/610305279 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f50c0097690 con 0x7f50c8100610 2024-03-23T15:41:24.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:23 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/644439095' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T15:41:24.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:23 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:active 2024-03-23T15:41:24.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:23 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-23T15:41:24.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:23 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:41:24.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:23 smithi192 ceph-mon[36653]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:24.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:23 smithi192 ceph-mon[36653]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:24.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:23 smithi192 ceph-mon[36653]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.7 KiB/s wr, 12 op/s 2024-03-23T15:41:24.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.070+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f50c80fdd20 con 0x7f50b40a20c0 2024-03-23T15:41:24.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/644439095' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T15:41:24.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:active 2024-03-23T15:41:24.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-23T15:41:24.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:41:24.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-mon[32176]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:24.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-mon[32176]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:24.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:23 smithi076 ceph-mon[32176]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.7 KiB/s wr, 12 op/s 2024-03-23T15:41:24.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.096+0000 7f50be7fc640 1 -- 172.21.15.76:0/610305279 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50c80fdd20 con 0x7f50b40a20c0 2024-03-23T15:41:24.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50b40a20c0 msgr2=0x7f50b40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50b40a20c0 0x7f50b40a4580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f50c8198c90 tx=0x7f50b800b5f0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c8100610 msgr2=0x7f50c8197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c8100610 0x7f50c8197780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ebc0 tx=0x7f50c000c6a0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 shutdown_connections 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50b40a20c0 0x7f50b40a4580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c81021c0 0x7f50c819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50c8101800 0x7f50c8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 --2- 172.21.15.76:0/610305279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50c8100610 0x7f50c8197780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 >> 172.21.15.76:0/610305279 conn(0x7f50c80fbba0 msgr2=0x7f50c80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 shutdown_connections 2024-03-23T15:41:24.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.098+0000 7f50cf807640 1 -- 172.21.15.76:0/610305279 wait complete. 2024-03-23T15:41:24.111 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-23T15:41:24.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:23 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/644439095' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-23T15:41:24.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:23 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:active 2024-03-23T15:41:24.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:23 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-23T15:41:24.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:23 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:41:24.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:23 smithi183 ceph-mon[36296]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:24.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:23 smithi183 ceph-mon[36296]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:24.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:23 smithi183 ceph-mon[36296]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.7 KiB/s wr, 12 op/s 2024-03-23T15:41:24.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 -- 172.21.15.76:0/1807946546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c101810 msgr2=0x7f801c101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 --2- 172.21.15.76:0/1807946546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c101810 0x7f801c101c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f800c009900 tx=0x7f800c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 -- 172.21.15.76:0/1807946546 shutdown_connections 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 --2- 172.21.15.76:0/1807946546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c1021d0 0x7f801c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 --2- 172.21.15.76:0/1807946546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c101810 0x7f801c101c90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 --2- 172.21.15.76:0/1807946546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c100620 0x7f801c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 -- 172.21.15.76:0/1807946546 >> 172.21.15.76:0/1807946546 conn(0x7f801c0fbbb0 msgr2=0x7f801c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 -- 172.21.15.76:0/1807946546 shutdown_connections 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.215+0000 7f80236c0640 1 -- 172.21.15.76:0/1807946546 wait complete. 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 Processor -- start 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 -- start start 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c100620 0x7f801c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c101810 0x7f801c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c1021d0 0x7f801c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f801c19f2f0 con 0x7f801c1021d0 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f801c19f460 con 0x7f801c100620 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f80236c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f801c19f620 con 0x7f801c101810 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c1021d0 0x7f801c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c1021d0 0x7f801c19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37300/0 (socket says 172.21.15.76:37300) 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 -- 172.21.15.76:0/1767922255 learned_addr learned my addr 172.21.15.76:0/1767922255 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f802145c640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c100620 0x7f801c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8020c5b640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c101810 0x7f801c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 -- 172.21.15.76:0/1767922255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c101810 msgr2=0x7f801c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c101810 0x7f801c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 -- 172.21.15.76:0/1767922255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c100620 msgr2=0x7f801c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c100620 0x7f801c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.216+0000 7f8021c5d640 1 -- 172.21.15.76:0/1767922255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c009560 con 0x7f801c1021d0 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.217+0000 7f8021c5d640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c1021d0 0x7f801c19edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f801400d750 tx=0x7f801400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:24.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.217+0000 7f80127fc640 1 -- 172.21.15.76:0/1767922255 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014009bd0 con 0x7f801c1021d0 2024-03-23T15:41:24.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.217+0000 7f80127fc640 1 -- 172.21.15.76:0/1767922255 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801400de60 con 0x7f801c1021d0 2024-03-23T15:41:24.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.217+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c19f900 con 0x7f801c1021d0 2024-03-23T15:41:24.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.217+0000 7f80127fc640 1 -- 172.21.15.76:0/1767922255 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014004e70 con 0x7f801c1021d0 2024-03-23T15:41:24.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.217+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c19fe30 con 0x7f801c1021d0 2024-03-23T15:41:24.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.218+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c10c030 con 0x7f801c1021d0 2024-03-23T15:41:24.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.219+0000 7f80127fc640 1 -- 172.21.15.76:0/1767922255 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7f8014005020 con 0x7f801c1021d0 2024-03-23T15:41:24.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.219+0000 7f80127fc640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f80080a2140 0x7f80080a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.219+0000 7f802145c640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f80080a2140 0x7f80080a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.219+0000 7f80127fc640 1 -- 172.21.15.76:0/1767922255 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f80140cd350 con 0x7f801c1021d0 2024-03-23T15:41:24.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.220+0000 7f802145c640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f80080a2140 0x7f80080a4600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f801c101670 tx=0x7f8018009720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:24.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.224+0000 7f80127fc640 1 -- 172.21.15.76:0/1767922255 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f801409a050 con 0x7f801c1021d0 2024-03-23T15:41:24.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.426+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f801c0fdd50 con 0x7f80080a2140 2024-03-23T15:41:24.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.431+0000 7f80127fc640 1 -- 172.21.15.76:0/1767922255 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f801c0fdd50 con 0x7f80080a2140 2024-03-23T15:41:24.433 INFO:teuthology.orchestra.run.smithi076.stdout:/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4 2024-03-23T15:41:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f80080a2140 msgr2=0x7f80080a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f80080a2140 0x7f80080a4600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f801c101670 tx=0x7f8018009720 comp rx=0 tx=0).stop 2024-03-23T15:41:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c1021d0 msgr2=0x7f801c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c1021d0 0x7f801c19edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f801400d750 tx=0x7f801400dc10 comp rx=0 tx=0).stop 2024-03-23T15:41:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 shutdown_connections 2024-03-23T15:41:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f801c1021d0 0x7f801c19edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f801c101810 0x7f801c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f80080a2140 0x7f80080a4600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 --2- 172.21.15.76:0/1767922255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c100620 0x7f801c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 >> 172.21.15.76:0/1767922255 conn(0x7f801c0fbbb0 msgr2=0x7f801c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:24.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 shutdown_connections 2024-03-23T15:41:24.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.433+0000 7f80236c0640 1 -- 172.21.15.76:0/1767922255 wait complete. 2024-03-23T15:41:24.446 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.544+0000 7fabe32d4640 1 -- 172.21.15.76:0/266724878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc101820 msgr2=0x7fabdc101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.544+0000 7fabe32d4640 1 --2- 172.21.15.76:0/266724878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc101820 0x7fabdc101ca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009900 tx=0x7fabd802bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- 172.21.15.76:0/266724878 shutdown_connections 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 --2- 172.21.15.76:0/266724878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabdc1021e0 0x7fabdc108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 --2- 172.21.15.76:0/266724878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc101820 0x7fabdc101ca0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 --2- 172.21.15.76:0/266724878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabdc100630 0x7fabdc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- 172.21.15.76:0/266724878 >> 172.21.15.76:0/266724878 conn(0x7fabdc0fbba0 msgr2=0x7fabdc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:24.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- 172.21.15.76:0/266724878 shutdown_connections 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- 172.21.15.76:0/266724878 wait complete. 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 Processor -- start 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- start start 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabdc100630 0x7fabdc197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabdc101820 0x7fabdc197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc1021e0 0x7fabdc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc19f280 con 0x7fabdc1021e0 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc19f3f0 con 0x7fabdc101820 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe32d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc19f5b0 con 0x7fabdc100630 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe1871640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc1021e0 0x7fabdc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe1871640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc1021e0 0x7fabdc19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36872/0 (socket says 172.21.15.76:36872) 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe1871640 1 -- 172.21.15.76:0/1241617129 learned_addr learned my addr 172.21.15.76:0/1241617129 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.545+0000 7fabe1070640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabdc100630 0x7fabdc197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe086f640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabdc101820 0x7fabdc197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe1871640 1 -- 172.21.15.76:0/1241617129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabdc100630 msgr2=0x7fabdc197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe1871640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabdc100630 0x7fabdc197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe1871640 1 -- 172.21.15.76:0/1241617129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabdc101820 msgr2=0x7fabdc197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe1871640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabdc101820 0x7fabdc197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe1871640 1 -- 172.21.15.76:0/1241617129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd8009560 con 0x7fabdc1021e0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe1871640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc1021e0 0x7fabdc19ed40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fabd400eb10 tx=0x7fabd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004b20 con 0x7fabdc1021e0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd4004c80 con 0x7fabdc1021e0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc19f890 con 0x7fabdc1021e0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4010650 con 0x7fabdc1021e0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.546+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc19fd70 con 0x7fabdc1021e0 2024-03-23T15:41:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.547+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc1078f0 con 0x7fabdc1021e0 2024-03-23T15:41:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.548+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) ==== 136756+0+0 (secure 0 0 0) 0x7fabd4005020 con 0x7fabdc1021e0 2024-03-23T15:41:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.549+0000 7fabce7fc640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc80a20c0 0x7fabc80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.549+0000 7fabe1070640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc80a20c0 0x7fabc80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.549+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fabd4014070 con 0x7fabdc1021e0 2024-03-23T15:41:24.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.549+0000 7fabe1070640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc80a20c0 0x7fabc80a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fabdc101680 tx=0x7fabd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:24.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.553+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fabd40971c0 con 0x7fabdc1021e0 2024-03-23T15:41:24.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.714+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fabd4097590 con 0x7fabdc1021e0 2024-03-23T15:41:24.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.769+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabdc10d140 con 0x7fabdc1021e0 2024-03-23T15:41:24.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.770+0000 7fabce7fc640 1 -- 172.21.15.76:0/1241617129 <== mon.0 v2:172.21.15.76:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7fabd40635a0 con 0x7fabdc1021e0 2024-03-23T15:41:24.772 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:24.772 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:41:22.694125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:41:24.773 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 21 2024-03-23T15:41:24.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc80a20c0 msgr2=0x7fabc80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc80a20c0 0x7fabc80a4580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fabdc101680 tx=0x7fabd0005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc1021e0 msgr2=0x7fabdc19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc1021e0 0x7fabdc19ed40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fabd400eb10 tx=0x7fabd400c2d0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 shutdown_connections 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fabdc1021e0 0x7fabdc19ed40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fabc80a20c0 0x7fabc80a4580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabdc101820 0x7fabdc197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 --2- 172.21.15.76:0/1241617129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabdc100630 0x7fabdc197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 >> 172.21.15.76:0/1241617129 conn(0x7fabdc0fbba0 msgr2=0x7fabdc104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 shutdown_connections 2024-03-23T15:41:24.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.772+0000 7fabe32d4640 1 -- 172.21.15.76:0/1241617129 wait complete. 2024-03-23T15:41:24.785 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:41:24.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.885+0000 7f11a38ae640 1 -- 172.21.15.76:0/4074006660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c101820 msgr2=0x7f119c101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.885+0000 7f11a38ae640 1 --2- 172.21.15.76:0/4074006660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c101820 0x7f119c101ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f118c009900 tx=0x7f118c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.885+0000 7f11a38ae640 1 -- 172.21.15.76:0/4074006660 shutdown_connections 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.885+0000 7f11a38ae640 1 --2- 172.21.15.76:0/4074006660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f119c1021e0 0x7f119c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.885+0000 7f11a38ae640 1 --2- 172.21.15.76:0/4074006660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c101820 0x7f119c101ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.885+0000 7f11a38ae640 1 --2- 172.21.15.76:0/4074006660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119c100630 0x7f119c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.885+0000 7f11a38ae640 1 -- 172.21.15.76:0/4074006660 >> 172.21.15.76:0/4074006660 conn(0x7f119c0fbba0 msgr2=0x7f119c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 -- 172.21.15.76:0/4074006660 shutdown_connections 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 -- 172.21.15.76:0/4074006660 wait complete. 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 Processor -- start 2024-03-23T15:41:24.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 -- start start 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119c100630 0x7f119c104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f119c101820 0x7f119c102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c1021e0 0x7f119c103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f119c103970 con 0x7f119c1021e0 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f119c103ae0 con 0x7f119c101820 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a38ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f119c1a1800 con 0x7f119c100630 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a1e4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c1021e0 0x7f119c103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a1e4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c1021e0 0x7f119c103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36890/0 (socket says 172.21.15.76:36890) 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a1e4b640 1 -- 172.21.15.76:0/1427591157 learned_addr learned my addr 172.21.15.76:0/1427591157 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a164a640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119c100630 0x7f119c104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.886+0000 7f11a0e49640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f119c101820 0x7f119c102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a1e4b640 1 -- 172.21.15.76:0/1427591157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119c100630 msgr2=0x7f119c104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a1e4b640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119c100630 0x7f119c104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a1e4b640 1 -- 172.21.15.76:0/1427591157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f119c101820 msgr2=0x7f119c102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a1e4b640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f119c101820 0x7f119c102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a1e4b640 1 -- 172.21.15.76:0/1427591157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c009560 con 0x7f119c1021e0 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a1e4b640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c1021e0 0x7f119c103210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f119400eb40 tx=0x7f119400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11927fc640 1 -- 172.21.15.76:0/1427591157 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194004b20 con 0x7f119c1021e0 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11927fc640 1 -- 172.21.15.76:0/1427591157 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1194004c80 con 0x7f119c1021e0 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c1a1ab0 con 0x7f119c1021e0 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11927fc640 1 -- 172.21.15.76:0/1427591157 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194010670 con 0x7f119c1021e0 2024-03-23T15:41:24.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.887+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c1a1f40 con 0x7f119c1021e0 2024-03-23T15:41:24.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.888+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119c10c040 con 0x7f119c1021e0 2024-03-23T15:41:24.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.889+0000 7f11927fc640 1 -- 172.21.15.76:0/1427591157 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f1194005020 con 0x7f119c1021e0 2024-03-23T15:41:24.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.890+0000 7f11927fc640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f11880a2150 0x7f11880a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:24.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.890+0000 7f11a164a640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f11880a2150 0x7f11880a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:24.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.890+0000 7f11927fc640 1 -- 172.21.15.76:0/1427591157 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1194014070 con 0x7f119c1021e0 2024-03-23T15:41:24.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.890+0000 7f11a164a640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f11880a2150 0x7f11880a4610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f119c101680 tx=0x7f1198005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:24.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:24.894+0000 7f11927fc640 1 -- 172.21.15.76:0/1427591157 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1194097260 con 0x7f119c1021e0 2024-03-23T15:41:25.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:24 smithi192 ceph-mon[36653]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:24 smithi192 ceph-mon[36653]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:24 smithi192 ceph-mon[36653]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:24 smithi076 ceph-mon[32176]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:24 smithi076 ceph-mon[32176]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:24 smithi076 ceph-mon[32176]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.074+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f119c1a2450 con 0x7f119c1021e0 2024-03-23T15:41:25.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.075+0000 7f11927fc640 1 -- 172.21.15.76:0/1427591157 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22203 (secure 0 0 0) 0x7f119409a920 con 0x7f119c1021e0 2024-03-23T15:41:25.078 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:25.078 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":85,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:41:19.628233+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:41:25.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f11880a2150 msgr2=0x7f11880a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f11880a2150 0x7f11880a4610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f119c101680 tx=0x7f1198005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c1021e0 msgr2=0x7f119c103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c1021e0 0x7f119c103210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f119400eb40 tx=0x7f119400c2d0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 shutdown_connections 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f119c1021e0 0x7f119c103210 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f11880a2150 0x7f11880a4610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f119c101820 0x7f119c102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 --2- 172.21.15.76:0/1427591157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f119c100630 0x7f119c104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 >> 172.21.15.76:0/1427591157 conn(0x7f119c0fbba0 msgr2=0x7f119c069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 shutdown_connections 2024-03-23T15:41:25.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.079+0000 7f11a38ae640 1 -- 172.21.15.76:0/1427591157 wait complete. 2024-03-23T15:41:25.094 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T15:41:25.095 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T15:41:25.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:24 smithi183 ceph-mon[36296]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:24 smithi183 ceph-mon[36296]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:24 smithi183 ceph-mon[36296]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 -- 172.21.15.76:0/2615200149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16601021f0 msgr2=0x7f1660108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 --2- 172.21.15.76:0/2615200149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16601021f0 0x7f1660108a80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f165c009900 tx=0x7f165c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 -- 172.21.15.76:0/2615200149 shutdown_connections 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 --2- 172.21.15.76:0/2615200149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16601021f0 0x7f1660108a80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 --2- 172.21.15.76:0/2615200149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1660101830 0x7f1660101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 --2- 172.21.15.76:0/2615200149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1660100640 0x7f1660100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 -- 172.21.15.76:0/2615200149 >> 172.21.15.76:0/2615200149 conn(0x7f16600fbbf0 msgr2=0x7f16600fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 -- 172.21.15.76:0/2615200149 shutdown_connections 2024-03-23T15:41:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 -- 172.21.15.76:0/2615200149 wait complete. 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 Processor -- start 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.195+0000 7f1667ec9640 1 -- start start 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1660100640 0x7f1660197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1660101830 0x7f1660197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16601021f0 0x7f166019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f166019f280 con 0x7f1660100640 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f166019f3f0 con 0x7f1660101830 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f166019f5b0 con 0x7f16601021f0 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1660100640 0x7f1660197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1660100640 0x7f1660197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36900/0 (socket says 172.21.15.76:36900) 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 -- 172.21.15.76:0/3604407987 learned_addr learned my addr 172.21.15.76:0/3604407987 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665464640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1660101830 0x7f1660197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1666466640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16601021f0 0x7f166019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 -- 172.21.15.76:0/3604407987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16601021f0 msgr2=0x7f166019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16601021f0 0x7f166019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 -- 172.21.15.76:0/3604407987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1660101830 msgr2=0x7f1660197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1660101830 0x7f1660197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 -- 172.21.15.76:0/3604407987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165c009560 con 0x7f1660100640 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1665c65640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1660100640 0x7f1660197790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f165800eb70 tx=0x7f165800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1656ffd640 1 -- 172.21.15.76:0/3604407987 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658003b70 con 0x7f1660100640 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1656ffd640 1 -- 172.21.15.76:0/3604407987 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f165800ee70 con 0x7f1660100640 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1656ffd640 1 -- 172.21.15.76:0/3604407987 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580107f0 con 0x7f1660100640 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166019f890 con 0x7f1660100640 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.196+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166019fd50 con 0x7f1660100640 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.197+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166010c050 con 0x7f1660100640 2024-03-23T15:41:25.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.198+0000 7f1656ffd640 1 -- 172.21.15.76:0/3604407987 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f1658004020 con 0x7f1660100640 2024-03-23T15:41:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.199+0000 7f1656ffd640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f164c0a2150 0x7f164c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.199+0000 7f1665464640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f164c0a2150 0x7f164c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.199+0000 7f1656ffd640 1 -- 172.21.15.76:0/3604407987 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1658014070 con 0x7f1660100640 2024-03-23T15:41:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.199+0000 7f1665464640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f164c0a2150 0x7f164c0a4610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1660198ca0 tx=0x7f1650005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:25.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.203+0000 7f1656ffd640 1 -- 172.21.15.76:0/3604407987 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f16580972a0 con 0x7f1660100640 2024-03-23T15:41:25.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.375+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f166006a490 con 0x7f164c0a2150 2024-03-23T15:41:25.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.383+0000 7f1656ffd640 1 -- 172.21.15.76:0/3604407987 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258863 (secure 0 0 0) 0x7f166006a490 con 0x7f164c0a2150 2024-03-23T15:41:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f164c0a2150 msgr2=0x7f164c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f164c0a2150 0x7f164c0a4610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1660198ca0 tx=0x7f1650005f90 comp rx=0 tx=0).stop 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1660100640 msgr2=0x7f1660197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1660100640 0x7f1660197790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f165800eb70 tx=0x7f165800c6a0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 shutdown_connections 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16601021f0 0x7f166019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f164c0a2150 0x7f164c0a4610 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1660101830 0x7f1660197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 --2- 172.21.15.76:0/3604407987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1660100640 0x7f1660197790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 >> 172.21.15.76:0/3604407987 conn(0x7f16600fbbf0 msgr2=0x7f1660069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:25.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 shutdown_connections 2024-03-23T15:41:25.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.386+0000 7f1667ec9640 1 -- 172.21.15.76:0/3604407987 wait complete. 2024-03-23T15:41:25.399 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-23T15:41:23.476338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592890,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":252,"num_write_kb":2954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368408,"kb_used_data":10864,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573416,"statfs":{"total":1151940427776,"available":1151563177984,"internally_reserved":0,"allocated":11124736,"data_stored":6964909,"data_compressed":106812,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2522,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":71,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001325"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.656493+0000","last_change":"2024-03-23T15:41:18.651731+0000","last_active":"2024-03-23T15:41:18.656493+0000","last_peered":"2024-03-23T15:41:18.656493+0000","last_clean":"2024-03-23T15:41:18.656493+0000","last_became_active":"2024-03-23T15:41:18.651312+0000","last_became_peered":"2024-03-23T15:41:18.651312+0000","last_unstale":"2024-03-23T15:41:18.656493+0000","last_undegraded":"2024-03-23T15:41:18.656493+0000","last_fullsized":"2024-03-23T15:41:18.656493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:50:27.332304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879191+0000","last_change":"2024-03-23T15:41:18.655736+0000","last_active":"2024-03-23T15:41:19.879191+0000","last_peered":"2024-03-23T15:41:19.879191+0000","last_clean":"2024-03-23T15:41:19.879191+0000","last_became_active":"2024-03-23T15:41:18.655389+0000","last_became_peered":"2024-03-23T15:41:18.655389+0000","last_unstale":"2024-03-23T15:41:19.879191+0000","last_undegraded":"2024-03-23T15:41:19.879191+0000","last_fullsized":"2024-03-23T15:41:19.879191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:35:56.260056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878534+0000","last_change":"2024-03-23T15:41:18.650709+0000","last_active":"2024-03-23T15:41:19.878534+0000","last_peered":"2024-03-23T15:41:19.878534+0000","last_clean":"2024-03-23T15:41:19.878534+0000","last_became_active":"2024-03-23T15:41:18.650316+0000","last_became_peered":"2024-03-23T15:41:18.650316+0000","last_unstale":"2024-03-23T15:41:19.878534+0000","last_undegraded":"2024-03-23T15:41:19.878534+0000","last_fullsized":"2024-03-23T15:41:19.878534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:50:57.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.653074+0000","last_change":"2024-03-23T15:41:18.652216+0000","last_active":"2024-03-23T15:41:18.653074+0000","last_peered":"2024-03-23T15:41:18.653074+0000","last_clean":"2024-03-23T15:41:18.653074+0000","last_became_active":"2024-03-23T15:41:18.650525+0000","last_became_peered":"2024-03-23T15:41:18.650525+0000","last_unstale":"2024-03-23T15:41:18.653074+0000","last_undegraded":"2024-03-23T15:41:18.653074+0000","last_fullsized":"2024-03-23T15:41:18.653074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:31:40.891854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878504+0000","last_change":"2024-03-23T15:41:18.649146+0000","last_active":"2024-03-23T15:41:19.878504+0000","last_peered":"2024-03-23T15:41:19.878504+0000","last_clean":"2024-03-23T15:41:19.878504+0000","last_became_active":"2024-03-23T15:41:18.648728+0000","last_became_peered":"2024-03-23T15:41:18.648728+0000","last_unstale":"2024-03-23T15:41:19.878504+0000","last_undegraded":"2024-03-23T15:41:19.878504+0000","last_fullsized":"2024-03-23T15:41:19.878504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:56.251250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879291+0000","last_change":"2024-03-23T15:41:18.652957+0000","last_active":"2024-03-23T15:41:19.879291+0000","last_peered":"2024-03-23T15:41:19.879291+0000","last_clean":"2024-03-23T15:41:19.879291+0000","last_became_active":"2024-03-23T15:41:18.652578+0000","last_became_peered":"2024-03-23T15:41:18.652578+0000","last_unstale":"2024-03-23T15:41:19.879291+0000","last_undegraded":"2024-03-23T15:41:19.879291+0000","last_fullsized":"2024-03-23T15:41:19.879291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:46:38.961554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.653807+0000","last_change":"2024-03-23T15:41:18.646134+0000","last_active":"2024-03-23T15:41:18.653807+0000","last_peered":"2024-03-23T15:41:18.653807+0000","last_clean":"2024-03-23T15:41:18.653807+0000","last_became_active":"2024-03-23T15:41:18.645680+0000","last_became_peered":"2024-03-23T15:41:18.645680+0000","last_unstale":"2024-03-23T15:41:18.653807+0000","last_undegraded":"2024-03-23T15:41:18.653807+0000","last_fullsized":"2024-03-23T15:41:18.653807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:26:06.249368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.648020+0000","last_change":"2024-03-23T15:41:18.646143+0000","last_active":"2024-03-23T15:41:18.648020+0000","last_peered":"2024-03-23T15:41:18.648020+0000","last_clean":"2024-03-23T15:41:18.648020+0000","last_became_active":"2024-03-23T15:41:18.645771+0000","last_became_peered":"2024-03-23T15:41:18.645771+0000","last_unstale":"2024-03-23T15:41:18.648020+0000","last_undegraded":"2024-03-23T15:41:18.648020+0000","last_fullsized":"2024-03-23T15:41:18.648020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:06:17.019265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.635038+0000","last_change":"2024-03-23T15:41:18.665083+0000","last_active":"2024-03-23T15:41:19.635038+0000","last_peered":"2024-03-23T15:41:19.635038+0000","last_clean":"2024-03-23T15:41:19.635038+0000","last_became_active":"2024-03-23T15:41:18.663143+0000","last_became_peered":"2024-03-23T15:41:18.663143+0000","last_unstale":"2024-03-23T15:41:19.635038+0000","last_undegraded":"2024-03-23T15:41:19.635038+0000","last_fullsized":"2024-03-23T15:41:19.635038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:52:23.421645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879109+0000","last_change":"2024-03-23T15:41:18.650014+0000","last_active":"2024-03-23T15:41:19.879109+0000","last_peered":"2024-03-23T15:41:19.879109+0000","last_clean":"2024-03-23T15:41:19.879109+0000","last_became_active":"2024-03-23T15:41:18.649673+0000","last_became_peered":"2024-03-23T15:41:18.649673+0000","last_unstale":"2024-03-23T15:41:19.879109+0000","last_undegraded":"2024-03-23T15:41:19.879109+0000","last_fullsized":"2024-03-23T15:41:19.879109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:30.387993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.666171+0000","last_change":"2024-03-23T15:41:18.665622+0000","last_active":"2024-03-23T15:41:18.666171+0000","last_peered":"2024-03-23T15:41:18.666171+0000","last_clean":"2024-03-23T15:41:18.666171+0000","last_became_active":"2024-03-23T15:41:18.665274+0000","last_became_peered":"2024-03-23T15:41:18.665274+0000","last_unstale":"2024-03-23T15:41:18.666171+0000","last_undegraded":"2024-03-23T15:41:18.666171+0000","last_fullsized":"2024-03-23T15:41:18.666171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:02:34.712224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.651924+0000","last_change":"2024-03-23T15:41:18.650888+0000","last_active":"2024-03-23T15:41:18.651924+0000","last_peered":"2024-03-23T15:41:18.651924+0000","last_clean":"2024-03-23T15:41:18.651924+0000","last_became_active":"2024-03-23T15:41:18.650363+0000","last_became_peered":"2024-03-23T15:41:18.650363+0000","last_unstale":"2024-03-23T15:41:18.651924+0000","last_undegraded":"2024-03-23T15:41:18.651924+0000","last_fullsized":"2024-03-23T15:41:18.651924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:30:03.635233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631546+0000","last_change":"2024-03-23T15:41:18.634183+0000","last_active":"2024-03-23T15:41:19.631546+0000","last_peered":"2024-03-23T15:41:19.631546+0000","last_clean":"2024-03-23T15:41:19.631546+0000","last_became_active":"2024-03-23T15:41:18.633849+0000","last_became_peered":"2024-03-23T15:41:18.633849+0000","last_unstale":"2024-03-23T15:41:19.631546+0000","last_undegraded":"2024-03-23T15:41:19.631546+0000","last_fullsized":"2024-03-23T15:41:19.631546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:13.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877999+0000","last_change":"2024-03-23T15:41:18.645269+0000","last_active":"2024-03-23T15:41:19.877999+0000","last_peered":"2024-03-23T15:41:19.877999+0000","last_clean":"2024-03-23T15:41:19.877999+0000","last_became_active":"2024-03-23T15:41:18.644940+0000","last_became_peered":"2024-03-23T15:41:18.644940+0000","last_unstale":"2024-03-23T15:41:19.877999+0000","last_undegraded":"2024-03-23T15:41:19.877999+0000","last_fullsized":"2024-03-23T15:41:19.877999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:26.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877820+0000","last_change":"2024-03-23T15:41:18.652789+0000","last_active":"2024-03-23T15:41:19.877820+0000","last_peered":"2024-03-23T15:41:19.877820+0000","last_clean":"2024-03-23T15:41:19.877820+0000","last_became_active":"2024-03-23T15:41:18.651520+0000","last_became_peered":"2024-03-23T15:41:18.651520+0000","last_unstale":"2024-03-23T15:41:19.877820+0000","last_undegraded":"2024-03-23T15:41:19.877820+0000","last_fullsized":"2024-03-23T15:41:19.877820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:00.388217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634507+0000","last_change":"2024-03-23T15:41:18.660398+0000","last_active":"2024-03-23T15:41:19.634507+0000","last_peered":"2024-03-23T15:41:19.634507+0000","last_clean":"2024-03-23T15:41:19.634507+0000","last_became_active":"2024-03-23T15:41:18.660084+0000","last_became_peered":"2024-03-23T15:41:18.660084+0000","last_unstale":"2024-03-23T15:41:19.634507+0000","last_undegraded":"2024-03-23T15:41:19.634507+0000","last_fullsized":"2024-03-23T15:41:19.634507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:04:33.115173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.666788+0000","last_change":"2024-03-23T15:41:18.663192+0000","last_active":"2024-03-23T15:41:18.666788+0000","last_peered":"2024-03-23T15:41:18.666788+0000","last_clean":"2024-03-23T15:41:18.666788+0000","last_became_active":"2024-03-23T15:41:18.662844+0000","last_became_peered":"2024-03-23T15:41:18.662844+0000","last_unstale":"2024-03-23T15:41:18.666788+0000","last_undegraded":"2024-03-23T15:41:18.666788+0000","last_fullsized":"2024-03-23T15:41:18.666788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:47:03.852603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878908+0000","last_change":"2024-03-23T15:41:18.646405+0000","last_active":"2024-03-23T15:41:19.878908+0000","last_peered":"2024-03-23T15:41:19.878908+0000","last_clean":"2024-03-23T15:41:19.878908+0000","last_became_active":"2024-03-23T15:41:18.645516+0000","last_became_peered":"2024-03-23T15:41:18.645516+0000","last_unstale":"2024-03-23T15:41:19.878908+0000","last_undegraded":"2024-03-23T15:41:19.878908+0000","last_fullsized":"2024-03-23T15:41:19.878908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:29:18.588190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632579+0000","last_change":"2024-03-23T15:41:18.652344+0000","last_active":"2024-03-23T15:41:19.632579+0000","last_peered":"2024-03-23T15:41:19.632579+0000","last_clean":"2024-03-23T15:41:19.632579+0000","last_became_active":"2024-03-23T15:41:18.650985+0000","last_became_peered":"2024-03-23T15:41:18.650985+0000","last_unstale":"2024-03-23T15:41:19.632579+0000","last_undegraded":"2024-03-23T15:41:19.632579+0000","last_fullsized":"2024-03-23T15:41:19.632579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:40.741299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.701617+0000","last_change":"2024-03-23T15:41:17.655443+0000","last_active":"2024-03-23T15:41:21.701617+0000","last_peered":"2024-03-23T15:41:21.701617+0000","last_clean":"2024-03-23T15:41:21.701617+0000","last_became_active":"2024-03-23T15:41:17.654834+0000","last_became_peered":"2024-03-23T15:41:17.654834+0000","last_unstale":"2024-03-23T15:41:21.701617+0000","last_undegraded":"2024-03-23T15:41:21.701617+0000","last_fullsized":"2024-03-23T15:41:21.701617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:49:43.215644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.654026+0000","last_change":"2024-03-23T15:41:18.650241+0000","last_active":"2024-03-23T15:41:18.654026+0000","last_peered":"2024-03-23T15:41:18.654026+0000","last_clean":"2024-03-23T15:41:18.654026+0000","last_became_active":"2024-03-23T15:41:18.649914+0000","last_became_peered":"2024-03-23T15:41:18.649914+0000","last_unstale":"2024-03-23T15:41:18.654026+0000","last_undegraded":"2024-03-23T15:41:18.654026+0000","last_fullsized":"2024-03-23T15:41:18.654026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:25:28.894862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.669956+0000","last_change":"2024-03-23T15:41:17.663308+0000","last_active":"2024-03-23T15:41:18.669956+0000","last_peered":"2024-03-23T15:41:18.669956+0000","last_clean":"2024-03-23T15:41:18.669956+0000","last_became_active":"2024-03-23T15:41:17.662958+0000","last_became_peered":"2024-03-23T15:41:17.662958+0000","last_unstale":"2024-03-23T15:41:18.669956+0000","last_undegraded":"2024-03-23T15:41:18.669956+0000","last_fullsized":"2024-03-23T15:41:18.669956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:43:37.512893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878962+0000","last_change":"2024-03-23T15:41:18.651225+0000","last_active":"2024-03-23T15:41:19.878962+0000","last_peered":"2024-03-23T15:41:19.878962+0000","last_clean":"2024-03-23T15:41:19.878962+0000","last_became_active":"2024-03-23T15:41:18.650884+0000","last_became_peered":"2024-03-23T15:41:18.650884+0000","last_unstale":"2024-03-23T15:41:19.878962+0000","last_undegraded":"2024-03-23T15:41:19.878962+0000","last_fullsized":"2024-03-23T15:41:19.878962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:00.445203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879181+0000","last_change":"2024-03-23T15:41:17.662753+0000","last_active":"2024-03-23T15:41:19.879181+0000","last_peered":"2024-03-23T15:41:19.879181+0000","last_clean":"2024-03-23T15:41:19.879181+0000","last_became_active":"2024-03-23T15:41:17.662378+0000","last_became_peered":"2024-03-23T15:41:17.662378+0000","last_unstale":"2024-03-23T15:41:19.879181+0000","last_undegraded":"2024-03-23T15:41:19.879181+0000","last_fullsized":"2024-03-23T15:41:19.879181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:58:55.674739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634632+0000","last_change":"2024-03-23T15:41:18.659171+0000","last_active":"2024-03-23T15:41:19.634632+0000","last_peered":"2024-03-23T15:41:19.634632+0000","last_clean":"2024-03-23T15:41:19.634632+0000","last_became_active":"2024-03-23T15:41:18.658836+0000","last_became_peered":"2024-03-23T15:41:18.658836+0000","last_unstale":"2024-03-23T15:41:19.634632+0000","last_undegraded":"2024-03-23T15:41:19.634632+0000","last_fullsized":"2024-03-23T15:41:19.634632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:15.056528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879027+0000","last_change":"2024-03-23T15:41:17.655842+0000","last_active":"2024-03-23T15:41:19.879027+0000","last_peered":"2024-03-23T15:41:19.879027+0000","last_clean":"2024-03-23T15:41:19.879027+0000","last_became_active":"2024-03-23T15:41:17.655452+0000","last_became_peered":"2024-03-23T15:41:17.655452+0000","last_unstale":"2024-03-23T15:41:19.879027+0000","last_undegraded":"2024-03-23T15:41:19.879027+0000","last_fullsized":"2024-03-23T15:41:19.879027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:27:50.518854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.649896+0000","last_change":"2024-03-23T15:41:18.648500+0000","last_active":"2024-03-23T15:41:18.649896+0000","last_peered":"2024-03-23T15:41:18.649896+0000","last_clean":"2024-03-23T15:41:18.649896+0000","last_became_active":"2024-03-23T15:41:18.648139+0000","last_became_peered":"2024-03-23T15:41:18.648139+0000","last_unstale":"2024-03-23T15:41:18.649896+0000","last_undegraded":"2024-03-23T15:41:18.649896+0000","last_fullsized":"2024-03-23T15:41:18.649896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:21:12.223013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.668204+0000","last_change":"2024-03-23T15:41:17.658471+0000","last_active":"2024-03-23T15:41:18.668204+0000","last_peered":"2024-03-23T15:41:18.668204+0000","last_clean":"2024-03-23T15:41:18.668204+0000","last_became_active":"2024-03-23T15:41:17.658142+0000","last_became_peered":"2024-03-23T15:41:17.658142+0000","last_unstale":"2024-03-23T15:41:18.668204+0000","last_undegraded":"2024-03-23T15:41:18.668204+0000","last_fullsized":"2024-03-23T15:41:18.668204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:23.224790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878537+0000","last_change":"2024-03-23T15:41:18.646046+0000","last_active":"2024-03-23T15:41:19.878537+0000","last_peered":"2024-03-23T15:41:19.878537+0000","last_clean":"2024-03-23T15:41:19.878537+0000","last_became_active":"2024-03-23T15:41:18.645727+0000","last_became_peered":"2024-03-23T15:41:18.645727+0000","last_unstale":"2024-03-23T15:41:19.878537+0000","last_undegraded":"2024-03-23T15:41:19.878537+0000","last_fullsized":"2024-03-23T15:41:19.878537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:55.552079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.699051+0000","last_change":"2024-03-23T15:41:17.657285+0000","last_active":"2024-03-23T15:41:21.699051+0000","last_peered":"2024-03-23T15:41:21.699051+0000","last_clean":"2024-03-23T15:41:21.699051+0000","last_became_active":"2024-03-23T15:41:17.656942+0000","last_became_peered":"2024-03-23T15:41:17.656942+0000","last_unstale":"2024-03-23T15:41:21.699051+0000","last_undegraded":"2024-03-23T15:41:21.699051+0000","last_fullsized":"2024-03-23T15:41:21.699051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:18.107467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.656597+0000","last_change":"2024-03-23T15:41:18.655175+0000","last_active":"2024-03-23T15:41:18.656597+0000","last_peered":"2024-03-23T15:41:18.656597+0000","last_clean":"2024-03-23T15:41:18.656597+0000","last_became_active":"2024-03-23T15:41:18.654846+0000","last_became_peered":"2024-03-23T15:41:18.654846+0000","last_unstale":"2024-03-23T15:41:18.656597+0000","last_undegraded":"2024-03-23T15:41:18.656597+0000","last_fullsized":"2024-03-23T15:41:18.656597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:13.133333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631232+0000","last_change":"2024-03-23T15:41:17.657391+0000","last_active":"2024-03-23T15:41:19.631232+0000","last_peered":"2024-03-23T15:41:19.631232+0000","last_clean":"2024-03-23T15:41:19.631232+0000","last_became_active":"2024-03-23T15:41:17.657081+0000","last_became_peered":"2024-03-23T15:41:17.657081+0000","last_unstale":"2024-03-23T15:41:19.631232+0000","last_undegraded":"2024-03-23T15:41:19.631232+0000","last_fullsized":"2024-03-23T15:41:19.631232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:08.744477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.657099+0000","last_change":"2024-03-23T15:41:18.656268+0000","last_active":"2024-03-23T15:41:18.657099+0000","last_peered":"2024-03-23T15:41:18.657099+0000","last_clean":"2024-03-23T15:41:18.657099+0000","last_became_active":"2024-03-23T15:41:18.655910+0000","last_became_peered":"2024-03-23T15:41:18.655910+0000","last_unstale":"2024-03-23T15:41:18.657099+0000","last_undegraded":"2024-03-23T15:41:18.657099+0000","last_fullsized":"2024-03-23T15:41:18.657099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:28.086044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.631930+0000","last_change":"2024-03-23T15:41:17.656555+0000","last_active":"2024-03-23T15:41:18.631930+0000","last_peered":"2024-03-23T15:41:18.631930+0000","last_clean":"2024-03-23T15:41:18.631930+0000","last_became_active":"2024-03-23T15:41:17.655732+0000","last_became_peered":"2024-03-23T15:41:17.655732+0000","last_unstale":"2024-03-23T15:41:18.631930+0000","last_undegraded":"2024-03-23T15:41:18.631930+0000","last_fullsized":"2024-03-23T15:41:18.631930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:53.839841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878572+0000","last_change":"2024-03-23T15:41:18.647334+0000","last_active":"2024-03-23T15:41:19.878572+0000","last_peered":"2024-03-23T15:41:19.878572+0000","last_clean":"2024-03-23T15:41:19.878572+0000","last_became_active":"2024-03-23T15:41:18.646679+0000","last_became_peered":"2024-03-23T15:41:18.646679+0000","last_unstale":"2024-03-23T15:41:19.878572+0000","last_undegraded":"2024-03-23T15:41:19.878572+0000","last_fullsized":"2024-03-23T15:41:19.878572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.710759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.688484+0000","last_change":"2024-03-23T15:41:17.655400+0000","last_active":"2024-03-23T15:41:20.688484+0000","last_peered":"2024-03-23T15:41:20.688484+0000","last_clean":"2024-03-23T15:41:20.688484+0000","last_became_active":"2024-03-23T15:41:17.655034+0000","last_became_peered":"2024-03-23T15:41:17.655034+0000","last_unstale":"2024-03-23T15:41:20.688484+0000","last_undegraded":"2024-03-23T15:41:20.688484+0000","last_fullsized":"2024-03-23T15:41:20.688484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:30.973048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634925+0000","last_change":"2024-03-23T15:41:18.660248+0000","last_active":"2024-03-23T15:41:19.634925+0000","last_peered":"2024-03-23T15:41:19.634925+0000","last_clean":"2024-03-23T15:41:19.634925+0000","last_became_active":"2024-03-23T15:41:18.659728+0000","last_became_peered":"2024-03-23T15:41:18.659728+0000","last_unstale":"2024-03-23T15:41:19.634925+0000","last_undegraded":"2024-03-23T15:41:19.634925+0000","last_fullsized":"2024-03-23T15:41:19.634925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:03.681499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.638545+0000","last_change":"2024-03-23T15:41:17.659784+0000","last_active":"2024-03-23T15:41:18.638545+0000","last_peered":"2024-03-23T15:41:18.638545+0000","last_clean":"2024-03-23T15:41:18.638545+0000","last_became_active":"2024-03-23T15:41:17.659422+0000","last_became_peered":"2024-03-23T15:41:17.659422+0000","last_unstale":"2024-03-23T15:41:18.638545+0000","last_undegraded":"2024-03-23T15:41:18.638545+0000","last_fullsized":"2024-03-23T15:41:18.638545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:48.607487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878809+0000","last_change":"2024-03-23T15:41:18.651777+0000","last_active":"2024-03-23T15:41:19.878809+0000","last_peered":"2024-03-23T15:41:19.878809+0000","last_clean":"2024-03-23T15:41:19.878809+0000","last_became_active":"2024-03-23T15:41:18.651457+0000","last_became_peered":"2024-03-23T15:41:18.651457+0000","last_unstale":"2024-03-23T15:41:19.878809+0000","last_undegraded":"2024-03-23T15:41:19.878809+0000","last_fullsized":"2024-03-23T15:41:19.878809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:25:28.831493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.670130+0000","last_change":"2024-03-23T15:41:17.656357+0000","last_active":"2024-03-23T15:41:18.670130+0000","last_peered":"2024-03-23T15:41:18.670130+0000","last_clean":"2024-03-23T15:41:18.670130+0000","last_became_active":"2024-03-23T15:41:17.655977+0000","last_became_peered":"2024-03-23T15:41:17.655977+0000","last_unstale":"2024-03-23T15:41:18.670130+0000","last_undegraded":"2024-03-23T15:41:18.670130+0000","last_fullsized":"2024-03-23T15:41:18.670130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:07:45.248954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634728+0000","last_change":"2024-03-23T15:41:18.664487+0000","last_active":"2024-03-23T15:41:19.634728+0000","last_peered":"2024-03-23T15:41:19.634728+0000","last_clean":"2024-03-23T15:41:19.634728+0000","last_became_active":"2024-03-23T15:41:18.664141+0000","last_became_peered":"2024-03-23T15:41:18.664141+0000","last_unstale":"2024-03-23T15:41:19.634728+0000","last_undegraded":"2024-03-23T15:41:19.634728+0000","last_fullsized":"2024-03-23T15:41:19.634728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:21.960392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:22.702537+0000","last_change":"2024-03-23T15:41:17.659838+0000","last_active":"2024-03-23T15:41:22.702537+0000","last_peered":"2024-03-23T15:41:22.702537+0000","last_clean":"2024-03-23T15:41:22.702537+0000","last_became_active":"2024-03-23T15:41:17.659326+0000","last_became_peered":"2024-03-23T15:41:17.659326+0000","last_unstale":"2024-03-23T15:41:22.702537+0000","last_undegraded":"2024-03-23T15:41:22.702537+0000","last_fullsized":"2024-03-23T15:41:22.702537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:07.651340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878964+0000","last_change":"2024-03-23T15:41:18.656629+0000","last_active":"2024-03-23T15:41:19.878964+0000","last_peered":"2024-03-23T15:41:19.878964+0000","last_clean":"2024-03-23T15:41:19.878964+0000","last_became_active":"2024-03-23T15:41:18.656196+0000","last_became_peered":"2024-03-23T15:41:18.656196+0000","last_unstale":"2024-03-23T15:41:19.878964+0000","last_undegraded":"2024-03-23T15:41:19.878964+0000","last_fullsized":"2024-03-23T15:41:19.878964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:30.013411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632499+0000","last_change":"2024-03-23T15:41:17.663999+0000","last_active":"2024-03-23T15:41:19.632499+0000","last_peered":"2024-03-23T15:41:19.632499+0000","last_clean":"2024-03-23T15:41:19.632499+0000","last_became_active":"2024-03-23T15:41:17.663039+0000","last_became_peered":"2024-03-23T15:41:17.663039+0000","last_unstale":"2024-03-23T15:41:19.632499+0000","last_undegraded":"2024-03-23T15:41:19.632499+0000","last_fullsized":"2024-03-23T15:41:19.632499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:00.271821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631428+0000","last_change":"2024-03-23T15:41:18.646822+0000","last_active":"2024-03-23T15:41:19.631428+0000","last_peered":"2024-03-23T15:41:19.631428+0000","last_clean":"2024-03-23T15:41:19.631428+0000","last_became_active":"2024-03-23T15:41:18.646476+0000","last_became_peered":"2024-03-23T15:41:18.646476+0000","last_unstale":"2024-03-23T15:41:19.631428+0000","last_undegraded":"2024-03-23T15:41:19.631428+0000","last_fullsized":"2024-03-23T15:41:19.631428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:18.352065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.631535+0000","last_change":"2024-03-23T15:41:17.663507+0000","last_active":"2024-03-23T15:41:18.631535+0000","last_peered":"2024-03-23T15:41:18.631535+0000","last_clean":"2024-03-23T15:41:18.631535+0000","last_became_active":"2024-03-23T15:41:17.663122+0000","last_became_peered":"2024-03-23T15:41:17.663122+0000","last_unstale":"2024-03-23T15:41:18.631535+0000","last_undegraded":"2024-03-23T15:41:18.631535+0000","last_fullsized":"2024-03-23T15:41:18.631535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:32.202576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.653986+0000","last_change":"2024-03-23T15:41:18.650110+0000","last_active":"2024-03-23T15:41:18.653986+0000","last_peered":"2024-03-23T15:41:18.653986+0000","last_clean":"2024-03-23T15:41:18.653986+0000","last_became_active":"2024-03-23T15:41:18.649737+0000","last_became_peered":"2024-03-23T15:41:18.649737+0000","last_unstale":"2024-03-23T15:41:18.653986+0000","last_undegraded":"2024-03-23T15:41:18.653986+0000","last_fullsized":"2024-03-23T15:41:18.653986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:13:59.528772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.644753+0000","last_change":"2024-03-23T15:41:17.653930+0000","last_active":"2024-03-23T15:41:18.644753+0000","last_peered":"2024-03-23T15:41:18.644753+0000","last_clean":"2024-03-23T15:41:18.644753+0000","last_became_active":"2024-03-23T15:41:17.652677+0000","last_became_peered":"2024-03-23T15:41:17.652677+0000","last_unstale":"2024-03-23T15:41:18.644753+0000","last_undegraded":"2024-03-23T15:41:18.644753+0000","last_fullsized":"2024-03-23T15:41:18.644753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:47.167052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.654252+0000","last_change":"2024-03-23T15:41:18.647269+0000","last_active":"2024-03-23T15:41:18.654252+0000","last_peered":"2024-03-23T15:41:18.654252+0000","last_clean":"2024-03-23T15:41:18.654252+0000","last_became_active":"2024-03-23T15:41:18.646876+0000","last_became_peered":"2024-03-23T15:41:18.646876+0000","last_unstale":"2024-03-23T15:41:18.654252+0000","last_undegraded":"2024-03-23T15:41:18.654252+0000","last_fullsized":"2024-03-23T15:41:18.654252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:59:39.889577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.689253+0000","last_change":"2024-03-23T15:41:17.662091+0000","last_active":"2024-03-23T15:41:20.689253+0000","last_peered":"2024-03-23T15:41:20.689253+0000","last_clean":"2024-03-23T15:41:20.689253+0000","last_became_active":"2024-03-23T15:41:17.661747+0000","last_became_peered":"2024-03-23T15:41:17.661747+0000","last_unstale":"2024-03-23T15:41:20.689253+0000","last_undegraded":"2024-03-23T15:41:20.689253+0000","last_fullsized":"2024-03-23T15:41:20.689253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:08.661262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.655688+0000","last_change":"2024-03-23T15:41:18.654356+0000","last_active":"2024-03-23T15:41:18.655688+0000","last_peered":"2024-03-23T15:41:18.655688+0000","last_clean":"2024-03-23T15:41:18.655688+0000","last_became_active":"2024-03-23T15:41:18.654021+0000","last_became_peered":"2024-03-23T15:41:18.654021+0000","last_unstale":"2024-03-23T15:41:18.655688+0000","last_undegraded":"2024-03-23T15:41:18.655688+0000","last_fullsized":"2024-03-23T15:41:18.655688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:56:51.521163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878671+0000","last_change":"2024-03-23T15:41:17.660898+0000","last_active":"2024-03-23T15:41:19.878671+0000","last_peered":"2024-03-23T15:41:19.878671+0000","last_clean":"2024-03-23T15:41:19.878671+0000","last_became_active":"2024-03-23T15:41:17.660515+0000","last_became_peered":"2024-03-23T15:41:17.660515+0000","last_unstale":"2024-03-23T15:41:19.878671+0000","last_undegraded":"2024-03-23T15:41:19.878671+0000","last_fullsized":"2024-03-23T15:41:19.878671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:58.911321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.656694+0000","last_change":"2024-03-23T15:41:18.653582+0000","last_active":"2024-03-23T15:41:18.656694+0000","last_peered":"2024-03-23T15:41:18.656694+0000","last_clean":"2024-03-23T15:41:18.656694+0000","last_became_active":"2024-03-23T15:41:18.653184+0000","last_became_peered":"2024-03-23T15:41:18.653184+0000","last_unstale":"2024-03-23T15:41:18.656694+0000","last_undegraded":"2024-03-23T15:41:18.656694+0000","last_fullsized":"2024-03-23T15:41:18.656694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:01:13.025017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878175+0000","last_change":"2024-03-23T15:41:17.651886+0000","last_active":"2024-03-23T15:41:19.878175+0000","last_peered":"2024-03-23T15:41:19.878175+0000","last_clean":"2024-03-23T15:41:19.878175+0000","last_became_active":"2024-03-23T15:41:17.651472+0000","last_became_peered":"2024-03-23T15:41:17.651472+0000","last_unstale":"2024-03-23T15:41:19.878175+0000","last_undegraded":"2024-03-23T15:41:19.878175+0000","last_fullsized":"2024-03-23T15:41:19.878175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:37.098333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878233+0000","last_change":"2024-03-23T15:41:18.644205+0000","last_active":"2024-03-23T15:41:19.878233+0000","last_peered":"2024-03-23T15:41:19.878233+0000","last_clean":"2024-03-23T15:41:19.878233+0000","last_became_active":"2024-03-23T15:41:18.643850+0000","last_became_peered":"2024-03-23T15:41:18.643850+0000","last_unstale":"2024-03-23T15:41:19.878233+0000","last_undegraded":"2024-03-23T15:41:19.878233+0000","last_fullsized":"2024-03-23T15:41:19.878233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:48:10.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.639456+0000","last_change":"2024-03-23T15:41:17.661051+0000","last_active":"2024-03-23T15:41:18.639456+0000","last_peered":"2024-03-23T15:41:18.639456+0000","last_clean":"2024-03-23T15:41:18.639456+0000","last_became_active":"2024-03-23T15:41:17.659673+0000","last_became_peered":"2024-03-23T15:41:17.659673+0000","last_unstale":"2024-03-23T15:41:18.639456+0000","last_undegraded":"2024-03-23T15:41:18.639456+0000","last_fullsized":"2024-03-23T15:41:18.639456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.028104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878885+0000","last_change":"2024-03-23T15:41:18.653630+0000","last_active":"2024-03-23T15:41:19.878885+0000","last_peered":"2024-03-23T15:41:19.878885+0000","last_clean":"2024-03-23T15:41:19.878885+0000","last_became_active":"2024-03-23T15:41:18.653119+0000","last_became_peered":"2024-03-23T15:41:18.653119+0000","last_unstale":"2024-03-23T15:41:19.878885+0000","last_undegraded":"2024-03-23T15:41:19.878885+0000","last_fullsized":"2024-03-23T15:41:19.878885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:02:49.635220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.655567+0000","last_change":"2024-03-23T15:41:18.653560+0000","last_active":"2024-03-23T15:41:18.655567+0000","last_peered":"2024-03-23T15:41:18.655567+0000","last_clean":"2024-03-23T15:41:18.655567+0000","last_became_active":"2024-03-23T15:41:18.653249+0000","last_became_peered":"2024-03-23T15:41:18.653249+0000","last_unstale":"2024-03-23T15:41:18.655567+0000","last_undegraded":"2024-03-23T15:41:18.655567+0000","last_fullsized":"2024-03-23T15:41:18.655567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:29:45.563866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631132+0000","last_change":"2024-03-23T15:41:17.656318+0000","last_active":"2024-03-23T15:41:19.631132+0000","last_peered":"2024-03-23T15:41:19.631132+0000","last_clean":"2024-03-23T15:41:19.631132+0000","last_became_active":"2024-03-23T15:41:17.655958+0000","last_became_peered":"2024-03-23T15:41:17.655958+0000","last_unstale":"2024-03-23T15:41:19.631132+0000","last_undegraded":"2024-03-23T15:41:19.631132+0000","last_fullsized":"2024-03-23T15:41:19.631132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:12.333504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.658705+0000","last_change":"2024-03-23T15:41:18.657616+0000","last_active":"2024-03-23T15:41:18.658705+0000","last_peered":"2024-03-23T15:41:18.658705+0000","last_clean":"2024-03-23T15:41:18.658705+0000","last_became_active":"2024-03-23T15:41:18.657271+0000","last_became_peered":"2024-03-23T15:41:18.657271+0000","last_unstale":"2024-03-23T15:41:18.658705+0000","last_undegraded":"2024-03-23T15:41:18.658705+0000","last_fullsized":"2024-03-23T15:41:18.658705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:56:40.412235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.644367+0000","last_change":"2024-03-23T15:41:17.654358+0000","last_active":"2024-03-23T15:41:18.644367+0000","last_peered":"2024-03-23T15:41:18.644367+0000","last_clean":"2024-03-23T15:41:18.644367+0000","last_became_active":"2024-03-23T15:41:17.653558+0000","last_became_peered":"2024-03-23T15:41:17.653558+0000","last_unstale":"2024-03-23T15:41:18.644367+0000","last_undegraded":"2024-03-23T15:41:18.644367+0000","last_fullsized":"2024-03-23T15:41:18.644367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:32.377883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.638786+0000","last_change":"2024-03-23T15:40:05.871801+0000","last_active":"2024-03-23T15:41:18.638786+0000","last_peered":"2024-03-23T15:41:18.638786+0000","last_clean":"2024-03-23T15:41:18.638786+0000","last_became_active":"2024-03-23T15:40:05.871417+0000","last_became_peered":"2024-03-23T15:40:05.871417+0000","last_unstale":"2024-03-23T15:41:18.638786+0000","last_undegraded":"2024-03-23T15:41:18.638786+0000","last_fullsized":"2024-03-23T15:41:18.638786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_clean_scrub_stamp":"2024-03-23T15:36:39.745753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:55.235176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631227+0000","last_change":"2024-03-23T15:41:18.652251+0000","last_active":"2024-03-23T15:41:19.631227+0000","last_peered":"2024-03-23T15:41:19.631227+0000","last_clean":"2024-03-23T15:41:19.631227+0000","last_became_active":"2024-03-23T15:41:18.651876+0000","last_became_peered":"2024-03-23T15:41:18.651876+0000","last_unstale":"2024-03-23T15:41:19.631227+0000","last_undegraded":"2024-03-23T15:41:19.631227+0000","last_fullsized":"2024-03-23T15:41:19.631227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:45.235243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.639014+0000","last_change":"2024-03-23T15:41:17.657338+0000","last_active":"2024-03-23T15:41:18.639014+0000","last_peered":"2024-03-23T15:41:18.639014+0000","last_clean":"2024-03-23T15:41:18.639014+0000","last_became_active":"2024-03-23T15:41:17.656991+0000","last_became_peered":"2024-03-23T15:41:17.656991+0000","last_unstale":"2024-03-23T15:41:18.639014+0000","last_undegraded":"2024-03-23T15:41:18.639014+0000","last_fullsized":"2024-03-23T15:41:18.639014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:23.694865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.689121+0000","last_change":"2024-03-23T15:41:17.660703+0000","last_active":"2024-03-23T15:41:20.689121+0000","last_peered":"2024-03-23T15:41:20.689121+0000","last_clean":"2024-03-23T15:41:20.689121+0000","last_became_active":"2024-03-23T15:41:17.660343+0000","last_became_peered":"2024-03-23T15:41:17.660343+0000","last_unstale":"2024-03-23T15:41:20.689121+0000","last_undegraded":"2024-03-23T15:41:20.689121+0000","last_fullsized":"2024-03-23T15:41:20.689121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:14:03.063397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631450+0000","last_change":"2024-03-23T15:41:18.644624+0000","last_active":"2024-03-23T15:41:19.631450+0000","last_peered":"2024-03-23T15:41:19.631450+0000","last_clean":"2024-03-23T15:41:19.631450+0000","last_became_active":"2024-03-23T15:41:18.644275+0000","last_became_peered":"2024-03-23T15:41:18.644275+0000","last_unstale":"2024-03-23T15:41:19.631450+0000","last_undegraded":"2024-03-23T15:41:19.631450+0000","last_fullsized":"2024-03-23T15:41:19.631450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:25.901935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.701083+0000","last_change":"2024-03-23T15:41:17.650068+0000","last_active":"2024-03-23T15:41:21.701083+0000","last_peered":"2024-03-23T15:41:21.701083+0000","last_clean":"2024-03-23T15:41:21.701083+0000","last_became_active":"2024-03-23T15:41:17.649331+0000","last_became_peered":"2024-03-23T15:41:17.649331+0000","last_unstale":"2024-03-23T15:41:21.701083+0000","last_undegraded":"2024-03-23T15:41:21.701083+0000","last_fullsized":"2024-03-23T15:41:21.701083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:47.578645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.657449+0000","last_change":"2024-03-23T15:41:18.656413+0000","last_active":"2024-03-23T15:41:18.657449+0000","last_peered":"2024-03-23T15:41:18.657449+0000","last_clean":"2024-03-23T15:41:18.657449+0000","last_became_active":"2024-03-23T15:41:18.656043+0000","last_became_peered":"2024-03-23T15:41:18.656043+0000","last_unstale":"2024-03-23T15:41:18.657449+0000","last_undegraded":"2024-03-23T15:41:18.657449+0000","last_fullsized":"2024-03-23T15:41:18.657449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:11:28.952997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.670043+0000","last_change":"2024-03-23T15:41:17.654492+0000","last_active":"2024-03-23T15:41:18.670043+0000","last_peered":"2024-03-23T15:41:18.670043+0000","last_clean":"2024-03-23T15:41:18.670043+0000","last_became_active":"2024-03-23T15:41:17.654132+0000","last_became_peered":"2024-03-23T15:41:17.654132+0000","last_unstale":"2024-03-23T15:41:18.670043+0000","last_undegraded":"2024-03-23T15:41:18.670043+0000","last_fullsized":"2024-03-23T15:41:18.670043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:39:58.210703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879037+0000","last_change":"2024-03-23T15:41:18.652659+0000","last_active":"2024-03-23T15:41:19.879037+0000","last_peered":"2024-03-23T15:41:19.879037+0000","last_clean":"2024-03-23T15:41:19.879037+0000","last_became_active":"2024-03-23T15:41:18.652246+0000","last_became_peered":"2024-03-23T15:41:18.652246+0000","last_unstale":"2024-03-23T15:41:19.879037+0000","last_undegraded":"2024-03-23T15:41:19.879037+0000","last_fullsized":"2024-03-23T15:41:19.879037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:50.868694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634413+0000","last_change":"2024-03-23T15:41:17.656928+0000","last_active":"2024-03-23T15:41:19.634413+0000","last_peered":"2024-03-23T15:41:19.634413+0000","last_clean":"2024-03-23T15:41:19.634413+0000","last_became_active":"2024-03-23T15:41:17.656478+0000","last_became_peered":"2024-03-23T15:41:17.656478+0000","last_unstale":"2024-03-23T15:41:19.634413+0000","last_undegraded":"2024-03-23T15:41:19.634413+0000","last_fullsized":"2024-03-23T15:41:19.634413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:14:46.716508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878350+0000","last_change":"2024-03-23T15:41:18.654049+0000","last_active":"2024-03-23T15:41:19.878350+0000","last_peered":"2024-03-23T15:41:19.878350+0000","last_clean":"2024-03-23T15:41:19.878350+0000","last_became_active":"2024-03-23T15:41:18.653765+0000","last_became_peered":"2024-03-23T15:41:18.653765+0000","last_unstale":"2024-03-23T15:41:19.878350+0000","last_undegraded":"2024-03-23T15:41:19.878350+0000","last_fullsized":"2024-03-23T15:41:19.878350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:12:11.467744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.636242+0000","last_change":"2024-03-23T15:41:17.656679+0000","last_active":"2024-03-23T15:41:18.636242+0000","last_peered":"2024-03-23T15:41:18.636242+0000","last_clean":"2024-03-23T15:41:18.636242+0000","last_became_active":"2024-03-23T15:41:17.656368+0000","last_became_peered":"2024-03-23T15:41:17.656368+0000","last_unstale":"2024-03-23T15:41:18.636242+0000","last_undegraded":"2024-03-23T15:41:18.636242+0000","last_fullsized":"2024-03-23T15:41:18.636242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:14.176288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.656818+0000","last_change":"2024-03-23T15:41:18.653637+0000","last_active":"2024-03-23T15:41:18.656818+0000","last_peered":"2024-03-23T15:41:18.656818+0000","last_clean":"2024-03-23T15:41:18.656818+0000","last_became_active":"2024-03-23T15:41:18.652864+0000","last_became_peered":"2024-03-23T15:41:18.652864+0000","last_unstale":"2024-03-23T15:41:18.656818+0000","last_undegraded":"2024-03-23T15:41:18.656818+0000","last_fullsized":"2024-03-23T15:41:18.656818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:14.198057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632185+0000","last_change":"2024-03-23T15:41:17.664417+0000","last_active":"2024-03-23T15:41:19.632185+0000","last_peered":"2024-03-23T15:41:19.632185+0000","last_clean":"2024-03-23T15:41:19.632185+0000","last_became_active":"2024-03-23T15:41:17.663454+0000","last_became_peered":"2024-03-23T15:41:17.663454+0000","last_unstale":"2024-03-23T15:41:19.632185+0000","last_undegraded":"2024-03-23T15:41:19.632185+0000","last_fullsized":"2024-03-23T15:41:19.632185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:14.947767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.649298+0000","last_change":"2024-03-23T15:41:18.647857+0000","last_active":"2024-03-23T15:41:18.649298+0000","last_peered":"2024-03-23T15:41:18.649298+0000","last_clean":"2024-03-23T15:41:18.649298+0000","last_became_active":"2024-03-23T15:41:18.646924+0000","last_became_peered":"2024-03-23T15:41:18.646924+0000","last_unstale":"2024-03-23T15:41:18.649298+0000","last_undegraded":"2024-03-23T15:41:18.649298+0000","last_fullsized":"2024-03-23T15:41:18.649298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:43:13.803086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.688714+0000","last_change":"2024-03-23T15:41:17.661995+0000","last_active":"2024-03-23T15:41:20.688714+0000","last_peered":"2024-03-23T15:41:20.688714+0000","last_clean":"2024-03-23T15:41:20.688714+0000","last_became_active":"2024-03-23T15:41:17.661632+0000","last_became_peered":"2024-03-23T15:41:17.661632+0000","last_unstale":"2024-03-23T15:41:20.688714+0000","last_undegraded":"2024-03-23T15:41:20.688714+0000","last_fullsized":"2024-03-23T15:41:20.688714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:12.540038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.654329+0000","last_change":"2024-03-23T15:41:18.651258+0000","last_active":"2024-03-23T15:41:18.654329+0000","last_peered":"2024-03-23T15:41:18.654329+0000","last_clean":"2024-03-23T15:41:18.654329+0000","last_became_active":"2024-03-23T15:41:18.650910+0000","last_became_peered":"2024-03-23T15:41:18.650910+0000","last_unstale":"2024-03-23T15:41:18.654329+0000","last_undegraded":"2024-03-23T15:41:18.654329+0000","last_fullsized":"2024-03-23T15:41:18.654329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:39:09.524046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632681+0000","last_change":"2024-03-23T15:41:17.654232+0000","last_active":"2024-03-23T15:41:19.632681+0000","last_peered":"2024-03-23T15:41:19.632681+0000","last_clean":"2024-03-23T15:41:19.632681+0000","last_became_active":"2024-03-23T15:41:17.653832+0000","last_became_peered":"2024-03-23T15:41:17.653832+0000","last_unstale":"2024-03-23T15:41:19.632681+0000","last_undegraded":"2024-03-23T15:41:19.632681+0000","last_fullsized":"2024-03-23T15:41:19.632681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:05.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.664942+0000","last_change":"2024-03-23T15:41:18.664444+0000","last_active":"2024-03-23T15:41:18.664942+0000","last_peered":"2024-03-23T15:41:18.664942+0000","last_clean":"2024-03-23T15:41:18.664942+0000","last_became_active":"2024-03-23T15:41:18.664127+0000","last_became_peered":"2024-03-23T15:41:18.664127+0000","last_unstale":"2024-03-23T15:41:18.664942+0000","last_undegraded":"2024-03-23T15:41:18.664942+0000","last_fullsized":"2024-03-23T15:41:18.664942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:40:47.182831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.667907+0000","last_change":"2024-03-23T15:41:17.664834+0000","last_active":"2024-03-23T15:41:18.667907+0000","last_peered":"2024-03-23T15:41:18.667907+0000","last_clean":"2024-03-23T15:41:18.667907+0000","last_became_active":"2024-03-23T15:41:17.664286+0000","last_became_peered":"2024-03-23T15:41:17.664286+0000","last_unstale":"2024-03-23T15:41:18.667907+0000","last_undegraded":"2024-03-23T15:41:18.667907+0000","last_fullsized":"2024-03-23T15:41:18.667907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:50.595420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.654075+0000","last_change":"2024-03-23T15:41:18.646341+0000","last_active":"2024-03-23T15:41:18.654075+0000","last_peered":"2024-03-23T15:41:18.654075+0000","last_clean":"2024-03-23T15:41:18.654075+0000","last_became_active":"2024-03-23T15:41:18.645995+0000","last_became_peered":"2024-03-23T15:41:18.645995+0000","last_unstale":"2024-03-23T15:41:18.654075+0000","last_undegraded":"2024-03-23T15:41:18.654075+0000","last_fullsized":"2024-03-23T15:41:18.654075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:01:35.846423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631339+0000","last_change":"2024-03-23T15:41:17.656637+0000","last_active":"2024-03-23T15:41:19.631339+0000","last_peered":"2024-03-23T15:41:19.631339+0000","last_clean":"2024-03-23T15:41:19.631339+0000","last_became_active":"2024-03-23T15:41:17.656271+0000","last_became_peered":"2024-03-23T15:41:17.656271+0000","last_unstale":"2024-03-23T15:41:19.631339+0000","last_undegraded":"2024-03-23T15:41:19.631339+0000","last_fullsized":"2024-03-23T15:41:19.631339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:01:15.747209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.659212+0000","last_change":"2024-03-23T15:41:18.655250+0000","last_active":"2024-03-23T15:41:18.659212+0000","last_peered":"2024-03-23T15:41:18.659212+0000","last_clean":"2024-03-23T15:41:18.659212+0000","last_became_active":"2024-03-23T15:41:18.654888+0000","last_became_peered":"2024-03-23T15:41:18.654888+0000","last_unstale":"2024-03-23T15:41:18.659212+0000","last_undegraded":"2024-03-23T15:41:18.659212+0000","last_fullsized":"2024-03-23T15:41:18.659212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:13:46.326611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.656836+0000","last_change":"2024-03-23T15:41:18.654309+0000","last_active":"2024-03-23T15:41:18.656836+0000","last_peered":"2024-03-23T15:41:18.656836+0000","last_clean":"2024-03-23T15:41:18.656836+0000","last_became_active":"2024-03-23T15:41:18.653319+0000","last_became_peered":"2024-03-23T15:41:18.653319+0000","last_unstale":"2024-03-23T15:41:18.656836+0000","last_undegraded":"2024-03-23T15:41:18.656836+0000","last_fullsized":"2024-03-23T15:41:18.656836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:10:05.748046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632334+0000","last_change":"2024-03-23T15:41:18.655179+0000","last_active":"2024-03-23T15:41:19.632334+0000","last_peered":"2024-03-23T15:41:19.632334+0000","last_clean":"2024-03-23T15:41:19.632334+0000","last_became_active":"2024-03-23T15:41:18.654830+0000","last_became_peered":"2024-03-23T15:41:18.654830+0000","last_unstale":"2024-03-23T15:41:19.632334+0000","last_undegraded":"2024-03-23T15:41:19.632334+0000","last_fullsized":"2024-03-23T15:41:19.632334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:29.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632307+0000","last_change":"2024-03-23T15:41:18.650734+0000","last_active":"2024-03-23T15:41:19.632307+0000","last_peered":"2024-03-23T15:41:19.632307+0000","last_clean":"2024-03-23T15:41:19.632307+0000","last_became_active":"2024-03-23T15:41:18.649993+0000","last_became_peered":"2024-03-23T15:41:18.649993+0000","last_unstale":"2024-03-23T15:41:19.632307+0000","last_undegraded":"2024-03-23T15:41:19.632307+0000","last_fullsized":"2024-03-23T15:41:19.632307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:06.463525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878316+0000","last_change":"2024-03-23T15:41:18.655741+0000","last_active":"2024-03-23T15:41:19.878316+0000","last_peered":"2024-03-23T15:41:19.878316+0000","last_clean":"2024-03-23T15:41:19.878316+0000","last_became_active":"2024-03-23T15:41:18.653387+0000","last_became_peered":"2024-03-23T15:41:18.653387+0000","last_unstale":"2024-03-23T15:41:19.878316+0000","last_undegraded":"2024-03-23T15:41:19.878316+0000","last_fullsized":"2024-03-23T15:41:19.878316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:20:26.758549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631518+0000","last_change":"2024-03-23T15:41:18.654739+0000","last_active":"2024-03-23T15:41:19.631518+0000","last_peered":"2024-03-23T15:41:19.631518+0000","last_clean":"2024-03-23T15:41:19.631518+0000","last_became_active":"2024-03-23T15:41:18.654151+0000","last_became_peered":"2024-03-23T15:41:18.654151+0000","last_unstale":"2024-03-23T15:41:19.631518+0000","last_undegraded":"2024-03-23T15:41:19.631518+0000","last_fullsized":"2024-03-23T15:41:19.631518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:01.358959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.656782+0000","last_change":"2024-03-23T15:41:18.654081+0000","last_active":"2024-03-23T15:41:18.656782+0000","last_peered":"2024-03-23T15:41:18.656782+0000","last_clean":"2024-03-23T15:41:18.656782+0000","last_became_active":"2024-03-23T15:41:18.653696+0000","last_became_peered":"2024-03-23T15:41:18.653696+0000","last_unstale":"2024-03-23T15:41:18.656782+0000","last_undegraded":"2024-03-23T15:41:18.656782+0000","last_fullsized":"2024-03-23T15:41:18.656782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:25.209294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632755+0000","last_change":"2024-03-23T15:41:18.652356+0000","last_active":"2024-03-23T15:41:19.632755+0000","last_peered":"2024-03-23T15:41:19.632755+0000","last_clean":"2024-03-23T15:41:19.632755+0000","last_became_active":"2024-03-23T15:41:18.651976+0000","last_became_peered":"2024-03-23T15:41:18.651976+0000","last_unstale":"2024-03-23T15:41:19.632755+0000","last_undegraded":"2024-03-23T15:41:19.632755+0000","last_fullsized":"2024-03-23T15:41:19.632755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:38.972186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.654939+0000","last_change":"2024-03-23T15:41:18.653775+0000","last_active":"2024-03-23T15:41:18.654939+0000","last_peered":"2024-03-23T15:41:18.654939+0000","last_clean":"2024-03-23T15:41:18.654939+0000","last_became_active":"2024-03-23T15:41:18.653404+0000","last_became_peered":"2024-03-23T15:41:18.653404+0000","last_unstale":"2024-03-23T15:41:18.654939+0000","last_undegraded":"2024-03-23T15:41:18.654939+0000","last_fullsized":"2024-03-23T15:41:18.654939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:44:06.044922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.647241+0000","last_change":"2024-03-23T15:41:18.646098+0000","last_active":"2024-03-23T15:41:18.647241+0000","last_peered":"2024-03-23T15:41:18.647241+0000","last_clean":"2024-03-23T15:41:18.647241+0000","last_became_active":"2024-03-23T15:41:18.645815+0000","last_became_peered":"2024-03-23T15:41:18.645815+0000","last_unstale":"2024-03-23T15:41:18.647241+0000","last_undegraded":"2024-03-23T15:41:18.647241+0000","last_fullsized":"2024-03-23T15:41:18.647241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:03.896521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.654874+0000","last_change":"2024-03-23T15:41:18.652487+0000","last_active":"2024-03-23T15:41:18.654874+0000","last_peered":"2024-03-23T15:41:18.654874+0000","last_clean":"2024-03-23T15:41:18.654874+0000","last_became_active":"2024-03-23T15:41:18.652151+0000","last_became_peered":"2024-03-23T15:41:18.652151+0000","last_unstale":"2024-03-23T15:41:18.654874+0000","last_undegraded":"2024-03-23T15:41:18.654874+0000","last_fullsized":"2024-03-23T15:41:18.654874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:37:19.755844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634597+0000","last_change":"2024-03-23T15:41:18.657265+0000","last_active":"2024-03-23T15:41:19.634597+0000","last_peered":"2024-03-23T15:41:19.634597+0000","last_clean":"2024-03-23T15:41:19.634597+0000","last_became_active":"2024-03-23T15:41:18.656970+0000","last_became_peered":"2024-03-23T15:41:18.656970+0000","last_unstale":"2024-03-23T15:41:19.634597+0000","last_undegraded":"2024-03-23T15:41:19.634597+0000","last_fullsized":"2024-03-23T15:41:19.634597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:22.808034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631480+0000","last_change":"2024-03-23T15:41:18.655254+0000","last_active":"2024-03-23T15:41:19.631480+0000","last_peered":"2024-03-23T15:41:19.631480+0000","last_clean":"2024-03-23T15:41:19.631480+0000","last_became_active":"2024-03-23T15:41:18.654879+0000","last_became_peered":"2024-03-23T15:41:18.654879+0000","last_unstale":"2024-03-23T15:41:19.631480+0000","last_undegraded":"2024-03-23T15:41:19.631480+0000","last_fullsized":"2024-03-23T15:41:19.631480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:36:18.923723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-23T15:41:18.660143+0000","last_change":"2024-03-23T15:41:18.658652+0000","last_active":"2024-03-23T15:41:18.660143+0000","last_peered":"2024-03-23T15:41:18.660143+0000","last_clean":"2024-03-23T15:41:18.660143+0000","last_became_active":"2024-03-23T15:41:18.658261+0000","last_became_peered":"2024-03-23T15:41:18.658261+0000","last_unstale":"2024-03-23T15:41:18.660143+0000","last_undegraded":"2024-03-23T15:41:18.660143+0000","last_fullsized":"2024-03-23T15:41:18.660143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:42.038362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245950,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2522,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":71,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":9476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":1020901,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":913408,"data_stored":431563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":431597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":429973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":1020319,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T15:41:25.403 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T15:41:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.539+0000 7ff870e8d640 1 -- 172.21.15.76:0/842232277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c1021e0 msgr2=0x7ff86c108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.539+0000 7ff870e8d640 1 --2- 172.21.15.76:0/842232277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c1021e0 0x7ff86c108a70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff864009860 tx=0x7ff86402bb70 comp rx=0 tx=0).stop 2024-03-23T15:41:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- 172.21.15.76:0/842232277 shutdown_connections 2024-03-23T15:41:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 --2- 172.21.15.76:0/842232277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c1021e0 0x7ff86c108a70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 --2- 172.21.15.76:0/842232277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c101820 0x7ff86c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 --2- 172.21.15.76:0/842232277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c100630 0x7ff86c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- 172.21.15.76:0/842232277 >> 172.21.15.76:0/842232277 conn(0x7ff86c0fbba0 msgr2=0x7ff86c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- 172.21.15.76:0/842232277 shutdown_connections 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- 172.21.15.76:0/842232277 wait complete. 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 Processor -- start 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- start start 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100630 0x7ff86c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c101820 0x7ff86c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1021e0 0x7ff86c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c19f2a0 con 0x7ff86c1021e0 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c19f410 con 0x7ff86c101820 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.540+0000 7ff870e8d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c19f5d0 con 0x7ff86c100630 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1021e0 0x7ff86c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1021e0 0x7ff86c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36914/0 (socket says 172.21.15.76:36914) 2024-03-23T15:41:25.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff869d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c101820 0x7ff86c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 -- 172.21.15.76:0/1008433092 learned_addr learned my addr 172.21.15.76:0/1008433092 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86a59c640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100630 0x7ff86c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 -- 172.21.15.76:0/1008433092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100630 msgr2=0x7ff86c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100630 0x7ff86c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 -- 172.21.15.76:0/1008433092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c101820 msgr2=0x7ff86c19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c101820 0x7ff86c19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 -- 172.21.15.76:0/1008433092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff860009560 con 0x7ff86c1021e0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff86ad9d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1021e0 0x7ff86c19eb10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8640057c0 tx=0x7ff864035f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1008433092 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8640348d0 con 0x7ff86c1021e0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff864009490 con 0x7ff86c1021e0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1008433092 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86402c9f0 con 0x7ff86c1021e0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1008433092 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864032600 con 0x7ff86c1021e0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.541+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c19fa80 con 0x7ff86c1021e0 2024-03-23T15:41:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.542+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86c1078f0 con 0x7ff86c1021e0 2024-03-23T15:41:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.543+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1008433092 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7ff86403b040 con 0x7ff86c1021e0 2024-03-23T15:41:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.544+0000 7ff85b7fe640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8540a2150 0x7ff8540a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.544+0000 7ff86a59c640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8540a2150 0x7ff8540a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.544+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1008433092 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff8640efa60 con 0x7ff86c1021e0 2024-03-23T15:41:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.544+0000 7ff86a59c640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8540a2150 0x7ff8540a4610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff86c101680 tx=0x7ff85c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.547+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1008433092 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff8640b8cd0 con 0x7ff86c1021e0 2024-03-23T15:41:25.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.723+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff86c02d240 con 0x7ff8540a2150 2024-03-23T15:41:25.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.730+0000 7ff85b7fe640 1 -- 172.21.15.76:0/1008433092 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258943 (secure 0 0 0) 0x7ff86c02d240 con 0x7ff8540a2150 2024-03-23T15:41:25.735 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:25.736 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8540a2150 msgr2=0x7ff8540a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8540a2150 0x7ff8540a4610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff86c101680 tx=0x7ff85c009500 comp rx=0 tx=0).stop 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1021e0 msgr2=0x7ff86c19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1021e0 0x7ff86c19eb10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8640057c0 tx=0x7ff864035f20 comp rx=0 tx=0).stop 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 shutdown_connections 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1021e0 0x7ff86c19eb10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8540a2150 0x7ff8540a4610 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c101820 0x7ff86c19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 --2- 172.21.15.76:0/1008433092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100630 0x7ff86c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:25.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 >> 172.21.15.76:0/1008433092 conn(0x7ff86c0fbba0 msgr2=0x7ff86c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:25.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 shutdown_connections 2024-03-23T15:41:25.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:25.734+0000 7ff870e8d640 1 -- 172.21.15.76:0/1008433092 wait complete. 2024-03-23T15:41:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-03-23T15:41:25.476853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595986,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":161,"num_read_kb":139,"num_write":300,"num_write_kb":2970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368624,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573200,"statfs":{"total":1151940427776,"available":1151562956800,"internally_reserved":0,"allocated":11403264,"data_stored":7122695,"data_compressed":110550,"data_compressed_allocated":143360,"data_compressed_original":1785856,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5618,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":25,"num_read_kb":20,"num_write":119,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.838603"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.629598+0000","last_change":"2024-03-23T15:41:18.651731+0000","last_active":"2024-03-23T15:41:20.629598+0000","last_peered":"2024-03-23T15:41:20.629598+0000","last_clean":"2024-03-23T15:41:20.629598+0000","last_became_active":"2024-03-23T15:41:18.651312+0000","last_became_peered":"2024-03-23T15:41:18.651312+0000","last_unstale":"2024-03-23T15:41:20.629598+0000","last_undegraded":"2024-03-23T15:41:20.629598+0000","last_fullsized":"2024-03-23T15:41:20.629598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:39:20.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879191+0000","last_change":"2024-03-23T15:41:18.655736+0000","last_active":"2024-03-23T15:41:19.879191+0000","last_peered":"2024-03-23T15:41:19.879191+0000","last_clean":"2024-03-23T15:41:19.879191+0000","last_became_active":"2024-03-23T15:41:18.655389+0000","last_became_peered":"2024-03-23T15:41:18.655389+0000","last_unstale":"2024-03-23T15:41:19.879191+0000","last_undegraded":"2024-03-23T15:41:19.879191+0000","last_fullsized":"2024-03-23T15:41:19.879191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:35:56.260056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878534+0000","last_change":"2024-03-23T15:41:18.650709+0000","last_active":"2024-03-23T15:41:19.878534+0000","last_peered":"2024-03-23T15:41:19.878534+0000","last_clean":"2024-03-23T15:41:19.878534+0000","last_became_active":"2024-03-23T15:41:18.650316+0000","last_became_peered":"2024-03-23T15:41:18.650316+0000","last_unstale":"2024-03-23T15:41:19.878534+0000","last_undegraded":"2024-03-23T15:41:19.878534+0000","last_fullsized":"2024-03-23T15:41:19.878534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:50:57.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878482+0000","last_change":"2024-03-23T15:41:18.652216+0000","last_active":"2024-03-23T15:41:19.878482+0000","last_peered":"2024-03-23T15:41:19.878482+0000","last_clean":"2024-03-23T15:41:19.878482+0000","last_became_active":"2024-03-23T15:41:18.650525+0000","last_became_peered":"2024-03-23T15:41:18.650525+0000","last_unstale":"2024-03-23T15:41:19.878482+0000","last_undegraded":"2024-03-23T15:41:19.878482+0000","last_fullsized":"2024-03-23T15:41:19.878482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:16.458257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878504+0000","last_change":"2024-03-23T15:41:18.649146+0000","last_active":"2024-03-23T15:41:19.878504+0000","last_peered":"2024-03-23T15:41:19.878504+0000","last_clean":"2024-03-23T15:41:19.878504+0000","last_became_active":"2024-03-23T15:41:18.648728+0000","last_became_peered":"2024-03-23T15:41:18.648728+0000","last_unstale":"2024-03-23T15:41:19.878504+0000","last_undegraded":"2024-03-23T15:41:19.878504+0000","last_fullsized":"2024-03-23T15:41:19.878504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:56.251250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879291+0000","last_change":"2024-03-23T15:41:18.652957+0000","last_active":"2024-03-23T15:41:19.879291+0000","last_peered":"2024-03-23T15:41:19.879291+0000","last_clean":"2024-03-23T15:41:19.879291+0000","last_became_active":"2024-03-23T15:41:18.652578+0000","last_became_peered":"2024-03-23T15:41:18.652578+0000","last_unstale":"2024-03-23T15:41:19.879291+0000","last_undegraded":"2024-03-23T15:41:19.879291+0000","last_fullsized":"2024-03-23T15:41:19.879291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:46:38.961554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877920+0000","last_change":"2024-03-23T15:41:18.646134+0000","last_active":"2024-03-23T15:41:19.877920+0000","last_peered":"2024-03-23T15:41:19.877920+0000","last_clean":"2024-03-23T15:41:19.877920+0000","last_became_active":"2024-03-23T15:41:18.645680+0000","last_became_peered":"2024-03-23T15:41:18.645680+0000","last_unstale":"2024-03-23T15:41:19.877920+0000","last_undegraded":"2024-03-23T15:41:19.877920+0000","last_fullsized":"2024-03-23T15:41:19.877920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:34:31.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877558+0000","last_change":"2024-03-23T15:41:18.646143+0000","last_active":"2024-03-23T15:41:19.877558+0000","last_peered":"2024-03-23T15:41:19.877558+0000","last_clean":"2024-03-23T15:41:19.877558+0000","last_became_active":"2024-03-23T15:41:18.645771+0000","last_became_peered":"2024-03-23T15:41:18.645771+0000","last_unstale":"2024-03-23T15:41:19.877558+0000","last_undegraded":"2024-03-23T15:41:19.877558+0000","last_fullsized":"2024-03-23T15:41:19.877558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:39.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.635038+0000","last_change":"2024-03-23T15:41:18.665083+0000","last_active":"2024-03-23T15:41:19.635038+0000","last_peered":"2024-03-23T15:41:19.635038+0000","last_clean":"2024-03-23T15:41:19.635038+0000","last_became_active":"2024-03-23T15:41:18.663143+0000","last_became_peered":"2024-03-23T15:41:18.663143+0000","last_unstale":"2024-03-23T15:41:19.635038+0000","last_undegraded":"2024-03-23T15:41:19.635038+0000","last_fullsized":"2024-03-23T15:41:19.635038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:52:23.421645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879109+0000","last_change":"2024-03-23T15:41:18.650014+0000","last_active":"2024-03-23T15:41:19.879109+0000","last_peered":"2024-03-23T15:41:19.879109+0000","last_clean":"2024-03-23T15:41:19.879109+0000","last_became_active":"2024-03-23T15:41:18.649673+0000","last_became_peered":"2024-03-23T15:41:18.649673+0000","last_unstale":"2024-03-23T15:41:19.879109+0000","last_undegraded":"2024-03-23T15:41:19.879109+0000","last_fullsized":"2024-03-23T15:41:19.879109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:30.387993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878458+0000","last_change":"2024-03-23T15:41:18.665622+0000","last_active":"2024-03-23T15:41:19.878458+0000","last_peered":"2024-03-23T15:41:19.878458+0000","last_clean":"2024-03-23T15:41:19.878458+0000","last_became_active":"2024-03-23T15:41:18.665274+0000","last_became_peered":"2024-03-23T15:41:18.665274+0000","last_unstale":"2024-03-23T15:41:19.878458+0000","last_undegraded":"2024-03-23T15:41:19.878458+0000","last_fullsized":"2024-03-23T15:41:19.878458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:12:18.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878310+0000","last_change":"2024-03-23T15:41:18.650888+0000","last_active":"2024-03-23T15:41:19.878310+0000","last_peered":"2024-03-23T15:41:19.878310+0000","last_clean":"2024-03-23T15:41:19.878310+0000","last_became_active":"2024-03-23T15:41:18.650363+0000","last_became_peered":"2024-03-23T15:41:18.650363+0000","last_unstale":"2024-03-23T15:41:19.878310+0000","last_undegraded":"2024-03-23T15:41:19.878310+0000","last_fullsized":"2024-03-23T15:41:19.878310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:41.289697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631546+0000","last_change":"2024-03-23T15:41:18.634183+0000","last_active":"2024-03-23T15:41:19.631546+0000","last_peered":"2024-03-23T15:41:19.631546+0000","last_clean":"2024-03-23T15:41:19.631546+0000","last_became_active":"2024-03-23T15:41:18.633849+0000","last_became_peered":"2024-03-23T15:41:18.633849+0000","last_unstale":"2024-03-23T15:41:19.631546+0000","last_undegraded":"2024-03-23T15:41:19.631546+0000","last_fullsized":"2024-03-23T15:41:19.631546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:13.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877999+0000","last_change":"2024-03-23T15:41:18.645269+0000","last_active":"2024-03-23T15:41:19.877999+0000","last_peered":"2024-03-23T15:41:19.877999+0000","last_clean":"2024-03-23T15:41:19.877999+0000","last_became_active":"2024-03-23T15:41:18.644940+0000","last_became_peered":"2024-03-23T15:41:18.644940+0000","last_unstale":"2024-03-23T15:41:19.877999+0000","last_undegraded":"2024-03-23T15:41:19.877999+0000","last_fullsized":"2024-03-23T15:41:19.877999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:26.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877820+0000","last_change":"2024-03-23T15:41:18.652789+0000","last_active":"2024-03-23T15:41:19.877820+0000","last_peered":"2024-03-23T15:41:19.877820+0000","last_clean":"2024-03-23T15:41:19.877820+0000","last_became_active":"2024-03-23T15:41:18.651520+0000","last_became_peered":"2024-03-23T15:41:18.651520+0000","last_unstale":"2024-03-23T15:41:19.877820+0000","last_undegraded":"2024-03-23T15:41:19.877820+0000","last_fullsized":"2024-03-23T15:41:19.877820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:00.388217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634507+0000","last_change":"2024-03-23T15:41:18.660398+0000","last_active":"2024-03-23T15:41:19.634507+0000","last_peered":"2024-03-23T15:41:19.634507+0000","last_clean":"2024-03-23T15:41:19.634507+0000","last_became_active":"2024-03-23T15:41:18.660084+0000","last_became_peered":"2024-03-23T15:41:18.660084+0000","last_unstale":"2024-03-23T15:41:19.634507+0000","last_undegraded":"2024-03-23T15:41:19.634507+0000","last_fullsized":"2024-03-23T15:41:19.634507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:04:33.115173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.629688+0000","last_change":"2024-03-23T15:41:18.663192+0000","last_active":"2024-03-23T15:41:20.629688+0000","last_peered":"2024-03-23T15:41:20.629688+0000","last_clean":"2024-03-23T15:41:20.629688+0000","last_became_active":"2024-03-23T15:41:18.662844+0000","last_became_peered":"2024-03-23T15:41:18.662844+0000","last_unstale":"2024-03-23T15:41:20.629688+0000","last_undegraded":"2024-03-23T15:41:20.629688+0000","last_fullsized":"2024-03-23T15:41:20.629688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:24.260706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878908+0000","last_change":"2024-03-23T15:41:18.646405+0000","last_active":"2024-03-23T15:41:19.878908+0000","last_peered":"2024-03-23T15:41:19.878908+0000","last_clean":"2024-03-23T15:41:19.878908+0000","last_became_active":"2024-03-23T15:41:18.645516+0000","last_became_peered":"2024-03-23T15:41:18.645516+0000","last_unstale":"2024-03-23T15:41:19.878908+0000","last_undegraded":"2024-03-23T15:41:19.878908+0000","last_fullsized":"2024-03-23T15:41:19.878908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:29:18.588190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632579+0000","last_change":"2024-03-23T15:41:18.652344+0000","last_active":"2024-03-23T15:41:19.632579+0000","last_peered":"2024-03-23T15:41:19.632579+0000","last_clean":"2024-03-23T15:41:19.632579+0000","last_became_active":"2024-03-23T15:41:18.650985+0000","last_became_peered":"2024-03-23T15:41:18.650985+0000","last_unstale":"2024-03-23T15:41:19.632579+0000","last_undegraded":"2024-03-23T15:41:19.632579+0000","last_fullsized":"2024-03-23T15:41:19.632579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:17:40.741299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.701617+0000","last_change":"2024-03-23T15:41:17.655443+0000","last_active":"2024-03-23T15:41:21.701617+0000","last_peered":"2024-03-23T15:41:21.701617+0000","last_clean":"2024-03-23T15:41:21.701617+0000","last_became_active":"2024-03-23T15:41:17.654834+0000","last_became_peered":"2024-03-23T15:41:17.654834+0000","last_unstale":"2024-03-23T15:41:21.701617+0000","last_undegraded":"2024-03-23T15:41:21.701617+0000","last_fullsized":"2024-03-23T15:41:21.701617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:49:43.215644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878394+0000","last_change":"2024-03-23T15:41:18.650241+0000","last_active":"2024-03-23T15:41:19.878394+0000","last_peered":"2024-03-23T15:41:19.878394+0000","last_clean":"2024-03-23T15:41:19.878394+0000","last_became_active":"2024-03-23T15:41:18.649914+0000","last_became_peered":"2024-03-23T15:41:18.649914+0000","last_unstale":"2024-03-23T15:41:19.878394+0000","last_undegraded":"2024-03-23T15:41:19.878394+0000","last_fullsized":"2024-03-23T15:41:19.878394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:35:18.892649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878452+0000","last_change":"2024-03-23T15:41:17.663308+0000","last_active":"2024-03-23T15:41:19.878452+0000","last_peered":"2024-03-23T15:41:19.878452+0000","last_clean":"2024-03-23T15:41:19.878452+0000","last_became_active":"2024-03-23T15:41:17.662958+0000","last_became_peered":"2024-03-23T15:41:17.662958+0000","last_unstale":"2024-03-23T15:41:19.878452+0000","last_undegraded":"2024-03-23T15:41:19.878452+0000","last_fullsized":"2024-03-23T15:41:19.878452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:43:37.512893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878962+0000","last_change":"2024-03-23T15:41:18.651225+0000","last_active":"2024-03-23T15:41:19.878962+0000","last_peered":"2024-03-23T15:41:19.878962+0000","last_clean":"2024-03-23T15:41:19.878962+0000","last_became_active":"2024-03-23T15:41:18.650884+0000","last_became_peered":"2024-03-23T15:41:18.650884+0000","last_unstale":"2024-03-23T15:41:19.878962+0000","last_undegraded":"2024-03-23T15:41:19.878962+0000","last_fullsized":"2024-03-23T15:41:19.878962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:00.445203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879181+0000","last_change":"2024-03-23T15:41:17.662753+0000","last_active":"2024-03-23T15:41:19.879181+0000","last_peered":"2024-03-23T15:41:19.879181+0000","last_clean":"2024-03-23T15:41:19.879181+0000","last_became_active":"2024-03-23T15:41:17.662378+0000","last_became_peered":"2024-03-23T15:41:17.662378+0000","last_unstale":"2024-03-23T15:41:19.879181+0000","last_undegraded":"2024-03-23T15:41:19.879181+0000","last_fullsized":"2024-03-23T15:41:19.879181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:58:55.674739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634632+0000","last_change":"2024-03-23T15:41:18.659171+0000","last_active":"2024-03-23T15:41:19.634632+0000","last_peered":"2024-03-23T15:41:19.634632+0000","last_clean":"2024-03-23T15:41:19.634632+0000","last_became_active":"2024-03-23T15:41:18.658836+0000","last_became_peered":"2024-03-23T15:41:18.658836+0000","last_unstale":"2024-03-23T15:41:19.634632+0000","last_undegraded":"2024-03-23T15:41:19.634632+0000","last_fullsized":"2024-03-23T15:41:19.634632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:15.056528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879027+0000","last_change":"2024-03-23T15:41:17.655842+0000","last_active":"2024-03-23T15:41:19.879027+0000","last_peered":"2024-03-23T15:41:19.879027+0000","last_clean":"2024-03-23T15:41:19.879027+0000","last_became_active":"2024-03-23T15:41:17.655452+0000","last_became_peered":"2024-03-23T15:41:17.655452+0000","last_unstale":"2024-03-23T15:41:19.879027+0000","last_undegraded":"2024-03-23T15:41:19.879027+0000","last_fullsized":"2024-03-23T15:41:19.879027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:27:50.518854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879286+0000","last_change":"2024-03-23T15:41:18.648500+0000","last_active":"2024-03-23T15:41:19.879286+0000","last_peered":"2024-03-23T15:41:19.879286+0000","last_clean":"2024-03-23T15:41:19.879286+0000","last_became_active":"2024-03-23T15:41:18.648139+0000","last_became_peered":"2024-03-23T15:41:18.648139+0000","last_unstale":"2024-03-23T15:41:19.879286+0000","last_undegraded":"2024-03-23T15:41:19.879286+0000","last_fullsized":"2024-03-23T15:41:19.879286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:38.071057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:23.704787+0000","last_change":"2024-03-23T15:41:17.658471+0000","last_active":"2024-03-23T15:41:23.704787+0000","last_peered":"2024-03-23T15:41:23.704787+0000","last_clean":"2024-03-23T15:41:23.704787+0000","last_became_active":"2024-03-23T15:41:17.658142+0000","last_became_peered":"2024-03-23T15:41:17.658142+0000","last_unstale":"2024-03-23T15:41:23.704787+0000","last_undegraded":"2024-03-23T15:41:23.704787+0000","last_fullsized":"2024-03-23T15:41:23.704787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:23.224790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878537+0000","last_change":"2024-03-23T15:41:18.646046+0000","last_active":"2024-03-23T15:41:19.878537+0000","last_peered":"2024-03-23T15:41:19.878537+0000","last_clean":"2024-03-23T15:41:19.878537+0000","last_became_active":"2024-03-23T15:41:18.645727+0000","last_became_peered":"2024-03-23T15:41:18.645727+0000","last_unstale":"2024-03-23T15:41:19.878537+0000","last_undegraded":"2024-03-23T15:41:19.878537+0000","last_fullsized":"2024-03-23T15:41:19.878537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:55.552079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.699051+0000","last_change":"2024-03-23T15:41:17.657285+0000","last_active":"2024-03-23T15:41:21.699051+0000","last_peered":"2024-03-23T15:41:21.699051+0000","last_clean":"2024-03-23T15:41:21.699051+0000","last_became_active":"2024-03-23T15:41:17.656942+0000","last_became_peered":"2024-03-23T15:41:17.656942+0000","last_unstale":"2024-03-23T15:41:21.699051+0000","last_undegraded":"2024-03-23T15:41:21.699051+0000","last_fullsized":"2024-03-23T15:41:21.699051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:18.107467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878872+0000","last_change":"2024-03-23T15:41:18.655175+0000","last_active":"2024-03-23T15:41:19.878872+0000","last_peered":"2024-03-23T15:41:19.878872+0000","last_clean":"2024-03-23T15:41:19.878872+0000","last_became_active":"2024-03-23T15:41:18.654846+0000","last_became_peered":"2024-03-23T15:41:18.654846+0000","last_unstale":"2024-03-23T15:41:19.878872+0000","last_undegraded":"2024-03-23T15:41:19.878872+0000","last_fullsized":"2024-03-23T15:41:19.878872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:43:10.008538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.700154+0000","last_change":"2024-03-23T15:41:17.657391+0000","last_active":"2024-03-23T15:41:21.700154+0000","last_peered":"2024-03-23T15:41:21.700154+0000","last_clean":"2024-03-23T15:41:21.700154+0000","last_became_active":"2024-03-23T15:41:17.657081+0000","last_became_peered":"2024-03-23T15:41:17.657081+0000","last_unstale":"2024-03-23T15:41:21.700154+0000","last_undegraded":"2024-03-23T15:41:21.700154+0000","last_fullsized":"2024-03-23T15:41:21.700154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:08.744477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877484+0000","last_change":"2024-03-23T15:41:18.656268+0000","last_active":"2024-03-23T15:41:19.877484+0000","last_peered":"2024-03-23T15:41:19.877484+0000","last_clean":"2024-03-23T15:41:19.877484+0000","last_became_active":"2024-03-23T15:41:18.655910+0000","last_became_peered":"2024-03-23T15:41:18.655910+0000","last_unstale":"2024-03-23T15:41:19.877484+0000","last_undegraded":"2024-03-23T15:41:19.877484+0000","last_fullsized":"2024-03-23T15:41:19.877484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:38:57.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879058+0000","last_change":"2024-03-23T15:41:17.656555+0000","last_active":"2024-03-23T15:41:19.879058+0000","last_peered":"2024-03-23T15:41:19.879058+0000","last_clean":"2024-03-23T15:41:19.879058+0000","last_became_active":"2024-03-23T15:41:17.655732+0000","last_became_peered":"2024-03-23T15:41:17.655732+0000","last_unstale":"2024-03-23T15:41:19.879058+0000","last_undegraded":"2024-03-23T15:41:19.879058+0000","last_fullsized":"2024-03-23T15:41:19.879058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:53.839841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878572+0000","last_change":"2024-03-23T15:41:18.647334+0000","last_active":"2024-03-23T15:41:19.878572+0000","last_peered":"2024-03-23T15:41:19.878572+0000","last_clean":"2024-03-23T15:41:19.878572+0000","last_became_active":"2024-03-23T15:41:18.646679+0000","last_became_peered":"2024-03-23T15:41:18.646679+0000","last_unstale":"2024-03-23T15:41:19.878572+0000","last_undegraded":"2024-03-23T15:41:19.878572+0000","last_fullsized":"2024-03-23T15:41:19.878572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.710759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.688484+0000","last_change":"2024-03-23T15:41:17.655400+0000","last_active":"2024-03-23T15:41:20.688484+0000","last_peered":"2024-03-23T15:41:20.688484+0000","last_clean":"2024-03-23T15:41:20.688484+0000","last_became_active":"2024-03-23T15:41:17.655034+0000","last_became_peered":"2024-03-23T15:41:17.655034+0000","last_unstale":"2024-03-23T15:41:20.688484+0000","last_undegraded":"2024-03-23T15:41:20.688484+0000","last_fullsized":"2024-03-23T15:41:20.688484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:30.973048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634925+0000","last_change":"2024-03-23T15:41:18.660248+0000","last_active":"2024-03-23T15:41:19.634925+0000","last_peered":"2024-03-23T15:41:19.634925+0000","last_clean":"2024-03-23T15:41:19.634925+0000","last_became_active":"2024-03-23T15:41:18.659728+0000","last_became_peered":"2024-03-23T15:41:18.659728+0000","last_unstale":"2024-03-23T15:41:19.634925+0000","last_undegraded":"2024-03-23T15:41:19.634925+0000","last_fullsized":"2024-03-23T15:41:19.634925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:03.681499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.687051+0000","last_change":"2024-03-23T15:41:17.659784+0000","last_active":"2024-03-23T15:41:20.687051+0000","last_peered":"2024-03-23T15:41:20.687051+0000","last_clean":"2024-03-23T15:41:20.687051+0000","last_became_active":"2024-03-23T15:41:17.659422+0000","last_became_peered":"2024-03-23T15:41:17.659422+0000","last_unstale":"2024-03-23T15:41:20.687051+0000","last_undegraded":"2024-03-23T15:41:20.687051+0000","last_fullsized":"2024-03-23T15:41:20.687051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:48.607487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878809+0000","last_change":"2024-03-23T15:41:18.651777+0000","last_active":"2024-03-23T15:41:19.878809+0000","last_peered":"2024-03-23T15:41:19.878809+0000","last_clean":"2024-03-23T15:41:19.878809+0000","last_became_active":"2024-03-23T15:41:18.651457+0000","last_became_peered":"2024-03-23T15:41:18.651457+0000","last_unstale":"2024-03-23T15:41:19.878809+0000","last_undegraded":"2024-03-23T15:41:19.878809+0000","last_fullsized":"2024-03-23T15:41:19.878809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:25:28.831493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.689405+0000","last_change":"2024-03-23T15:41:17.656357+0000","last_active":"2024-03-23T15:41:20.689405+0000","last_peered":"2024-03-23T15:41:20.689405+0000","last_clean":"2024-03-23T15:41:20.689405+0000","last_became_active":"2024-03-23T15:41:17.655977+0000","last_became_peered":"2024-03-23T15:41:17.655977+0000","last_unstale":"2024-03-23T15:41:20.689405+0000","last_undegraded":"2024-03-23T15:41:20.689405+0000","last_fullsized":"2024-03-23T15:41:20.689405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:07:45.248954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634728+0000","last_change":"2024-03-23T15:41:18.664487+0000","last_active":"2024-03-23T15:41:19.634728+0000","last_peered":"2024-03-23T15:41:19.634728+0000","last_clean":"2024-03-23T15:41:19.634728+0000","last_became_active":"2024-03-23T15:41:18.664141+0000","last_became_peered":"2024-03-23T15:41:18.664141+0000","last_unstale":"2024-03-23T15:41:19.634728+0000","last_undegraded":"2024-03-23T15:41:19.634728+0000","last_fullsized":"2024-03-23T15:41:19.634728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:14:21.960392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:22.702537+0000","last_change":"2024-03-23T15:41:17.659838+0000","last_active":"2024-03-23T15:41:22.702537+0000","last_peered":"2024-03-23T15:41:22.702537+0000","last_clean":"2024-03-23T15:41:22.702537+0000","last_became_active":"2024-03-23T15:41:17.659326+0000","last_became_peered":"2024-03-23T15:41:17.659326+0000","last_unstale":"2024-03-23T15:41:22.702537+0000","last_undegraded":"2024-03-23T15:41:22.702537+0000","last_fullsized":"2024-03-23T15:41:22.702537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:07.651340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878964+0000","last_change":"2024-03-23T15:41:18.656629+0000","last_active":"2024-03-23T15:41:19.878964+0000","last_peered":"2024-03-23T15:41:19.878964+0000","last_clean":"2024-03-23T15:41:19.878964+0000","last_became_active":"2024-03-23T15:41:18.656196+0000","last_became_peered":"2024-03-23T15:41:18.656196+0000","last_unstale":"2024-03-23T15:41:19.878964+0000","last_undegraded":"2024-03-23T15:41:19.878964+0000","last_fullsized":"2024-03-23T15:41:19.878964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:30.013411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.699634+0000","last_change":"2024-03-23T15:41:17.663999+0000","last_active":"2024-03-23T15:41:21.699634+0000","last_peered":"2024-03-23T15:41:21.699634+0000","last_clean":"2024-03-23T15:41:21.699634+0000","last_became_active":"2024-03-23T15:41:17.663039+0000","last_became_peered":"2024-03-23T15:41:17.663039+0000","last_unstale":"2024-03-23T15:41:21.699634+0000","last_undegraded":"2024-03-23T15:41:21.699634+0000","last_fullsized":"2024-03-23T15:41:21.699634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:00.271821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631428+0000","last_change":"2024-03-23T15:41:18.646822+0000","last_active":"2024-03-23T15:41:19.631428+0000","last_peered":"2024-03-23T15:41:19.631428+0000","last_clean":"2024-03-23T15:41:19.631428+0000","last_became_active":"2024-03-23T15:41:18.646476+0000","last_became_peered":"2024-03-23T15:41:18.646476+0000","last_unstale":"2024-03-23T15:41:19.631428+0000","last_undegraded":"2024-03-23T15:41:19.631428+0000","last_fullsized":"2024-03-23T15:41:19.631428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:18.352065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.688600+0000","last_change":"2024-03-23T15:41:17.663507+0000","last_active":"2024-03-23T15:41:20.688600+0000","last_peered":"2024-03-23T15:41:20.688600+0000","last_clean":"2024-03-23T15:41:20.688600+0000","last_became_active":"2024-03-23T15:41:17.663122+0000","last_became_peered":"2024-03-23T15:41:17.663122+0000","last_unstale":"2024-03-23T15:41:20.688600+0000","last_undegraded":"2024-03-23T15:41:20.688600+0000","last_fullsized":"2024-03-23T15:41:20.688600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:32.202576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878458+0000","last_change":"2024-03-23T15:41:18.650110+0000","last_active":"2024-03-23T15:41:19.878458+0000","last_peered":"2024-03-23T15:41:19.878458+0000","last_clean":"2024-03-23T15:41:19.878458+0000","last_became_active":"2024-03-23T15:41:18.649737+0000","last_became_peered":"2024-03-23T15:41:18.649737+0000","last_unstale":"2024-03-23T15:41:19.878458+0000","last_undegraded":"2024-03-23T15:41:19.878458+0000","last_fullsized":"2024-03-23T15:41:19.878458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:31.610961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:22.703653+0000","last_change":"2024-03-23T15:41:17.653930+0000","last_active":"2024-03-23T15:41:22.703653+0000","last_peered":"2024-03-23T15:41:22.703653+0000","last_clean":"2024-03-23T15:41:22.703653+0000","last_became_active":"2024-03-23T15:41:17.652677+0000","last_became_peered":"2024-03-23T15:41:17.652677+0000","last_unstale":"2024-03-23T15:41:22.703653+0000","last_undegraded":"2024-03-23T15:41:22.703653+0000","last_fullsized":"2024-03-23T15:41:22.703653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:47.167052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877876+0000","last_change":"2024-03-23T15:41:18.647269+0000","last_active":"2024-03-23T15:41:19.877876+0000","last_peered":"2024-03-23T15:41:19.877876+0000","last_clean":"2024-03-23T15:41:19.877876+0000","last_became_active":"2024-03-23T15:41:18.646876+0000","last_became_peered":"2024-03-23T15:41:18.646876+0000","last_unstale":"2024-03-23T15:41:19.877876+0000","last_undegraded":"2024-03-23T15:41:19.877876+0000","last_fullsized":"2024-03-23T15:41:19.877876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:31:02.262951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.689253+0000","last_change":"2024-03-23T15:41:17.662091+0000","last_active":"2024-03-23T15:41:20.689253+0000","last_peered":"2024-03-23T15:41:20.689253+0000","last_clean":"2024-03-23T15:41:20.689253+0000","last_became_active":"2024-03-23T15:41:17.661747+0000","last_became_peered":"2024-03-23T15:41:17.661747+0000","last_unstale":"2024-03-23T15:41:20.689253+0000","last_undegraded":"2024-03-23T15:41:20.689253+0000","last_fullsized":"2024-03-23T15:41:20.689253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:08.661262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878605+0000","last_change":"2024-03-23T15:41:18.654356+0000","last_active":"2024-03-23T15:41:19.878605+0000","last_peered":"2024-03-23T15:41:19.878605+0000","last_clean":"2024-03-23T15:41:19.878605+0000","last_became_active":"2024-03-23T15:41:18.654021+0000","last_became_peered":"2024-03-23T15:41:18.654021+0000","last_unstale":"2024-03-23T15:41:19.878605+0000","last_undegraded":"2024-03-23T15:41:19.878605+0000","last_fullsized":"2024-03-23T15:41:19.878605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:28:13.797332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878671+0000","last_change":"2024-03-23T15:41:17.660898+0000","last_active":"2024-03-23T15:41:19.878671+0000","last_peered":"2024-03-23T15:41:19.878671+0000","last_clean":"2024-03-23T15:41:19.878671+0000","last_became_active":"2024-03-23T15:41:17.660515+0000","last_became_peered":"2024-03-23T15:41:17.660515+0000","last_unstale":"2024-03-23T15:41:19.878671+0000","last_undegraded":"2024-03-23T15:41:19.878671+0000","last_fullsized":"2024-03-23T15:41:19.878671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:58.911321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.629434+0000","last_change":"2024-03-23T15:41:18.653582+0000","last_active":"2024-03-23T15:41:20.629434+0000","last_peered":"2024-03-23T15:41:20.629434+0000","last_clean":"2024-03-23T15:41:20.629434+0000","last_became_active":"2024-03-23T15:41:18.653184+0000","last_became_peered":"2024-03-23T15:41:18.653184+0000","last_unstale":"2024-03-23T15:41:20.629434+0000","last_undegraded":"2024-03-23T15:41:20.629434+0000","last_fullsized":"2024-03-23T15:41:20.629434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:59:16.726981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878175+0000","last_change":"2024-03-23T15:41:17.651886+0000","last_active":"2024-03-23T15:41:19.878175+0000","last_peered":"2024-03-23T15:41:19.878175+0000","last_clean":"2024-03-23T15:41:19.878175+0000","last_became_active":"2024-03-23T15:41:17.651472+0000","last_became_peered":"2024-03-23T15:41:17.651472+0000","last_unstale":"2024-03-23T15:41:19.878175+0000","last_undegraded":"2024-03-23T15:41:19.878175+0000","last_fullsized":"2024-03-23T15:41:19.878175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:37.098333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878233+0000","last_change":"2024-03-23T15:41:18.644205+0000","last_active":"2024-03-23T15:41:19.878233+0000","last_peered":"2024-03-23T15:41:19.878233+0000","last_clean":"2024-03-23T15:41:19.878233+0000","last_became_active":"2024-03-23T15:41:18.643850+0000","last_became_peered":"2024-03-23T15:41:18.643850+0000","last_unstale":"2024-03-23T15:41:19.878233+0000","last_undegraded":"2024-03-23T15:41:19.878233+0000","last_fullsized":"2024-03-23T15:41:19.878233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:48:10.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879022+0000","last_change":"2024-03-23T15:41:17.661051+0000","last_active":"2024-03-23T15:41:19.879022+0000","last_peered":"2024-03-23T15:41:19.879022+0000","last_clean":"2024-03-23T15:41:19.879022+0000","last_became_active":"2024-03-23T15:41:17.659673+0000","last_became_peered":"2024-03-23T15:41:17.659673+0000","last_unstale":"2024-03-23T15:41:19.879022+0000","last_undegraded":"2024-03-23T15:41:19.879022+0000","last_fullsized":"2024-03-23T15:41:19.879022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.028104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878885+0000","last_change":"2024-03-23T15:41:18.653630+0000","last_active":"2024-03-23T15:41:19.878885+0000","last_peered":"2024-03-23T15:41:19.878885+0000","last_clean":"2024-03-23T15:41:19.878885+0000","last_became_active":"2024-03-23T15:41:18.653119+0000","last_became_peered":"2024-03-23T15:41:18.653119+0000","last_unstale":"2024-03-23T15:41:19.878885+0000","last_undegraded":"2024-03-23T15:41:19.878885+0000","last_fullsized":"2024-03-23T15:41:19.878885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:02:49.635220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879302+0000","last_change":"2024-03-23T15:41:18.653560+0000","last_active":"2024-03-23T15:41:19.879302+0000","last_peered":"2024-03-23T15:41:19.879302+0000","last_clean":"2024-03-23T15:41:19.879302+0000","last_became_active":"2024-03-23T15:41:18.653249+0000","last_became_peered":"2024-03-23T15:41:18.653249+0000","last_unstale":"2024-03-23T15:41:19.879302+0000","last_undegraded":"2024-03-23T15:41:19.879302+0000","last_fullsized":"2024-03-23T15:41:19.879302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:22.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:23.704368+0000","last_change":"2024-03-23T15:41:17.656318+0000","last_active":"2024-03-23T15:41:23.704368+0000","last_peered":"2024-03-23T15:41:23.704368+0000","last_clean":"2024-03-23T15:41:23.704368+0000","last_became_active":"2024-03-23T15:41:17.655958+0000","last_became_peered":"2024-03-23T15:41:17.655958+0000","last_unstale":"2024-03-23T15:41:23.704368+0000","last_undegraded":"2024-03-23T15:41:23.704368+0000","last_fullsized":"2024-03-23T15:41:23.704368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:12.333504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878150+0000","last_change":"2024-03-23T15:41:18.657616+0000","last_active":"2024-03-23T15:41:19.878150+0000","last_peered":"2024-03-23T15:41:19.878150+0000","last_clean":"2024-03-23T15:41:19.878150+0000","last_became_active":"2024-03-23T15:41:18.657271+0000","last_became_peered":"2024-03-23T15:41:18.657271+0000","last_unstale":"2024-03-23T15:41:19.878150+0000","last_undegraded":"2024-03-23T15:41:19.878150+0000","last_fullsized":"2024-03-23T15:41:19.878150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:02.241087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:22.699962+0000","last_change":"2024-03-23T15:41:17.654358+0000","last_active":"2024-03-23T15:41:22.699962+0000","last_peered":"2024-03-23T15:41:22.699962+0000","last_clean":"2024-03-23T15:41:22.699962+0000","last_became_active":"2024-03-23T15:41:17.653558+0000","last_became_peered":"2024-03-23T15:41:17.653558+0000","last_unstale":"2024-03-23T15:41:22.699962+0000","last_undegraded":"2024-03-23T15:41:22.699962+0000","last_fullsized":"2024-03-23T15:41:22.699962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:32.377883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877988+0000","last_change":"2024-03-23T15:40:05.871801+0000","last_active":"2024-03-23T15:41:19.877988+0000","last_peered":"2024-03-23T15:41:19.877988+0000","last_clean":"2024-03-23T15:41:19.877988+0000","last_became_active":"2024-03-23T15:40:05.871417+0000","last_became_peered":"2024-03-23T15:40:05.871417+0000","last_unstale":"2024-03-23T15:41:19.877988+0000","last_undegraded":"2024-03-23T15:41:19.877988+0000","last_fullsized":"2024-03-23T15:41:19.877988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_clean_scrub_stamp":"2024-03-23T15:36:39.745753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:55.235176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631227+0000","last_change":"2024-03-23T15:41:18.652251+0000","last_active":"2024-03-23T15:41:19.631227+0000","last_peered":"2024-03-23T15:41:19.631227+0000","last_clean":"2024-03-23T15:41:19.631227+0000","last_became_active":"2024-03-23T15:41:18.651876+0000","last_became_peered":"2024-03-23T15:41:18.651876+0000","last_unstale":"2024-03-23T15:41:19.631227+0000","last_undegraded":"2024-03-23T15:41:19.631227+0000","last_fullsized":"2024-03-23T15:41:19.631227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:45.235243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.699883+0000","last_change":"2024-03-23T15:41:17.657338+0000","last_active":"2024-03-23T15:41:21.699883+0000","last_peered":"2024-03-23T15:41:21.699883+0000","last_clean":"2024-03-23T15:41:21.699883+0000","last_became_active":"2024-03-23T15:41:17.656991+0000","last_became_peered":"2024-03-23T15:41:17.656991+0000","last_unstale":"2024-03-23T15:41:21.699883+0000","last_undegraded":"2024-03-23T15:41:21.699883+0000","last_fullsized":"2024-03-23T15:41:21.699883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:23.694865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.689121+0000","last_change":"2024-03-23T15:41:17.660703+0000","last_active":"2024-03-23T15:41:20.689121+0000","last_peered":"2024-03-23T15:41:20.689121+0000","last_clean":"2024-03-23T15:41:20.689121+0000","last_became_active":"2024-03-23T15:41:17.660343+0000","last_became_peered":"2024-03-23T15:41:17.660343+0000","last_unstale":"2024-03-23T15:41:20.689121+0000","last_undegraded":"2024-03-23T15:41:20.689121+0000","last_fullsized":"2024-03-23T15:41:20.689121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:14:03.063397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631450+0000","last_change":"2024-03-23T15:41:18.644624+0000","last_active":"2024-03-23T15:41:19.631450+0000","last_peered":"2024-03-23T15:41:19.631450+0000","last_clean":"2024-03-23T15:41:19.631450+0000","last_became_active":"2024-03-23T15:41:18.644275+0000","last_became_peered":"2024-03-23T15:41:18.644275+0000","last_unstale":"2024-03-23T15:41:19.631450+0000","last_undegraded":"2024-03-23T15:41:19.631450+0000","last_fullsized":"2024-03-23T15:41:19.631450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:25.901935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.701083+0000","last_change":"2024-03-23T15:41:17.650068+0000","last_active":"2024-03-23T15:41:21.701083+0000","last_peered":"2024-03-23T15:41:21.701083+0000","last_clean":"2024-03-23T15:41:21.701083+0000","last_became_active":"2024-03-23T15:41:17.649331+0000","last_became_peered":"2024-03-23T15:41:17.649331+0000","last_unstale":"2024-03-23T15:41:21.701083+0000","last_undegraded":"2024-03-23T15:41:21.701083+0000","last_fullsized":"2024-03-23T15:41:21.701083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:47.578645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878476+0000","last_change":"2024-03-23T15:41:18.656413+0000","last_active":"2024-03-23T15:41:19.878476+0000","last_peered":"2024-03-23T15:41:19.878476+0000","last_clean":"2024-03-23T15:41:19.878476+0000","last_became_active":"2024-03-23T15:41:18.656043+0000","last_became_peered":"2024-03-23T15:41:18.656043+0000","last_unstale":"2024-03-23T15:41:19.878476+0000","last_undegraded":"2024-03-23T15:41:19.878476+0000","last_fullsized":"2024-03-23T15:41:19.878476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:49:25.453420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.686607+0000","last_change":"2024-03-23T15:41:17.654492+0000","last_active":"2024-03-23T15:41:20.686607+0000","last_peered":"2024-03-23T15:41:20.686607+0000","last_clean":"2024-03-23T15:41:20.686607+0000","last_became_active":"2024-03-23T15:41:17.654132+0000","last_became_peered":"2024-03-23T15:41:17.654132+0000","last_unstale":"2024-03-23T15:41:20.686607+0000","last_undegraded":"2024-03-23T15:41:20.686607+0000","last_fullsized":"2024-03-23T15:41:20.686607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:39:58.210703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879037+0000","last_change":"2024-03-23T15:41:18.652659+0000","last_active":"2024-03-23T15:41:19.879037+0000","last_peered":"2024-03-23T15:41:19.879037+0000","last_clean":"2024-03-23T15:41:19.879037+0000","last_became_active":"2024-03-23T15:41:18.652246+0000","last_became_peered":"2024-03-23T15:41:18.652246+0000","last_unstale":"2024-03-23T15:41:19.879037+0000","last_undegraded":"2024-03-23T15:41:19.879037+0000","last_fullsized":"2024-03-23T15:41:19.879037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:50.868694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.688418+0000","last_change":"2024-03-23T15:41:17.656928+0000","last_active":"2024-03-23T15:41:20.688418+0000","last_peered":"2024-03-23T15:41:20.688418+0000","last_clean":"2024-03-23T15:41:20.688418+0000","last_became_active":"2024-03-23T15:41:17.656478+0000","last_became_peered":"2024-03-23T15:41:17.656478+0000","last_unstale":"2024-03-23T15:41:20.688418+0000","last_undegraded":"2024-03-23T15:41:20.688418+0000","last_fullsized":"2024-03-23T15:41:20.688418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:14:46.716508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878350+0000","last_change":"2024-03-23T15:41:18.654049+0000","last_active":"2024-03-23T15:41:19.878350+0000","last_peered":"2024-03-23T15:41:19.878350+0000","last_clean":"2024-03-23T15:41:19.878350+0000","last_became_active":"2024-03-23T15:41:18.653765+0000","last_became_peered":"2024-03-23T15:41:18.653765+0000","last_unstale":"2024-03-23T15:41:19.878350+0000","last_undegraded":"2024-03-23T15:41:19.878350+0000","last_fullsized":"2024-03-23T15:41:19.878350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:12:11.467744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.687573+0000","last_change":"2024-03-23T15:41:17.656679+0000","last_active":"2024-03-23T15:41:20.687573+0000","last_peered":"2024-03-23T15:41:20.687573+0000","last_clean":"2024-03-23T15:41:20.687573+0000","last_became_active":"2024-03-23T15:41:17.656368+0000","last_became_peered":"2024-03-23T15:41:17.656368+0000","last_unstale":"2024-03-23T15:41:20.687573+0000","last_undegraded":"2024-03-23T15:41:20.687573+0000","last_fullsized":"2024-03-23T15:41:20.687573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:14.176288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.629081+0000","last_change":"2024-03-23T15:41:18.653637+0000","last_active":"2024-03-23T15:41:20.629081+0000","last_peered":"2024-03-23T15:41:20.629081+0000","last_clean":"2024-03-23T15:41:20.629081+0000","last_became_active":"2024-03-23T15:41:18.652864+0000","last_became_peered":"2024-03-23T15:41:18.652864+0000","last_unstale":"2024-03-23T15:41:20.629081+0000","last_undegraded":"2024-03-23T15:41:20.629081+0000","last_fullsized":"2024-03-23T15:41:20.629081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:32.104901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.767279+0000","last_change":"2024-03-23T15:41:17.664417+0000","last_active":"2024-03-23T15:41:19.767279+0000","last_peered":"2024-03-23T15:41:19.767279+0000","last_clean":"2024-03-23T15:41:19.767279+0000","last_became_active":"2024-03-23T15:41:17.663454+0000","last_became_peered":"2024-03-23T15:41:17.663454+0000","last_unstale":"2024-03-23T15:41:19.767279+0000","last_undegraded":"2024-03-23T15:41:19.767279+0000","last_fullsized":"2024-03-23T15:41:19.767279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:14.947767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877872+0000","last_change":"2024-03-23T15:41:18.647857+0000","last_active":"2024-03-23T15:41:19.877872+0000","last_peered":"2024-03-23T15:41:19.877872+0000","last_clean":"2024-03-23T15:41:19.877872+0000","last_became_active":"2024-03-23T15:41:18.646924+0000","last_became_peered":"2024-03-23T15:41:18.646924+0000","last_unstale":"2024-03-23T15:41:19.877872+0000","last_undegraded":"2024-03-23T15:41:19.877872+0000","last_fullsized":"2024-03-23T15:41:19.877872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:19.494901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.688714+0000","last_change":"2024-03-23T15:41:17.661995+0000","last_active":"2024-03-23T15:41:20.688714+0000","last_peered":"2024-03-23T15:41:20.688714+0000","last_clean":"2024-03-23T15:41:20.688714+0000","last_became_active":"2024-03-23T15:41:17.661632+0000","last_became_peered":"2024-03-23T15:41:17.661632+0000","last_unstale":"2024-03-23T15:41:20.688714+0000","last_undegraded":"2024-03-23T15:41:20.688714+0000","last_fullsized":"2024-03-23T15:41:20.688714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:12.540038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878870+0000","last_change":"2024-03-23T15:41:18.651258+0000","last_active":"2024-03-23T15:41:19.878870+0000","last_peered":"2024-03-23T15:41:19.878870+0000","last_clean":"2024-03-23T15:41:19.878870+0000","last_became_active":"2024-03-23T15:41:18.650910+0000","last_became_peered":"2024-03-23T15:41:18.650910+0000","last_unstale":"2024-03-23T15:41:19.878870+0000","last_undegraded":"2024-03-23T15:41:19.878870+0000","last_fullsized":"2024-03-23T15:41:19.878870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:06:50.139644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:21.701893+0000","last_change":"2024-03-23T15:41:17.654232+0000","last_active":"2024-03-23T15:41:21.701893+0000","last_peered":"2024-03-23T15:41:21.701893+0000","last_clean":"2024-03-23T15:41:21.701893+0000","last_became_active":"2024-03-23T15:41:17.653832+0000","last_became_peered":"2024-03-23T15:41:17.653832+0000","last_unstale":"2024-03-23T15:41:21.701893+0000","last_undegraded":"2024-03-23T15:41:21.701893+0000","last_fullsized":"2024-03-23T15:41:21.701893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:05.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878491+0000","last_change":"2024-03-23T15:41:18.664444+0000","last_active":"2024-03-23T15:41:19.878491+0000","last_peered":"2024-03-23T15:41:19.878491+0000","last_clean":"2024-03-23T15:41:19.878491+0000","last_became_active":"2024-03-23T15:41:18.664127+0000","last_became_peered":"2024-03-23T15:41:18.664127+0000","last_unstale":"2024-03-23T15:41:19.878491+0000","last_undegraded":"2024-03-23T15:41:19.878491+0000","last_fullsized":"2024-03-23T15:41:19.878491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:25.942230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879058+0000","last_change":"2024-03-23T15:41:17.664834+0000","last_active":"2024-03-23T15:41:19.879058+0000","last_peered":"2024-03-23T15:41:19.879058+0000","last_clean":"2024-03-23T15:41:19.879058+0000","last_became_active":"2024-03-23T15:41:17.664286+0000","last_became_peered":"2024-03-23T15:41:17.664286+0000","last_unstale":"2024-03-23T15:41:19.879058+0000","last_undegraded":"2024-03-23T15:41:19.879058+0000","last_fullsized":"2024-03-23T15:41:19.879058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:50.595420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878154+0000","last_change":"2024-03-23T15:41:18.646341+0000","last_active":"2024-03-23T15:41:19.878154+0000","last_peered":"2024-03-23T15:41:19.878154+0000","last_clean":"2024-03-23T15:41:19.878154+0000","last_became_active":"2024-03-23T15:41:18.645995+0000","last_became_peered":"2024-03-23T15:41:18.645995+0000","last_unstale":"2024-03-23T15:41:19.878154+0000","last_undegraded":"2024-03-23T15:41:19.878154+0000","last_fullsized":"2024-03-23T15:41:19.878154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:56.408957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631339+0000","last_change":"2024-03-23T15:41:17.656637+0000","last_active":"2024-03-23T15:41:19.631339+0000","last_peered":"2024-03-23T15:41:19.631339+0000","last_clean":"2024-03-23T15:41:19.631339+0000","last_became_active":"2024-03-23T15:41:17.656271+0000","last_became_peered":"2024-03-23T15:41:17.656271+0000","last_unstale":"2024-03-23T15:41:19.631339+0000","last_undegraded":"2024-03-23T15:41:19.631339+0000","last_fullsized":"2024-03-23T15:41:19.631339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:01:15.747209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878186+0000","last_change":"2024-03-23T15:41:18.655250+0000","last_active":"2024-03-23T15:41:19.878186+0000","last_peered":"2024-03-23T15:41:19.878186+0000","last_clean":"2024-03-23T15:41:19.878186+0000","last_became_active":"2024-03-23T15:41:18.654888+0000","last_became_peered":"2024-03-23T15:41:18.654888+0000","last_unstale":"2024-03-23T15:41:19.878186+0000","last_undegraded":"2024-03-23T15:41:19.878186+0000","last_fullsized":"2024-03-23T15:41:19.878186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:36.730600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.629249+0000","last_change":"2024-03-23T15:41:18.654309+0000","last_active":"2024-03-23T15:41:20.629249+0000","last_peered":"2024-03-23T15:41:20.629249+0000","last_clean":"2024-03-23T15:41:20.629249+0000","last_became_active":"2024-03-23T15:41:18.653319+0000","last_became_peered":"2024-03-23T15:41:18.653319+0000","last_unstale":"2024-03-23T15:41:20.629249+0000","last_undegraded":"2024-03-23T15:41:20.629249+0000","last_fullsized":"2024-03-23T15:41:20.629249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:02:05.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632334+0000","last_change":"2024-03-23T15:41:18.655179+0000","last_active":"2024-03-23T15:41:19.632334+0000","last_peered":"2024-03-23T15:41:19.632334+0000","last_clean":"2024-03-23T15:41:19.632334+0000","last_became_active":"2024-03-23T15:41:18.654830+0000","last_became_peered":"2024-03-23T15:41:18.654830+0000","last_unstale":"2024-03-23T15:41:19.632334+0000","last_undegraded":"2024-03-23T15:41:19.632334+0000","last_fullsized":"2024-03-23T15:41:19.632334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:29.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632307+0000","last_change":"2024-03-23T15:41:18.650734+0000","last_active":"2024-03-23T15:41:19.632307+0000","last_peered":"2024-03-23T15:41:19.632307+0000","last_clean":"2024-03-23T15:41:19.632307+0000","last_became_active":"2024-03-23T15:41:18.649993+0000","last_became_peered":"2024-03-23T15:41:18.649993+0000","last_unstale":"2024-03-23T15:41:19.632307+0000","last_undegraded":"2024-03-23T15:41:19.632307+0000","last_fullsized":"2024-03-23T15:41:19.632307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:06.463525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878316+0000","last_change":"2024-03-23T15:41:18.655741+0000","last_active":"2024-03-23T15:41:19.878316+0000","last_peered":"2024-03-23T15:41:19.878316+0000","last_clean":"2024-03-23T15:41:19.878316+0000","last_became_active":"2024-03-23T15:41:18.653387+0000","last_became_peered":"2024-03-23T15:41:18.653387+0000","last_unstale":"2024-03-23T15:41:19.878316+0000","last_undegraded":"2024-03-23T15:41:19.878316+0000","last_fullsized":"2024-03-23T15:41:19.878316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:20:26.758549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631518+0000","last_change":"2024-03-23T15:41:18.654739+0000","last_active":"2024-03-23T15:41:19.631518+0000","last_peered":"2024-03-23T15:41:19.631518+0000","last_clean":"2024-03-23T15:41:19.631518+0000","last_became_active":"2024-03-23T15:41:18.654151+0000","last_became_peered":"2024-03-23T15:41:18.654151+0000","last_unstale":"2024-03-23T15:41:19.631518+0000","last_undegraded":"2024-03-23T15:41:19.631518+0000","last_fullsized":"2024-03-23T15:41:19.631518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:01.358959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.629356+0000","last_change":"2024-03-23T15:41:18.654081+0000","last_active":"2024-03-23T15:41:20.629356+0000","last_peered":"2024-03-23T15:41:20.629356+0000","last_clean":"2024-03-23T15:41:20.629356+0000","last_became_active":"2024-03-23T15:41:18.653696+0000","last_became_peered":"2024-03-23T15:41:18.653696+0000","last_unstale":"2024-03-23T15:41:20.629356+0000","last_undegraded":"2024-03-23T15:41:20.629356+0000","last_fullsized":"2024-03-23T15:41:20.629356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:23:19.741645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.632755+0000","last_change":"2024-03-23T15:41:18.652356+0000","last_active":"2024-03-23T15:41:19.632755+0000","last_peered":"2024-03-23T15:41:19.632755+0000","last_clean":"2024-03-23T15:41:19.632755+0000","last_became_active":"2024-03-23T15:41:18.651976+0000","last_became_peered":"2024-03-23T15:41:18.651976+0000","last_unstale":"2024-03-23T15:41:19.632755+0000","last_undegraded":"2024-03-23T15:41:19.632755+0000","last_fullsized":"2024-03-23T15:41:19.632755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:38.972186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.878262+0000","last_change":"2024-03-23T15:41:18.653775+0000","last_active":"2024-03-23T15:41:19.878262+0000","last_peered":"2024-03-23T15:41:19.878262+0000","last_clean":"2024-03-23T15:41:19.878262+0000","last_became_active":"2024-03-23T15:41:18.653404+0000","last_became_peered":"2024-03-23T15:41:18.653404+0000","last_unstale":"2024-03-23T15:41:19.878262+0000","last_undegraded":"2024-03-23T15:41:19.878262+0000","last_fullsized":"2024-03-23T15:41:19.878262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:28:57.534731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.879196+0000","last_change":"2024-03-23T15:41:18.646098+0000","last_active":"2024-03-23T15:41:19.879196+0000","last_peered":"2024-03-23T15:41:19.879196+0000","last_clean":"2024-03-23T15:41:19.879196+0000","last_became_active":"2024-03-23T15:41:18.645815+0000","last_became_peered":"2024-03-23T15:41:18.645815+0000","last_unstale":"2024-03-23T15:41:19.879196+0000","last_undegraded":"2024-03-23T15:41:19.879196+0000","last_fullsized":"2024-03-23T15:41:19.879196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:28:58.091867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.877608+0000","last_change":"2024-03-23T15:41:18.652487+0000","last_active":"2024-03-23T15:41:19.877608+0000","last_peered":"2024-03-23T15:41:19.877608+0000","last_clean":"2024-03-23T15:41:19.877608+0000","last_became_active":"2024-03-23T15:41:18.652151+0000","last_became_peered":"2024-03-23T15:41:18.652151+0000","last_unstale":"2024-03-23T15:41:19.877608+0000","last_undegraded":"2024-03-23T15:41:19.877608+0000","last_fullsized":"2024-03-23T15:41:19.877608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:39.133585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.634597+0000","last_change":"2024-03-23T15:41:18.657265+0000","last_active":"2024-03-23T15:41:19.634597+0000","last_peered":"2024-03-23T15:41:19.634597+0000","last_clean":"2024-03-23T15:41:19.634597+0000","last_became_active":"2024-03-23T15:41:18.656970+0000","last_became_peered":"2024-03-23T15:41:18.656970+0000","last_unstale":"2024-03-23T15:41:19.634597+0000","last_undegraded":"2024-03-23T15:41:19.634597+0000","last_fullsized":"2024-03-23T15:41:19.634597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:22.808034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:19.631480+0000","last_change":"2024-03-23T15:41:18.655254+0000","last_active":"2024-03-23T15:41:19.631480+0000","last_peered":"2024-03-23T15:41:19.631480+0000","last_clean":"2024-03-23T15:41:19.631480+0000","last_became_active":"2024-03-23T15:41:18.654879+0000","last_became_peered":"2024-03-23T15:41:18.654879+0000","last_unstale":"2024-03-23T15:41:19.631480+0000","last_undegraded":"2024-03-23T15:41:19.631480+0000","last_fullsized":"2024-03-23T15:41:19.631480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:36:18.923723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-23T15:41:20.629190+0000","last_change":"2024-03-23T15:41:18.658652+0000","last_active":"2024-03-23T15:41:20.629190+0000","last_peered":"2024-03-23T15:41:20.629190+0000","last_clean":"2024-03-23T15:41:20.629190+0000","last_became_active":"2024-03-23T15:41:18.658261+0000","last_became_peered":"2024-03-23T15:41:18.658261+0000","last_unstale":"2024-03-23T15:41:20.629190+0000","last_undegraded":"2024-03-23T15:41:20.629190+0000","last_fullsized":"2024-03-23T15:41:20.629190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:45.145101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245950,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5501,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":25,"num_read_kb":20,"num_write":118,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":89105,"data_compressed":3738,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":1031189,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":438728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":476714,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":478455,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1030756,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38193,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":39934,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T15:41:25.751 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T15:41:25.751 INFO:tasks.cephadm:Setting up client nodes... 2024-03-23T15:41:25.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-23T15:41:26.026 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:26.034 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:25 smithi183 ceph-mon[36296]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-23T15:41:26.034 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:25 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1241617129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:26.034 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:25 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1427591157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:26.034 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:25 smithi183 ceph-mon[36296]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:41:26.034 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:25 smithi183 ceph-mon[36296]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-03-23T15:41:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:25 smithi192 ceph-mon[36653]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-23T15:41:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:25 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1241617129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:25 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1427591157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:25 smithi192 ceph-mon[36653]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:41:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:25 smithi192 ceph-mon[36653]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-03-23T15:41:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:25 smithi076 ceph-mon[32176]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-23T15:41:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1241617129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:41:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1427591157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:25 smithi076 ceph-mon[32176]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:41:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:25 smithi076 ceph-mon[32176]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-03-23T15:41:26.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:26 smithi192 ceph-mon[36653]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:41:27.051 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:26 smithi183 ceph-mon[36296]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:41:27.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:26 smithi076 ceph-mon[32176]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:41:27.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 -- 172.21.15.76:0/79034780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320105660 msgr2=0x7fd320107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:27.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 --2- 172.21.15.76:0/79034780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320105660 0x7fd320107a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd308009990 tx=0x7fd308009ca0 comp rx=0 tx=0).stop 2024-03-23T15:41:27.723 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:27 smithi076 ceph-mon[32176]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-03-23T15:41:27.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 -- 172.21.15.76:0/79034780 shutdown_connections 2024-03-23T15:41:27.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 --2- 172.21.15.76:0/79034780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd320107f90 0x7fd32010a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:27.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 --2- 172.21.15.76:0/79034780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320102d30 0x7fd320105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 --2- 172.21.15.76:0/79034780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320105660 0x7fd320107a50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 -- 172.21.15.76:0/79034780 >> 172.21.15.76:0/79034780 conn(0x7fd3200fc700 msgr2=0x7fd3200feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 -- 172.21.15.76:0/79034780 shutdown_connections 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.720+0000 7fd3266d7700 1 -- 172.21.15.76:0/79034780 wait complete. 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 Processor -- start 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 -- start start 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320102d30 0x7fd320198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd320105660 0x7fd320198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320107f90 0x7fd32019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd32019fcc0 con 0x7fd320102d30 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd32019fe00 con 0x7fd320107f90 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd3266d7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd32019ff90 con 0x7fd320105660 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320102d30 0x7fd320198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320102d30 0x7fd320198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36946/0 (socket says 172.21.15.76:36946) 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd317fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd320105660 0x7fd320198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 -- 172.21.15.76:0/3973833336 learned_addr learned my addr 172.21.15.76:0/3973833336 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd324c74700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320107f90 0x7fd32019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 -- 172.21.15.76:0/3973833336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd320105660 msgr2=0x7fd320198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd320105660 0x7fd320198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 -- 172.21.15.76:0/3973833336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320107f90 msgr2=0x7fd32019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320107f90 0x7fd32019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.721+0000 7fd31ffff700 1 -- 172.21.15.76:0/3973833336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3080095f0 con 0x7fd320102d30 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.722+0000 7fd31ffff700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320102d30 0x7fd320198130 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd31000e9d0 tx=0x7fd31000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.722+0000 7fd31dffb700 1 -- 172.21.15.76:0/3973833336 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310009790 con 0x7fd320102d30 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.722+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3201a0270 con 0x7fd320102d30 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.722+0000 7fd31dffb700 1 -- 172.21.15.76:0/3973833336 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31000ee70 con 0x7fd320102d30 2024-03-23T15:41:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.722+0000 7fd31dffb700 1 -- 172.21.15.76:0/3973833336 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310018680 con 0x7fd320102d30 2024-03-23T15:41:27.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.722+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3201a0770 con 0x7fd320102d30 2024-03-23T15:41:27.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.723+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3200fe280 con 0x7fd320102d30 2024-03-23T15:41:27.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.724+0000 7fd31dffb700 1 -- 172.21.15.76:0/3973833336 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fd31000cbd0 con 0x7fd320102d30 2024-03-23T15:41:27.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.724+0000 7fd31dffb700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd30c0a2290 0x7fd30c0a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:27.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.725+0000 7fd317fff700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd30c0a2290 0x7fd30c0a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:27.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.725+0000 7fd31dffb700 1 -- 172.21.15.76:0/3973833336 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd310014070 con 0x7fd320102d30 2024-03-23T15:41:27.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.725+0000 7fd317fff700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd30c0a2290 0x7fd30c0a4750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd30800b820 tx=0x7fd308005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:27.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:27.729+0000 7fd31dffb700 1 -- 172.21.15.76:0/3973833336 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd310053280 con 0x7fd320102d30 2024-03-23T15:41:28.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.002+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd320050710 con 0x7fd320102d30 2024-03-23T15:41:28.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.004+0000 7fd31dffb700 1 -- 172.21.15.76:0/3973833336 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd310096e40 con 0x7fd320102d30 2024-03-23T15:41:28.006 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-03-23T15:41:28.006 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAo+P5llaQ0ABAAsFQvc+b9PSBgMaQ7tvdTlw== 2024-03-23T15:41:28.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd30c0a2290 msgr2=0x7fd30c0a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd30c0a2290 0x7fd30c0a4750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd30800b820 tx=0x7fd308005fb0 comp rx=0 tx=0).stop 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320102d30 msgr2=0x7fd320198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320102d30 0x7fd320198130 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd31000e9d0 tx=0x7fd31000ece0 comp rx=0 tx=0).stop 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 shutdown_connections 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd30c0a2290 0x7fd30c0a4750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320107f90 0x7fd32019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd320102d30 0x7fd320198130 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 --2- 172.21.15.76:0/3973833336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd320105660 0x7fd320198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.006+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 >> 172.21.15.76:0/3973833336 conn(0x7fd3200fc700 msgr2=0x7fd320103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.007+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 shutdown_connections 2024-03-23T15:41:28.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:28.007+0000 7fd3266d7700 1 -- 172.21.15.76:0/3973833336 wait complete. 2024-03-23T15:41:28.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:27 smithi192 ceph-mon[36653]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-03-23T15:41:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:27 smithi183 ceph-mon[36296]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-03-23T15:41:28.676 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:41:28.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-23T15:41:28.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T15:41:28.720 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T15:41:28.720 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-23T15:41:28.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-23T15:41:29.000 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:28 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3973833336' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T15:41:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:28 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3973833336' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T15:41:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:28 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3973833336' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T15:41:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:28 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3973833336' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T15:41:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:28 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3973833336' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T15:41:29.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:28 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3973833336' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T15:41:30.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:29 smithi192 ceph-mon[36653]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-03-23T15:41:30.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:29 smithi076 ceph-mon[32176]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-03-23T15:41:30.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:29 smithi183 ceph-mon[36296]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-03-23T15:41:30.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 -- 172.21.15.76:0/1002427930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 msgr2=0x7f62181026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:30.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 --2- 172.21.15.76:0/1002427930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181026d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f62080099c0 tx=0x7f6208009cd0 comp rx=0 tx=0).stop 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 -- 172.21.15.76:0/1002427930 shutdown_connections 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 --2- 172.21.15.76:0/1002427930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6218102c10 0x7f621810f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 --2- 172.21.15.76:0/1002427930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6218108230 0x7f6218108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 --2- 172.21.15.76:0/1002427930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181026d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 -- 172.21.15.76:0/1002427930 >> 172.21.15.76:0/1002427930 conn(0x7f6218079330 msgr2=0x7f6218079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 -- 172.21.15.76:0/1002427930 shutdown_connections 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.748+0000 7f62202a0700 1 -- 172.21.15.76:0/1002427930 wait complete. 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 Processor -- start 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 -- start start 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6218102c10 0x7f62181a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6218108230 0x7f621819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f621819b250 con 0x7f6218102250 2024-03-23T15:41:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f621819b3c0 con 0x7f6218102c10 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f62202a0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f621819b580 con 0x7f6218108230 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f621e03c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f621e03c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36950/0 (socket says 172.21.15.76:36950) 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.749+0000 7f621e03c700 1 -- 172.21.15.76:0/2629598711 learned_addr learned my addr 172.21.15.76:0/2629598711 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621d83b700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6218102c10 0x7f62181a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e83d700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6218108230 0x7f621819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e03c700 1 -- 172.21.15.76:0/2629598711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6218108230 msgr2=0x7f621819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e03c700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6218108230 0x7f621819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e03c700 1 -- 172.21.15.76:0/2629598711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6218102c10 msgr2=0x7f62181a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e03c700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6218102c10 0x7f62181a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e03c700 1 -- 172.21.15.76:0/2629598711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62080095f0 con 0x7f6218102250 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e83d700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6218108230 0x7f621819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f621e03c700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181a04c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f621000ef10 tx=0x7f621000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f620f7fe700 1 -- 172.21.15.76:0/2629598711 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621000ca70 con 0x7f6218102250 2024-03-23T15:41:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f620f7fe700 1 -- 172.21.15.76:0/2629598711 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621000f040 con 0x7f6218102250 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621819b800 con 0x7f6218102250 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f620f7fe700 1 -- 172.21.15.76:0/2629598711 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62100146e0 con 0x7f6218102250 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.750+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181a7540 con 0x7f6218102250 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.751+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218068ec0 con 0x7f6218102250 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.753+0000 7f620f7fe700 1 -- 172.21.15.76:0/2629598711 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f6210007500 con 0x7f6218102250 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.753+0000 7f620f7fe700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f62040a21e0 0x7f62040a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.753+0000 7f621d83b700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f62040a21e0 0x7f62040a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.754+0000 7f620f7fe700 1 -- 172.21.15.76:0/2629598711 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f62100ce650 con 0x7f6218102250 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.754+0000 7f621d83b700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f62040a21e0 0x7f62040a46a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6208000c00 tx=0x7f6208005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:30.757+0000 7f620f7fe700 1 -- 172.21.15.76:0/2629598711 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6210097670 con 0x7f6218102250 2024-03-23T15:41:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.006+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f6218074b90 con 0x7f6218102250 2024-03-23T15:41:31.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.011+0000 7f620f7fe700 1 -- 172.21.15.76:0/2629598711 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+261470 (secure 0 0 0) 0x7f621009ad30 con 0x7f6218102250 2024-03-23T15:41:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:31.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f62040a21e0 msgr2=0x7f62040a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:31.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f62040a21e0 0x7f62040a46a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6208000c00 tx=0x7f6208005fb0 comp rx=0 tx=0).stop 2024-03-23T15:41:31.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 msgr2=0x7f62181a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:31.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181a04c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f621000ef10 tx=0x7f621000c3a0 comp rx=0 tx=0).stop 2024-03-23T15:41:31.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 shutdown_connections 2024-03-23T15:41:31.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f62040a21e0 0x7f62040a46a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:31.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6218108230 0x7f621819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:31.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6218102250 0x7f62181a04c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:31.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 --2- 172.21.15.76:0/2629598711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6218102c10 0x7f62181a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:31.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 >> 172.21.15.76:0/2629598711 conn(0x7f6218079330 msgr2=0x7f6218100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:31.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 shutdown_connections 2024-03-23T15:41:31.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:31.014+0000 7f62202a0700 1 -- 172.21.15.76:0/2629598711 wait complete. 2024-03-23T15:41:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:31 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2629598711' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T15:41:31.328 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:31 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2629598711' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T15:41:31.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:31 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2629598711' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T15:41:31.733 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2479514030},{"type":"v1","addr":"172.21.15.76:6801","nonce":2479514030}]},"active_addr":"172.21.15.76:6801/2479514030","active_change":"2024-03-23T15:34:15.395585+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.76:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":4249446235}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":4023925236}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":1911361196}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":3432469273}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":1491581826}]}]} 2024-03-23T15:41:31.739 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-23T15:41:31.739 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-23T15:41:31.739 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T15:41:31.979 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:32.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:32 smithi192 ceph-mon[36653]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-23T15:41:32.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:32 smithi076 ceph-mon[32176]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-23T15:41:32.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:32 smithi183 ceph-mon[36296]: pgmap v293: 97 pgs: 97 active+clean; 623 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-23T15:41:33.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 -- 172.21.15.76:0/3405826095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 msgr2=0x7febf810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:33.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 --2- 172.21.15.76:0/3405826095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf810df80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7febf000b0a0 tx=0x7febf000b3b0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 -- 172.21.15.76:0/3405826095 shutdown_connections 2024-03-23T15:41:33.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 --2- 172.21.15.76:0/3405826095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf810df80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 --2- 172.21.15.76:0/3405826095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf8107140 0x7febf8107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 --2- 172.21.15.76:0/3405826095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf8101100 0x7febf8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 -- 172.21.15.76:0/3405826095 >> 172.21.15.76:0/3405826095 conn(0x7febf8078030 msgr2=0x7febf8078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:33.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 -- 172.21.15.76:0/3405826095 shutdown_connections 2024-03-23T15:41:33.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.661+0000 7febff014700 1 -- 172.21.15.76:0/3405826095 wait complete. 2024-03-23T15:41:33.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 Processor -- start 2024-03-23T15:41:33.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 -- start start 2024-03-23T15:41:33.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf8101100 0x7febf819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:33.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:33.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf8107140 0x7febf8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:33.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7febf8198cd0 con 0x7febf8101aa0 2024-03-23T15:41:33.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febf8198e10 con 0x7febf8101100 2024-03-23T15:41:33.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febff014700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7febf8198fd0 con 0x7febf8107140 2024-03-23T15:41:33.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:33.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36970/0 (socket says 172.21.15.76:36970) 2024-03-23T15:41:33.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 -- 172.21.15.76:0/3309418989 learned_addr learned my addr 172.21.15.76:0/3309418989 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:33.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febfcdb0700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf8101100 0x7febf819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:33.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febfd5b1700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf8107140 0x7febf8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:33.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 -- 172.21.15.76:0/3309418989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf8107140 msgr2=0x7febf8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:33.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf8107140 0x7febf8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 -- 172.21.15.76:0/3309418989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf8101100 msgr2=0x7febf819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:33.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf8101100 0x7febf819e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febf7fff700 1 -- 172.21.15.76:0/3309418989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf0009d00 con 0x7febf8101aa0 2024-03-23T15:41:33.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.662+0000 7febfcdb0700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf8101100 0x7febf819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:33.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.663+0000 7febf7fff700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf819e9a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7febe400e9a0 tx=0x7febe400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.663+0000 7febf5ffb700 1 -- 172.21.15.76:0/3309418989 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4009790 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.663+0000 7febf5ffb700 1 -- 172.21.15.76:0/3309418989 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe4010430 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.663+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf81992b0 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.663+0000 7febf5ffb700 1 -- 172.21.15.76:0/3309418989 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4018680 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.663+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf81a5500 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.664+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf8068ec0 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.665+0000 7febf5ffb700 1 -- 172.21.15.76:0/3309418989 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7febe4010910 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.665+0000 7febf5ffb700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7febe80a21e0 0x7febe80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.666+0000 7febfcdb0700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7febe80a21e0 0x7febe80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.666+0000 7febf5ffb700 1 -- 172.21.15.76:0/3309418989 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7febe4014070 con 0x7febf8101aa0 2024-03-23T15:41:33.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.666+0000 7febfcdb0700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7febe80a21e0 0x7febe80a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7febec009fd0 tx=0x7febec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:33.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.670+0000 7febf5ffb700 1 -- 172.21.15.76:0/3309418989 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7febe4053280 con 0x7febf8101aa0 2024-03-23T15:41:33.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.869+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febf8111110 con 0x7febf8101aa0 2024-03-23T15:41:33.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.871+0000 7febf5ffb700 1 -- 172.21.15.76:0/3309418989 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22203 (secure 0 0 0) 0x7febe4096ed0 con 0x7febf8101aa0 2024-03-23T15:41:33.873 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:33.873 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":85,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:41:19.628233+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:41:33.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7febe80a21e0 msgr2=0x7febe80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:33.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7febe80a21e0 0x7febe80a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7febec009fd0 tx=0x7febec009260 comp rx=0 tx=0).stop 2024-03-23T15:41:33.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 msgr2=0x7febf819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:33.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf819e9a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7febe400e9a0 tx=0x7febe400ecb0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 shutdown_connections 2024-03-23T15:41:33.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7febe80a21e0 0x7febe80a46a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febf8107140 0x7febf8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febf8101100 0x7febf819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 --2- 172.21.15.76:0/3309418989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febf8101aa0 0x7febf819e9a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:33.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 >> 172.21.15.76:0/3309418989 conn(0x7febf8078030 msgr2=0x7febf8100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:33.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 shutdown_connections 2024-03-23T15:41:33.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:33.873+0000 7febff014700 1 -- 172.21.15.76:0/3309418989 wait complete. 2024-03-23T15:41:34.583 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-23T15:41:34.583 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T15:41:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:34 smithi192 ceph-mon[36653]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-03-23T15:41:34.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:34 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3309418989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:34 smithi076 ceph-mon[32176]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-03-23T15:41:34.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:34 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3309418989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:34.827 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:34 smithi183 ceph-mon[36296]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-03-23T15:41:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:34 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3309418989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 -- 172.21.15.76:0/4111228189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06b550 msgr2=0x7f106c106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 --2- 172.21.15.76:0/4111228189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06b550 0x7f106c106f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10600099c0 tx=0x7f1060009cd0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 -- 172.21.15.76:0/4111228189 shutdown_connections 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 --2- 172.21.15.76:0/4111228189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f106c107640 0x7f106c10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 --2- 172.21.15.76:0/4111228189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f106c06aba0 0x7f106c06af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 --2- 172.21.15.76:0/4111228189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06b550 0x7f106c106f10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 -- 172.21.15.76:0/4111228189 >> 172.21.15.76:0/4111228189 conn(0x7f106c077f60 msgr2=0x7f106c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:36.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 -- 172.21.15.76:0/4111228189 shutdown_connections 2024-03-23T15:41:36.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.514+0000 7f10725fe700 1 -- 172.21.15.76:0/4111228189 wait complete. 2024-03-23T15:41:36.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 Processor -- start 2024-03-23T15:41:36.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 -- start start 2024-03-23T15:41:36.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06aba0 0x7f106c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f106c06b550 0x7f106c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f106c107640 0x7f106c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f106c198c20 con 0x7f106c06aba0 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f106c198d60 con 0x7f106c107640 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f10725fe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f106c198f20 con 0x7f106c06b550 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f106bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06aba0 0x7f106c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.515+0000 7f106bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06aba0 0x7f106c19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52282/0 (socket says 172.21.15.76:52282) 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106bfff700 1 -- 172.21.15.76:0/2361871234 learned_addr learned my addr 172.21.15.76:0/2361871234 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106b7fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f106c06b550 0x7f106c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f1070b9b700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f106c107640 0x7f106c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106bfff700 1 -- 172.21.15.76:0/2361871234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f106c06b550 msgr2=0x7f106c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106bfff700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f106c06b550 0x7f106c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106bfff700 1 -- 172.21.15.76:0/2361871234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f106c107640 msgr2=0x7f106c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:36.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106bfff700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f106c107640 0x7f106c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106bfff700 1 -- 172.21.15.76:0/2361871234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600095f0 con 0x7f106c06aba0 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f106bfff700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06aba0 0x7f106c19e400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e9c0 tx=0x7f105c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f10697fa700 1 -- 172.21.15.76:0/2361871234 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0097b0 con 0x7f106c06aba0 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f10697fa700 1 -- 172.21.15.76:0/2361871234 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105c00ee60 con 0x7f106c06aba0 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f10697fa700 1 -- 172.21.15.76:0/2361871234 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c005600 con 0x7f106c06aba0 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c199200 con 0x7f106c06aba0 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.516+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c1a54a0 con 0x7f106c06aba0 2024-03-23T15:41:36.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.517+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c10a8a0 con 0x7f106c06aba0 2024-03-23T15:41:36.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.518+0000 7f10697fa700 1 -- 172.21.15.76:0/2361871234 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f105c004750 con 0x7f106c06aba0 2024-03-23T15:41:36.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.519+0000 7f10697fa700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10540a21e0 0x7f10540a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:36.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.519+0000 7f106b7fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10540a21e0 0x7f10540a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:36.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.519+0000 7f10697fa700 1 -- 172.21.15.76:0/2361871234 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f105c014070 con 0x7f106c06aba0 2024-03-23T15:41:36.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.519+0000 7f106b7fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10540a21e0 0x7f10540a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f106000b580 tx=0x7f1060005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:36.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.522+0000 7f10697fa700 1 -- 172.21.15.76:0/2361871234 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f105c097130 con 0x7f106c06aba0 2024-03-23T15:41:36.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.716+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f106c1039c0 con 0x7f106c06aba0 2024-03-23T15:41:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.717+0000 7f10697fa700 1 -- 172.21.15.76:0/2361871234 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22203 (secure 0 0 0) 0x7f105c09a7f0 con 0x7f106c06aba0 2024-03-23T15:41:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:41:36.720 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":85,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:41:19.628233+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10540a21e0 msgr2=0x7f10540a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10540a21e0 0x7f10540a46a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f106000b580 tx=0x7f1060005fb0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06aba0 msgr2=0x7f106c19e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06aba0 0x7f106c19e400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e9c0 tx=0x7f105c00ecd0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 shutdown_connections 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f10540a21e0 0x7f10540a46a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f106c107640 0x7f106c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f106c06aba0 0x7f106c19e400 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 --2- 172.21.15.76:0/2361871234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f106c06b550 0x7f106c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 >> 172.21.15.76:0/2361871234 conn(0x7f106c077f60 msgr2=0x7f106c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 shutdown_connections 2024-03-23T15:41:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:36.720+0000 7f10725fe700 1 -- 172.21.15.76:0/2361871234 wait complete. 2024-03-23T15:41:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:36 smithi192 ceph-mon[36653]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-23T15:41:36.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:36 smithi076 ceph-mon[32176]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-23T15:41:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:36 smithi183 ceph-mon[36296]: pgmap v295: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-23T15:41:37.719 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-23T15:41:37.719 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-23T15:41:37.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-23T15:41:37.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-23T15:41:37.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-23T15:41:37.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-23T15:41:37.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-23T15:41:37.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-23T15:41:37.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-23T15:41:37.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-23T15:41:37.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-23T15:41:37.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-23T15:41:37.801 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:37 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2361871234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:37 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2361871234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:37.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:37 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2361871234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:41:38.424 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:38.468 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:38.490 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:38.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:38 smithi183 ceph-mon[36296]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-03-23T15:41:38.722 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:38 smithi076 ceph-mon[32176]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-03-23T15:41:38.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:38 smithi192 ceph-mon[36653]: pgmap v296: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-03-23T15:41:40.611 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:40.611 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:40.612 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:40.627 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:40 smithi076 ceph-mon[32176]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-23T15:41:40.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:40 smithi192 ceph-mon[36653]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-23T15:41:40.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:40 smithi183 ceph-mon[36296]: pgmap v297: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-23T15:41:41.041 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:41.044 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:41.045 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:42.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:42 smithi076 ceph-mon[32176]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-23T15:41:42.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:42 smithi192 ceph-mon[36653]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-23T15:41:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:42 smithi183 ceph-mon[36296]: pgmap v298: 97 pgs: 97 active+clean; 635 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-23T15:41:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:44 smithi192 ceph-mon[36653]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-23T15:41:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:44 smithi076 ceph-mon[32176]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-23T15:41:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:44 smithi183 ceph-mon[36296]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-23T15:41:45.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.544+0000 7fea5203c700 1 -- 172.21.15.76:0/1934635231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c1010e0 msgr2=0x7fea4c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:45.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.544+0000 7fea5203c700 1 --2- 172.21.15.76:0/1934635231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c1010e0 0x7fea4c101560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fea40009990 tx=0x7fea40009ca0 comp rx=0 tx=0).stop 2024-03-23T15:41:45.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.545+0000 7fea5203c700 1 -- 172.21.15.76:0/1934635231 shutdown_connections 2024-03-23T15:41:45.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.545+0000 7fea5203c700 1 --2- 172.21.15.76:0/1934635231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea4c101aa0 0x7fea4c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:45.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.545+0000 7fea5203c700 1 --2- 172.21.15.76:0/1934635231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea4c107190 0x7fea4c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:45.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.545+0000 7fea5203c700 1 --2- 172.21.15.76:0/1934635231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c1010e0 0x7fea4c101560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:45.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.545+0000 7fea5203c700 1 -- 172.21.15.76:0/1934635231 >> 172.21.15.76:0/1934635231 conn(0x7fea4c078110 msgr2=0x7fea4c078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:45.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.545+0000 7fea5203c700 1 -- 172.21.15.76:0/1934635231 shutdown_connections 2024-03-23T15:41:45.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.545+0000 7fea5203c700 1 -- 172.21.15.76:0/1934635231 wait complete. 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 Processor -- start 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 -- start start 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea4c1010e0 0x7fea4c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea4c101aa0 0x7fea4c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c107190 0x7fea4c1a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c19a2e0 con 0x7fea4c107190 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1a4aa0 con 0x7fea4c101aa0 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea5203c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1a4be0 con 0x7fea4c1010e0 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c107190 0x7fea4c1a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:45.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea4c1010e0 0x7fea4c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c107190 0x7fea4c1a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40964/0 (socket says 172.21.15.76:40964) 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea4bfff700 1 -- 172.21.15.76:0/1795715529 learned_addr learned my addr 172.21.15.76:0/1795715529 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea4b7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea4c1010e0 0x7fea4c1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:54806/0 (socket says 172.21.15.76:54806) 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.546+0000 7fea4affd700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea4c101aa0 0x7fea4c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4b7fe700 1 -- 172.21.15.76:0/1795715529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea4c101aa0 msgr2=0x7fea4c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4b7fe700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea4c101aa0 0x7fea4c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4b7fe700 1 -- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c107190 msgr2=0x7fea4c1a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4b7fe700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c107190 0x7fea4c1a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4b7fe700 1 -- 172.21.15.76:0/1795715529 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea400095f0 con 0x7fea4c1010e0 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4bfff700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fea4c107190 0x7fea4c1a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4affd700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea4c101aa0 0x7fea4c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea4b7fe700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea4c1010e0 0x7fea4c1994e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e9f0 tx=0x7fea3c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:45.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00c9c0 con 0x7fea4c1010e0 2024-03-23T15:41:45.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c1a4ec0 con 0x7fea4c1010e0 2024-03-23T15:41:45.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea3c010430 con 0x7fea4c1010e0 2024-03-23T15:41:45.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c018660 con 0x7fea4c1010e0 2024-03-23T15:41:45.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.547+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c1a52d0 con 0x7fea4c1010e0 2024-03-23T15:41:45.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.548+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea4c1027d0 con 0x7fea4c1010e0 2024-03-23T15:41:45.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.549+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fea3c010910 con 0x7fea4c1010e0 2024-03-23T15:41:45.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.550+0000 7fea48ff9700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fea340a21e0 0x7fea340a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:45.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.550+0000 7fea4affd700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fea340a21e0 0x7fea340a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:45.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.550+0000 7fea4affd700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fea340a21e0 0x7fea340a46a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea4000b2e0 tx=0x7fea40005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.551+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fea3c014070 con 0x7fea4c1010e0 2024-03-23T15:41:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.551+0000 7fea48ff9700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] conn(0x7fea340ab9c0 0x7fea340adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.551+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 --> [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea340ae490 con 0x7fea340ab9c0 2024-03-23T15:41:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.551+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fea3c097530 con 0x7fea4c1010e0 2024-03-23T15:41:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.551+0000 7fea4bfff700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] conn(0x7fea340ab9c0 0x7fea340adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.552+0000 7fea4bfff700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] conn(0x7fea340ab9c0 0x7fea340adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:45.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.552+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== osd.6 v2:172.21.15.76:6818/2821822561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea340ae490 con 0x7fea340ab9c0 2024-03-23T15:41:45.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.568+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 --> [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea4c111430 con 0x7fea340ab9c0 2024-03-23T15:41:45.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea48ff9700 1 -- 172.21.15.76:0/1795715529 <== osd.6 v2:172.21.15.76:6818/2821822561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea4c111430 con 0x7fea340ab9c0 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] conn(0x7fea340ab9c0 msgr2=0x7fea340adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] conn(0x7fea340ab9c0 0x7fea340adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fea340a21e0 msgr2=0x7fea340a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fea340a21e0 0x7fea340a46a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea4000b2e0 tx=0x7fea40005960 comp rx=0 tx=0).stop 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea4c1010e0 msgr2=0x7fea4c1994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 --2- 172.21.15.76:0/1795715529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea4c1010e0 0x7fea4c1994e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e9f0 tx=0x7fea3c00ed00 comp rx=0 tx=0).stop 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea4bfff700 1 -- 172.21.15.76:0/1795715529 reap_dead start 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 shutdown_connections 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 >> 172.21.15.76:0/1795715529 conn(0x7fea4c078110 msgr2=0x7fea4c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 shutdown_connections 2024-03-23T15:41:45.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:45.569+0000 7fea5203c700 1 -- 172.21.15.76:0/1795715529 wait complete. 2024-03-23T15:41:46.430 INFO:teuthology.orchestra.run.smithi076.stdout:197568495661 2024-03-23T15:41:46.430 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-23T15:41:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:46 smithi192 ceph-mon[36653]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:41:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:46 smithi076 ceph-mon[32176]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:41:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:46 smithi183 ceph-mon[36296]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:41:47.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.266+0000 7f74da516700 1 -- 172.21.15.76:0/2836903229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 msgr2=0x7f74d4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.266+0000 7f74da516700 1 --2- 172.21.15.76:0/2836903229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 0x7f74d4107650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009990 tx=0x7f74c4009ca0 comp rx=0 tx=0).stop 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.266+0000 7f74da516700 1 -- 172.21.15.76:0/2836903229 shutdown_connections 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.266+0000 7f74da516700 1 --2- 172.21.15.76:0/2836903229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d41011b0 0x7f74d410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.266+0000 7f74da516700 1 --2- 172.21.15.76:0/2836903229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 0x7f74d4107650 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.266+0000 7f74da516700 1 --2- 172.21.15.76:0/2836903229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d41007f0 0x7f74d4100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.266+0000 7f74da516700 1 -- 172.21.15.76:0/2836903229 >> 172.21.15.76:0/2836903229 conn(0x7f74d4077700 msgr2=0x7f74d40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.267+0000 7f74da516700 1 -- 172.21.15.76:0/2836903229 shutdown_connections 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.267+0000 7f74da516700 1 -- 172.21.15.76:0/2836903229 wait complete. 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.267+0000 7f74da516700 1 Processor -- start 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.267+0000 7f74da516700 1 -- start start 2024-03-23T15:41:47.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74da516700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d41007f0 0x7f74d419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74da516700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d41011b0 0x7f74d419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74da516700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 0x7f74d4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74da516700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4198e00 con 0x7f74d4107270 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74da516700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4198f70 con 0x7f74d41011b0 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74da516700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4199130 con 0x7f74d41007f0 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d8ab3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 0x7f74d4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d37fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d41011b0 0x7f74d419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d8ab3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 0x7f74d4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40990/0 (socket says 172.21.15.76:40990) 2024-03-23T15:41:47.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d8ab3700 1 -- 172.21.15.76:0/1827601350 learned_addr learned my addr 172.21.15.76:0/1827601350 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d3fff700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d41007f0 0x7f74d419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d37fe700 1 -- 172.21.15.76:0/1827601350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d41007f0 msgr2=0x7f74d419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d37fe700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d41007f0 0x7f74d419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d37fe700 1 -- 172.21.15.76:0/1827601350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 msgr2=0x7f74d4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d37fe700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74d4107270 0x7f74d4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d37fe700 1 -- 172.21.15.76:0/1827601350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c40095f0 con 0x7f74d41011b0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d37fe700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d41011b0 0x7f74d419eac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e9f0 tx=0x7f74c800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8004220 con 0x7f74d41011b0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d4199410 con 0x7f74d41011b0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8004d10 con 0x7f74d41011b0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.268+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8005020 con 0x7f74d41011b0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.269+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d41a5500 con 0x7f74d41011b0 2024-03-23T15:41:47.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.269+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f74d41126a0 con 0x7f74d41011b0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.271+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f74c8004830 con 0x7f74d41011b0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d17fa700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74bc0a21e0 0x7f74bc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f74c8014070 con 0x7f74d41011b0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d3fff700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74bc0a21e0 0x7f74bc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d17fa700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] conn(0x7f74bc0ab9c0 0x7f74bc0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 --> [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74bc0ae490 con 0x7f74bc0ab9c0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f74c80ce290 con 0x7f74d41011b0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d8ab3700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] conn(0x7f74bc0ab9c0 0x7f74bc0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.272+0000 7f74d3fff700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74bc0a21e0 0x7f74bc0a46a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f74c400b2e0 tx=0x7f74c4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.273+0000 7f74d8ab3700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] conn(0x7f74bc0ab9c0 0x7f74bc0adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.273+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== osd.4 v2:172.21.15.183:6808/3795301035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74bc0ae490 con 0x7f74bc0ab9c0 2024-03-23T15:41:47.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.291+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 --> [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f74d4062890 con 0x7f74bc0ab9c0 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74d17fa700 1 -- 172.21.15.76:0/1827601350 <== osd.4 v2:172.21.15.183:6808/3795301035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f74d4062890 con 0x7f74bc0ab9c0 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] conn(0x7f74bc0ab9c0 msgr2=0x7f74bc0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] conn(0x7f74bc0ab9c0 0x7f74bc0adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74bc0a21e0 msgr2=0x7f74bc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74bc0a21e0 0x7f74bc0a46a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f74c400b2e0 tx=0x7f74c4005960 comp rx=0 tx=0).stop 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d41011b0 msgr2=0x7f74d419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 --2- 172.21.15.76:0/1827601350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d41011b0 0x7f74d419eac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f74c800e9f0 tx=0x7f74c800ed00 comp rx=0 tx=0).stop 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74d8ab3700 1 -- 172.21.15.76:0/1827601350 reap_dead start 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 shutdown_connections 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 >> 172.21.15.76:0/1827601350 conn(0x7f74d4077700 msgr2=0x7f74d4105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 shutdown_connections 2024-03-23T15:41:47.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:47.292+0000 7f74da516700 1 -- 172.21.15.76:0/1827601350 wait complete. 2024-03-23T15:41:48.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:48 smithi183 ceph-mon[36296]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-23T15:41:49.049 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:48 smithi076 ceph-mon[32176]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-23T15:41:49.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:48 smithi192 ceph-mon[36653]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 -- 172.21.15.76:0/1906609832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1013e0 msgr2=0x7f7c4c101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 --2- 172.21.15.76:0/1906609832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1013e0 0x7f7c4c101860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009990 tx=0x7f7c3c009ca0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 -- 172.21.15.76:0/1906609832 shutdown_connections 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 --2- 172.21.15.76:0/1906609832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c4c101da0 0x7f7c4c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 --2- 172.21.15.76:0/1906609832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c4c1072f0 0x7f7c4c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 --2- 172.21.15.76:0/1906609832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1013e0 0x7f7c4c101860 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 -- 172.21.15.76:0/1906609832 >> 172.21.15.76:0/1906609832 conn(0x7f7c4c077f60 msgr2=0x7f7c4c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.558+0000 7f7c533e1700 1 -- 172.21.15.76:0/1906609832 shutdown_connections 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 -- 172.21.15.76:0/1906609832 wait complete. 2024-03-23T15:41:50.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 Processor -- start 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 -- start start 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c4c1013e0 0x7f7c4c10e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c4c101da0 0x7f7c4c10eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1072f0 0x7f7c4c10f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1a6bb0 con 0x7f7c4c1072f0 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1a6d20 con 0x7f7c4c101da0 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c5197e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1072f0 0x7f7c4c10f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c533e1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1a6ee0 con 0x7f7c4c1013e0 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c5197e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1072f0 0x7f7c4c10f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41028/0 (socket says 172.21.15.76:41028) 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c5097c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c4c101da0 0x7f7c4c10eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c5197e700 1 -- 172.21.15.76:0/3047748236 learned_addr learned my addr 172.21.15.76:0/3047748236 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:50.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.559+0000 7f7c5117d700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c4c1013e0 0x7f7c4c10e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c5197e700 1 -- 172.21.15.76:0/3047748236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c4c1013e0 msgr2=0x7f7c4c10e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c5197e700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c4c1013e0 0x7f7c4c10e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c5197e700 1 -- 172.21.15.76:0/3047748236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c4c101da0 msgr2=0x7f7c4c10eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c5197e700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c4c101da0 0x7f7c4c10eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c5197e700 1 -- 172.21.15.76:0/3047748236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c0095f0 con 0x7f7c4c1072f0 2024-03-23T15:41:50.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c5197e700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1072f0 0x7f7c4c10f3e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400efc0 tx=0x7f7c4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44018070 con 0x7f7c4c1072f0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4400f040 con 0x7f7c4c1072f0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c1a7160 con 0x7f7c4c1072f0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1a76b0 con 0x7f7c4c1072f0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.560+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400cc80 con 0x7f7c4c1072f0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.561+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c30001030 con 0x7f7c4c1072f0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.562+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f7c44004ad0 con 0x7f7c4c1072f0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c427fc700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7c380a22e0 0x7f7c380a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c5117d700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7c380a22e0 0x7f7c380a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7c440cdf30 con 0x7f7c4c1072f0 2024-03-23T15:41:50.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c427fc700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] conn(0x7f7c380abac0 0x7f7c380adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 --> [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c380ae590 con 0x7f7c380abac0 2024-03-23T15:41:50.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f7c440ce2e0 con 0x7f7c4c1072f0 2024-03-23T15:41:50.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c5117d700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7c380a22e0 0x7f7c380a47a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009be0 tx=0x7f7c48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:50.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.563+0000 7f7c5097c700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] conn(0x7f7c380abac0 0x7f7c380adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.564+0000 7f7c5097c700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] conn(0x7f7c380abac0 0x7f7c380adee0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:50.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.565+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== osd.10 v2:172.21.15.183:6824/596098792 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c380ae590 con 0x7f7c380abac0 2024-03-23T15:41:50.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.581+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 --> [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7c30002dc0 con 0x7f7c380abac0 2024-03-23T15:41:50.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.581+0000 7f7c427fc700 1 -- 172.21.15.76:0/3047748236 <== osd.10 v2:172.21.15.183:6824/596098792 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7c30002dc0 con 0x7f7c380abac0 2024-03-23T15:41:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 >> [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] conn(0x7f7c380abac0 msgr2=0x7f7c380adee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] conn(0x7f7c380abac0 0x7f7c380adee0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7c380a22e0 msgr2=0x7f7c380a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f7c380a22e0 0x7f7c380a47a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009be0 tx=0x7f7c48009260 comp rx=0 tx=0).stop 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1072f0 msgr2=0x7f7c4c10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 --2- 172.21.15.76:0/3047748236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7c4c1072f0 0x7f7c4c10f3e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400efc0 tx=0x7f7c4400c3a0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c5197e700 1 -- 172.21.15.76:0/3047748236 reap_dead start 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 shutdown_connections 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 >> 172.21.15.76:0/3047748236 conn(0x7f7c4c077f60 msgr2=0x7f7c4c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 shutdown_connections 2024-03-23T15:41:50.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.582+0000 7f7c533e1700 1 -- 172.21.15.76:0/3047748236 wait complete. 2024-03-23T15:41:50.675 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:50 smithi076 ceph-mon[32176]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:41:50.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/1758346289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34102250 msgr2=0x7fbb341026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 --2- 172.21.15.76:0/1758346289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34102250 0x7fbb341026d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb240099c0 tx=0x7fbb24009cd0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/1758346289 shutdown_connections 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 --2- 172.21.15.76:0/1758346289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb34102c10 0x7fbb3410f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 --2- 172.21.15.76:0/1758346289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb34108230 0x7fbb34108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 --2- 172.21.15.76:0/1758346289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34102250 0x7fbb341026d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/1758346289 >> 172.21.15.76:0/1758346289 conn(0x7fbb34079330 msgr2=0x7fbb34079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/1758346289 shutdown_connections 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.772+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/1758346289 wait complete. 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 Processor -- start 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 -- start start 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb34102250 0x7fbb34198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb34102c10 0x7fbb34198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34108230 0x7fbb3419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3419d5d0 con 0x7fbb34108230 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3419d740 con 0x7fbb34102250 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.773+0000 7fbb3b9c2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3419d900 con 0x7fbb34102c10 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34108230 0x7fbb3419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb3975e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb34102250 0x7fbb34198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34108230 0x7fbb3419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41046/0 (socket says 172.21.15.76:41046) 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 -- 172.21.15.76:0/2020886824 learned_addr learned my addr 172.21.15.76:0/2020886824 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb38f5d700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb34102c10 0x7fbb34198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 -- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb34102c10 msgr2=0x7fbb34198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb34102c10 0x7fbb34198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 -- 172.21.15.76:0/2020886824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb34102250 msgr2=0x7fbb34198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb34102250 0x7fbb34198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 -- 172.21.15.76:0/2020886824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb240095f0 con 0x7fbb34108230 2024-03-23T15:41:50.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb3975e700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb34102250 0x7fbb34198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:50.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb39f5f700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34108230 0x7fbb3419cfc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00ef30 tx=0x7fbb2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:50.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c018070 con 0x7fbb34108230 2024-03-23T15:41:50.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb2c00f040 con 0x7fbb34108230 2024-03-23T15:41:50.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00cd50 con 0x7fbb34108230 2024-03-23T15:41:50.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3419db80 con 0x7fbb34108230 2024-03-23T15:41:50.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.774+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3419de30 con 0x7fbb34108230 2024-03-23T15:41:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.775+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb341038e0 con 0x7fbb34108230 2024-03-23T15:41:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.776+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fbb2c007500 con 0x7fbb34108230 2024-03-23T15:41:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb2a7fc700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbb200a21e0 0x7fbb200a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb3975e700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbb200a21e0 0x7fbb200a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbb2c0ce060 con 0x7fbb34108230 2024-03-23T15:41:50.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb2a7fc700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] conn(0x7fbb200ab9c0 0x7fbb200adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 --> [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb200ae490 con 0x7fbb200ab9c0 2024-03-23T15:41:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fbb2c0ce480 con 0x7fbb34108230 2024-03-23T15:41:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb3975e700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbb200a21e0 0x7fbb200a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009fd0 tx=0x7fbb30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.777+0000 7fbb38f5d700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] conn(0x7fbb200ab9c0 0x7fbb200adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.778+0000 7fbb38f5d700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] conn(0x7fbb200ab9c0 0x7fbb200adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:50.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.778+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== osd.8 v2:172.21.15.192:6816/2740228504 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb200ae490 con 0x7fbb200ab9c0 2024-03-23T15:41:50.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.795+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 --> [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb34112540 con 0x7fbb200ab9c0 2024-03-23T15:41:50.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb2a7fc700 1 -- 172.21.15.76:0/2020886824 <== osd.8 v2:172.21.15.192:6816/2740228504 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb34112540 con 0x7fbb200ab9c0 2024-03-23T15:41:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] conn(0x7fbb200ab9c0 msgr2=0x7fbb200adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] conn(0x7fbb200ab9c0 0x7fbb200adde0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbb200a21e0 msgr2=0x7fbb200a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbb200a21e0 0x7fbb200a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009fd0 tx=0x7fbb30009260 comp rx=0 tx=0).stop 2024-03-23T15:41:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34108230 msgr2=0x7fbb3419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 --2- 172.21.15.76:0/2020886824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb34108230 0x7fbb3419cfc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00ef30 tx=0x7fbb2c00c3a0 comp rx=0 tx=0).stop 2024-03-23T15:41:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb39f5f700 1 -- 172.21.15.76:0/2020886824 reap_dead start 2024-03-23T15:41:50.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 shutdown_connections 2024-03-23T15:41:50.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 >> 172.21.15.76:0/2020886824 conn(0x7fbb34079330 msgr2=0x7fbb34100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:50.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 shutdown_connections 2024-03-23T15:41:50.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:50.796+0000 7fbb3b9c2700 1 -- 172.21.15.76:0/2020886824 wait complete. 2024-03-23T15:41:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:50 smithi183 ceph-mon[36296]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:41:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:50 smithi192 ceph-mon[36653]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:41:51.870 INFO:teuthology.orchestra.run.smithi076.stdout:309237645337 2024-03-23T15:41:51.871 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-23T15:41:51.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 -- 172.21.15.76:0/288585618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 msgr2=0x7ff86810f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:51.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 --2- 172.21.15.76:0/288585618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 0x7ff86810f130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b0a0 tx=0x7ff86000b3b0 comp rx=0 tx=0).stop 2024-03-23T15:41:51.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 -- 172.21.15.76:0/288585618 shutdown_connections 2024-03-23T15:41:51.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 --2- 172.21.15.76:0/288585618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 0x7ff86810f130 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:51.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 --2- 172.21.15.76:0/288585618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86806abc0 0x7ff86806afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 --2- 172.21.15.76:0/288585618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86806b570 0x7ff868106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 -- 172.21.15.76:0/288585618 >> 172.21.15.76:0/288585618 conn(0x7ff868078200 msgr2=0x7ff868078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 -- 172.21.15.76:0/288585618 shutdown_connections 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.518+0000 7ff86e4aa700 1 -- 172.21.15.76:0/288585618 wait complete. 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 Processor -- start 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 -- start start 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86806abc0 0x7ff86819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86806b570 0x7ff86819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 0x7ff868198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff868198e00 con 0x7ff868107560 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff868198f70 con 0x7ff86806b570 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86e4aa700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff868199130 con 0x7ff86806abc0 2024-03-23T15:41:51.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 0x7ff868198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 0x7ff868198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41018/0 (socket says 172.21.15.76:41018) 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 -- 172.21.15.76:0/148809636 learned_addr learned my addr 172.21.15.76:0/148809636 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff867fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86806abc0 0x7ff86819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff8677fe700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86806b570 0x7ff86819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 -- 172.21.15.76:0/148809636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86806abc0 msgr2=0x7ff86819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86806abc0 0x7ff86819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 -- 172.21.15.76:0/148809636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86806b570 msgr2=0x7ff86819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86806b570 0x7ff86819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:51.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.519+0000 7ff86ca47700 1 -- 172.21.15.76:0/148809636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff860009d00 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.520+0000 7ff86ca47700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 0x7ff868198650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b070 tx=0x7ff860007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.520+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86000e040 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.520+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff860004e60 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.520+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8600052a0 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.520+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681993b0 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.520+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681a5500 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.521+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8681125e0 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.522+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7ff86001d470 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.522+0000 7ff8657fa700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8500a2260 0x7ff8500a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff867fff700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8500a2260 0x7ff8500a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff8600cf630 con 0x7ff868107560 2024-03-23T15:41:51.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff8657fa700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] conn(0x7ff8500aba40 0x7ff8500ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 --> [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8500ae510 con 0x7ff8500aba40 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff8600cf9e0 con 0x7ff868107560 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff867fff700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8500a2260 0x7ff8500a4720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009c60 tx=0x7ff85c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff8677fe700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] conn(0x7ff8500aba40 0x7ff8500ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.523+0000 7ff8677fe700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] conn(0x7ff8500aba40 0x7ff8500ade60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.524+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== osd.11 v2:172.21.15.192:6824/603123871 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8500ae510 con 0x7ff8500aba40 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 --> [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8680ff9f0 con 0x7ff8500aba40 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff8657fa700 1 -- 172.21.15.76:0/148809636 <== osd.11 v2:172.21.15.192:6824/603123871 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff8680ff9f0 con 0x7ff8500aba40 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 >> [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] conn(0x7ff8500aba40 msgr2=0x7ff8500ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86e4aa700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] conn(0x7ff8500aba40 0x7ff8500ade60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8500a2260 msgr2=0x7ff8500a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86e4aa700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff8500a2260 0x7ff8500a4720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009c60 tx=0x7ff85c009260 comp rx=0 tx=0).stop 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 msgr2=0x7ff868198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:51.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86e4aa700 1 --2- 172.21.15.76:0/148809636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff868107560 0x7ff868198650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b070 tx=0x7ff860007870 comp rx=0 tx=0).stop 2024-03-23T15:41:51.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.542+0000 7ff86ca47700 1 -- 172.21.15.76:0/148809636 reap_dead start 2024-03-23T15:41:51.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.543+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 shutdown_connections 2024-03-23T15:41:51.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.543+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 >> 172.21.15.76:0/148809636 conn(0x7ff868078200 msgr2=0x7ff8680ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:51.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.543+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 shutdown_connections 2024-03-23T15:41:51.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:48.543+0000 7ff86e4aa700 1 -- 172.21.15.76:0/148809636 wait complete. 2024-03-23T15:41:52.673 INFO:teuthology.orchestra.run.smithi076.stdout:343597383699 2024-03-23T15:41:52.674 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-23T15:41:52.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:52 smithi183 ceph-mon[36296]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:41:53.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:52 smithi192 ceph-mon[36653]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:41:53.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:52 smithi076 ceph-mon[32176]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:41:53.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.301+0000 7fd145367700 1 -- 172.21.15.76:0/2118726918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140101390 msgr2=0x7fd140101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:53.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 --2- 172.21.15.76:0/2118726918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140101390 0x7fd140101770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd130009990 tx=0x7fd130009ca0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 -- 172.21.15.76:0/2118726918 shutdown_connections 2024-03-23T15:41:53.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 --2- 172.21.15.76:0/2118726918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd140106420 0x7fd14010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 --2- 172.21.15.76:0/2118726918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140101390 0x7fd140101770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 --2- 172.21.15.76:0/2118726918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd140101d40 0x7fd140105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 -- 172.21.15.76:0/2118726918 >> 172.21.15.76:0/2118726918 conn(0x7fd140077120 msgr2=0x7fd140077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:53.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 -- 172.21.15.76:0/2118726918 shutdown_connections 2024-03-23T15:41:53.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 -- 172.21.15.76:0/2118726918 wait complete. 2024-03-23T15:41:53.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 Processor -- start 2024-03-23T15:41:53.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.302+0000 7fd145367700 1 -- start start 2024-03-23T15:41:53.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd140101390 0x7fd1401a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd140101d40 0x7fd1401a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140106420 0x7fd14019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd14019af20 con 0x7fd140106420 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd14019b090 con 0x7fd140101390 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd14019b250 con 0x7fd140101d40 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140106420 0x7fd14019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140106420 0x7fd14019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41056/0 (socket says 172.21.15.76:41056) 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13effd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd140101390 0x7fd1401a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 -- 172.21.15.76:0/396425301 learned_addr learned my addr 172.21.15.76:0/396425301 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13e7fc700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd140101d40 0x7fd1401a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:53.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 -- 172.21.15.76:0/396425301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd140101d40 msgr2=0x7fd1401a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd140101d40 0x7fd1401a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 -- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd140101390 msgr2=0x7fd1401a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd140101390 0x7fd1401a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 -- 172.21.15.76:0/396425301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1300095f0 con 0x7fd140106420 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13effd700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd140101390 0x7fd1401a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd13f7fe700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140106420 0x7fd14019a770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd13800ef30 tx=0x7fd13800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138018070 con 0x7fd140106420 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14019b4d0 con 0x7fd140106420 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd13800f040 con 0x7fd140106420 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13800cd50 con 0x7fd140106420 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.303+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1401a7620 con 0x7fd140106420 2024-03-23T15:41:53.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.304+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd140111470 con 0x7fd140106420 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.305+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fd138007500 con 0x7fd140106420 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.306+0000 7fd127fff700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1280a21e0 0x7fd1280a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.306+0000 7fd13effd700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1280a21e0 0x7fd1280a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.306+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd1380ce150 con 0x7fd140106420 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.306+0000 7fd127fff700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] conn(0x7fd1280ab9c0 0x7fd1280adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.306+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 --> [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1280ae490 con 0x7fd1280ab9c0 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.306+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd1380ce5f0 con 0x7fd140106420 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.307+0000 7fd13effd700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1280a21e0 0x7fd1280a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd13000b2e0 tx=0x7fd130005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:53.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.307+0000 7fd13e7fc700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] conn(0x7fd1280ab9c0 0x7fd1280adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:53.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.307+0000 7fd13e7fc700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] conn(0x7fd1280ab9c0 0x7fd1280adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:53.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.308+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== osd.7 v2:172.21.15.183:6816/336323954 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1280ae490 con 0x7fd1280ab9c0 2024-03-23T15:41:53.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.324+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 --> [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd140100be0 con 0x7fd1280ab9c0 2024-03-23T15:41:53.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd127fff700 1 -- 172.21.15.76:0/396425301 <== osd.7 v2:172.21.15.183:6816/336323954 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd140100be0 con 0x7fd1280ab9c0 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] conn(0x7fd1280ab9c0 msgr2=0x7fd1280adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] conn(0x7fd1280ab9c0 0x7fd1280adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1280a21e0 msgr2=0x7fd1280a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd1280a21e0 0x7fd1280a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd13000b2e0 tx=0x7fd130005960 comp rx=0 tx=0).stop 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140106420 msgr2=0x7fd14019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 --2- 172.21.15.76:0/396425301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd140106420 0x7fd14019a770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd13800ef30 tx=0x7fd13800c3a0 comp rx=0 tx=0).stop 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd13f7fe700 1 -- 172.21.15.76:0/396425301 reap_dead start 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 shutdown_connections 2024-03-23T15:41:53.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.325+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 >> 172.21.15.76:0/396425301 conn(0x7fd140077120 msgr2=0x7fd1400ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:53.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.326+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 shutdown_connections 2024-03-23T15:41:53.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:53.326+0000 7fd145367700 1 -- 172.21.15.76:0/396425301 wait complete. 2024-03-23T15:41:54.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:54 smithi076 ceph-mon[32176]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-23T15:41:54.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:54 smithi183 ceph-mon[36296]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-23T15:41:55.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:54 smithi192 ceph-mon[36653]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-23T15:41:55.623 INFO:teuthology.orchestra.run.smithi076.stdout:219043332136 2024-03-23T15:41:55.623 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-23T15:41:56.029 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:56.033 INFO:teuthology.orchestra.run.smithi076.stdout:141733920823 2024-03-23T15:41:56.034 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-23T15:41:56.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.237+0000 7f82d8565700 1 -- 172.21.15.76:0/2386620908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01007f0 msgr2=0x7f82d0100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:56.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.237+0000 7f82d8565700 1 --2- 172.21.15.76:0/2386620908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01007f0 0x7f82d0100c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009990 tx=0x7f82c0009ca0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.237+0000 7f82d8565700 1 -- 172.21.15.76:0/2386620908 shutdown_connections 2024-03-23T15:41:56.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.237+0000 7f82d8565700 1 --2- 172.21.15.76:0/2386620908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82d01011b0 0x7f82d010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.237+0000 7f82d8565700 1 --2- 172.21.15.76:0/2386620908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0107270 0x7f82d0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.237+0000 7f82d8565700 1 --2- 172.21.15.76:0/2386620908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01007f0 0x7f82d0100c70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.237+0000 7f82d8565700 1 -- 172.21.15.76:0/2386620908 >> 172.21.15.76:0/2386620908 conn(0x7f82d0077700 msgr2=0x7f82d00feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:56.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 -- 172.21.15.76:0/2386620908 shutdown_connections 2024-03-23T15:41:56.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 -- 172.21.15.76:0/2386620908 wait complete. 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 Processor -- start 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 -- start start 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82d01007f0 0x7f82d019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01011b0 0x7f82d019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0107270 0x7f82d01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0198e50 con 0x7f82d01011b0 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0198fc0 con 0x7f82d0107270 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.238+0000 7f82d8565700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0199180 con 0x7f82d01007f0 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d5b00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01011b0 0x7f82d019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d5b00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01011b0 0x7f82d019eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37460/0 (socket says 172.21.15.76:37460) 2024-03-23T15:41:56.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d5b00700 1 -- 172.21.15.76:0/1220346762 learned_addr learned my addr 172.21.15.76:0/1220346762 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6301700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82d01007f0 0x7f82d019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6b02700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0107270 0x7f82d01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6301700 1 -- 172.21.15.76:0/1220346762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0107270 msgr2=0x7f82d01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6301700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82d0107270 0x7f82d01986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6301700 1 -- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01011b0 msgr2=0x7f82d019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6301700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01011b0 0x7f82d019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6301700 1 -- 172.21.15.76:0/1220346762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c00095f0 con 0x7f82d01007f0 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d5b00700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82d01011b0 0x7f82d019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d6301700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82d01007f0 0x7f82d019e5d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009db0 tx=0x7f82cc00c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc012050 con 0x7f82d01007f0 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc012690 con 0x7f82d01007f0 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc01a6d0 con 0x7f82d01007f0 2024-03-23T15:41:56.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.239+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d0199400 con 0x7f82d01007f0 2024-03-23T15:41:56.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.240+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d0101ad0 con 0x7f82d01007f0 2024-03-23T15:41:56.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.241+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82d01126d0 con 0x7f82d01007f0 2024-03-23T15:41:56.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.242+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f82cc0121b0 con 0x7f82d01007f0 2024-03-23T15:41:56.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.243+0000 7f82c77fe700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82bc0a2260 0x7f82bc0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:56.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.243+0000 7f82d5b00700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82bc0a2260 0x7f82bc0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:56.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.243+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f82cc0d1ff0 con 0x7f82d01007f0 2024-03-23T15:41:56.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.243+0000 7f82c77fe700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] conn(0x7f82bc0aba40 0x7f82bc0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:56.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.243+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 --> [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82bc0ae510 con 0x7f82bc0aba40 2024-03-23T15:41:56.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.243+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f82cc0d23e0 con 0x7f82d01007f0 2024-03-23T15:41:56.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.244+0000 7f82d5b00700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82bc0a2260 0x7f82bc0a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82d0199e60 tx=0x7f82c000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:56.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.244+0000 7f82d6b02700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] conn(0x7f82bc0aba40 0x7f82bc0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:56.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.244+0000 7f82d6b02700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] conn(0x7f82bc0aba40 0x7f82bc0ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:56.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.245+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== osd.2 v2:172.21.15.192:6800/4182864338 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82bc0ae510 con 0x7f82bc0aba40 2024-03-23T15:41:56.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.261+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 --> [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82d0062890 con 0x7f82bc0aba40 2024-03-23T15:41:56.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82c77fe700 1 -- 172.21.15.76:0/1220346762 <== osd.2 v2:172.21.15.192:6800/4182864338 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f82d0062890 con 0x7f82bc0aba40 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] conn(0x7f82bc0aba40 msgr2=0x7f82bc0ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] conn(0x7f82bc0aba40 0x7f82bc0ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82bc0a2260 msgr2=0x7f82bc0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82bc0a2260 0x7f82bc0a4720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82d0199e60 tx=0x7f82c000b260 comp rx=0 tx=0).stop 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82d01007f0 msgr2=0x7f82d019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 --2- 172.21.15.76:0/1220346762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82d01007f0 0x7f82d019e5d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009db0 tx=0x7f82cc00c5b0 comp rx=0 tx=0).stop 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d6b02700 1 -- 172.21.15.76:0/1220346762 reap_dead start 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 shutdown_connections 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.262+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 >> 172.21.15.76:0/1220346762 conn(0x7f82d0077700 msgr2=0x7f82d0105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.263+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 shutdown_connections 2024-03-23T15:41:56.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:56.263+0000 7f82d8565700 1 -- 172.21.15.76:0/1220346762 wait complete. 2024-03-23T15:41:56.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:56 smithi076 ceph-mon[32176]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-23T15:41:56.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:56 smithi192 ceph-mon[36653]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-23T15:41:56.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:56 smithi183 ceph-mon[36296]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-23T15:41:57.100 INFO:teuthology.orchestra.run.smithi076.stdout:253403070499 2024-03-23T15:41:57.100 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-23T15:41:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 -- 172.21.15.76:0/196648823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24107140 msgr2=0x7f5d24107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 --2- 172.21.15.76:0/196648823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24107140 0x7f5d24107520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009990 tx=0x7f5d14009ca0 comp rx=0 tx=0).stop 2024-03-23T15:41:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 -- 172.21.15.76:0/196648823 shutdown_connections 2024-03-23T15:41:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 --2- 172.21.15.76:0/196648823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d24101aa0 0x7f5d2410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 --2- 172.21.15.76:0/196648823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24107140 0x7f5d24107520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 --2- 172.21.15.76:0/196648823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24101100 0x7f5d24101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 -- 172.21.15.76:0/196648823 >> 172.21.15.76:0/196648823 conn(0x7f5d24078030 msgr2=0x7f5d24078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 -- 172.21.15.76:0/196648823 shutdown_connections 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.986+0000 7f5d29714700 1 -- 172.21.15.76:0/196648823 wait complete. 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 Processor -- start 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 -- start start 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24101100 0x7f5d24102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24101aa0 0x7f5d241032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d24107140 0x7f5d24103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5d241a6ae0 con 0x7f5d24101aa0 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d241a6c50 con 0x7f5d24101100 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d29714700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5d241a6e10 con 0x7f5d24107140 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24101aa0 0x7f5d241032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24101aa0 0x7f5d241032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37478/0 (socket says 172.21.15.76:37478) 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 -- 172.21.15.76:0/3663898820 learned_addr learned my addr 172.21.15.76:0/3663898820 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d22ffd700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24101100 0x7f5d24102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d237fe700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d24107140 0x7f5d24103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 -- 172.21.15.76:0/3663898820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d24107140 msgr2=0x7f5d24103830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d24107140 0x7f5d24103830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 -- 172.21.15.76:0/3663898820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24101100 msgr2=0x7f5d24102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24101100 0x7f5d24102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d227fc700 1 -- 172.21.15.76:0/3663898820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d140095f0 con 0x7f5d24101aa0 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.987+0000 7f5d22ffd700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24101100 0x7f5d24102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.988+0000 7f5d227fc700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24101aa0 0x7f5d241032f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800e9a0 tx=0x7f5d1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.988+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18009790 con 0x7f5d24101aa0 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.988+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d241a70f0 con 0x7f5d24101aa0 2024-03-23T15:41:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.988+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d18010430 con 0x7f5d24101aa0 2024-03-23T15:41:57.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.988+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18018680 con 0x7f5d24101aa0 2024-03-23T15:41:57.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.988+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d241a7640 con 0x7f5d24101aa0 2024-03-23T15:41:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.989+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d241113d0 con 0x7f5d24101aa0 2024-03-23T15:41:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.990+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f5d18010910 con 0x7f5d24101aa0 2024-03-23T15:41:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.990+0000 7f5d0bfff700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f5d0c0a2260 0x7f5d0c0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.990+0000 7f5d22ffd700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f5d0c0a2260 0x7f5d0c0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.990+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5d18014070 con 0x7f5d24101aa0 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.991+0000 7f5d0bfff700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] conn(0x7f5d0c0aba40 0x7f5d0c0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.991+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 --> [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d0c0ae510 con 0x7f5d0c0aba40 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.991+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f5d180d0360 con 0x7f5d24101aa0 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.991+0000 7f5d237fe700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] conn(0x7f5d0c0aba40 0x7f5d0c0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.991+0000 7f5d22ffd700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f5d0c0a2260 0x7f5d0c0a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400b2e0 tx=0x7f5d14005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.991+0000 7f5d237fe700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] conn(0x7f5d0c0aba40 0x7f5d0c0ade60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:41:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:57.992+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== osd.3 v2:172.21.15.76:6810/2500918591 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d0c0ae510 con 0x7f5d0c0aba40 2024-03-23T15:41:58.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.008+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 --> [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d241050b0 con 0x7f5d0c0aba40 2024-03-23T15:41:58.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.008+0000 7f5d0bfff700 1 -- 172.21.15.76:0/3663898820 <== osd.3 v2:172.21.15.76:6810/2500918591 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5d241050b0 con 0x7f5d0c0aba40 2024-03-23T15:41:58.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] conn(0x7f5d0c0aba40 msgr2=0x7f5d0c0ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:58.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] conn(0x7f5d0c0aba40 0x7f5d0c0ade60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:41:58.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f5d0c0a2260 msgr2=0x7f5d0c0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f5d0c0a2260 0x7f5d0c0a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400b2e0 tx=0x7f5d14005960 comp rx=0 tx=0).stop 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24101aa0 msgr2=0x7f5d241032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 --2- 172.21.15.76:0/3663898820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d24101aa0 0x7f5d241032f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800e9a0 tx=0x7f5d1800ecb0 comp rx=0 tx=0).stop 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d237fe700 1 -- 172.21.15.76:0/3663898820 reap_dead start 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 shutdown_connections 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 >> 172.21.15.76:0/3663898820 conn(0x7f5d24078030 msgr2=0x7f5d24100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 shutdown_connections 2024-03-23T15:41:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:41:58.009+0000 7f5d29714700 1 -- 172.21.15.76:0/3663898820 wait complete. 2024-03-23T15:41:58.851 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:41:58 smithi076 ceph-mon[32176]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:41:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:41:58 smithi183 ceph-mon[36296]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:41:59.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:41:58 smithi192 ceph-mon[36653]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:41:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:41:59.940 INFO:teuthology.orchestra.run.smithi076.stdout:77309411395 2024-03-23T15:41:59.940 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-23T15:42:00.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.120+0000 7f323be7b700 1 -- 172.21.15.76:0/3717394306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 msgr2=0x7f323410dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:00.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.120+0000 7f323be7b700 1 --2- 172.21.15.76:0/3717394306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 0x7f323410dfb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f322c00b0a0 tx=0x7f322c00b3b0 comp rx=0 tx=0).stop 2024-03-23T15:42:00.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 -- 172.21.15.76:0/3717394306 shutdown_connections 2024-03-23T15:42:00.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 --2- 172.21.15.76:0/3717394306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 0x7f323410dfb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 --2- 172.21.15.76:0/3717394306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3234101390 0x7f3234101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 --2- 172.21.15.76:0/3717394306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3234101d40 0x7f3234105d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 -- 172.21.15.76:0/3717394306 >> 172.21.15.76:0/3717394306 conn(0x7f3234077150 msgr2=0x7f3234077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 -- 172.21.15.76:0/3717394306 shutdown_connections 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 -- 172.21.15.76:0/3717394306 wait complete. 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 Processor -- start 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 -- start start 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.121+0000 7f323be7b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3234101390 0x7f32341a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323be7b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3234101d40 0x7f32341a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323be7b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 0x7f323419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323be7b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f323419aee0 con 0x7f32341063e0 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323be7b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f323419b050 con 0x7f3234101390 2024-03-23T15:42:00.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323be7b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f323419b210 con 0x7f3234101d40 2024-03-23T15:42:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323a418700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 0x7f323419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323a418700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 0x7f323419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37494/0 (socket says 172.21.15.76:37494) 2024-03-23T15:42:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f323a418700 1 -- 172.21.15.76:0/2767442108 learned_addr learned my addr 172.21.15.76:0/2767442108 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239c17700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3234101390 0x7f32341a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239416700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3234101d40 0x7f32341a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:00.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239c17700 1 -- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3234101d40 msgr2=0x7f32341a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239c17700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3234101d40 0x7f32341a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239c17700 1 -- 172.21.15.76:0/2767442108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 msgr2=0x7f323419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239c17700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32341063e0 0x7f323419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239c17700 1 -- 172.21.15.76:0/2767442108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c009d00 con 0x7f3234101390 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239416700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3234101d40 0x7f32341a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f3239c17700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3234101390 0x7f32341a06b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32300097d0 tx=0x7f3230009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.122+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230010040 con 0x7f3234101390 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.123+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3230013c70 con 0x7f3234101390 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.123+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323000e340 con 0x7f3234101390 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.123+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323419b4f0 con 0x7f3234101390 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.123+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32341a76e0 con 0x7f3234101390 2024-03-23T15:42:00.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.124+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32341027e0 con 0x7f3234101390 2024-03-23T15:42:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.125+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f32300145b0 con 0x7f3234101390 2024-03-23T15:42:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.125+0000 7f322affd700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f32200a2260 0x7f32200a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:00.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f3239416700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f32200a2260 0x7f32200a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f32300cd9e0 con 0x7f3234101390 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f322affd700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] conn(0x7f32200aba40 0x7f32200ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 --> [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32200ae510 con 0x7f32200aba40 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f32300d22a0 con 0x7f3234101390 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f323a418700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] conn(0x7f32200aba40 0x7f32200ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f3239416700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f32200a2260 0x7f32200a4720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3234100b60 tx=0x7f3224009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.126+0000 7f323a418700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] conn(0x7f32200aba40 0x7f32200ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.127+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== osd.5 v2:172.21.15.192:6808/2902288003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32200ae510 con 0x7f32200aba40 2024-03-23T15:42:00.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.144+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 --> [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32341103b0 con 0x7f32200aba40 2024-03-23T15:42:00.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.144+0000 7f322affd700 1 -- 172.21.15.76:0/2767442108 <== osd.5 v2:172.21.15.192:6808/2902288003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32341103b0 con 0x7f32200aba40 2024-03-23T15:42:00.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] conn(0x7f32200aba40 msgr2=0x7f32200ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:00.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] conn(0x7f32200aba40 0x7f32200ade60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f32200a2260 msgr2=0x7f32200a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f32200a2260 0x7f32200a4720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3234100b60 tx=0x7f3224009260 comp rx=0 tx=0).stop 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3234101390 msgr2=0x7f32341a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 --2- 172.21.15.76:0/2767442108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3234101390 0x7f32341a06b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32300097d0 tx=0x7f3230009b90 comp rx=0 tx=0).stop 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323a418700 1 -- 172.21.15.76:0/2767442108 reap_dead start 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 shutdown_connections 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 >> 172.21.15.76:0/2767442108 conn(0x7f3234077150 msgr2=0x7f32340ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 shutdown_connections 2024-03-23T15:42:00.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:00.145+0000 7f323be7b700 1 -- 172.21.15.76:0/2767442108 wait complete. 2024-03-23T15:42:00.746 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:00 smithi076 ceph-mon[32176]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-23T15:42:00.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:00 smithi183 ceph-mon[36296]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-23T15:42:01.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:00 smithi192 ceph-mon[36653]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-23T15:42:02.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 -- 172.21.15.76:0/2556849556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 msgr2=0x7fa3a006a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 --2- 172.21.15.76:0/2556849556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 0x7fa3a006a310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3940099c0 tx=0x7fa394009cd0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 -- 172.21.15.76:0/2556849556 shutdown_connections 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 --2- 172.21.15.76:0/2556849556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3a006a850 0x7fa3a010f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 --2- 172.21.15.76:0/2556849556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a0108250 0x7fa3a0108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 --2- 172.21.15.76:0/2556849556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 0x7fa3a006a310 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 -- 172.21.15.76:0/2556849556 >> 172.21.15.76:0/2556849556 conn(0x7fa3a0078030 msgr2=0x7fa3a0078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 -- 172.21.15.76:0/2556849556 shutdown_connections 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.070+0000 7fa3a81fd700 1 -- 172.21.15.76:0/2556849556 wait complete. 2024-03-23T15:42:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 Processor -- start 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 -- start start 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 0x7fa3a019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3a006a850 0x7fa3a019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a0108250 0x7fa3a0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0198bf0 con 0x7fa3a0069eb0 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0198d30 con 0x7fa3a006a850 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a81fd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0198ef0 con 0x7fa3a0108250 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a5f99700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 0x7fa3a019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a5f99700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 0x7fa3a019e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37514/0 (socket says 172.21.15.76:37514) 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a5f99700 1 -- 172.21.15.76:0/641650272 learned_addr learned my addr 172.21.15.76:0/641650272 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a5798700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3a006a850 0x7fa3a019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a679a700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a0108250 0x7fa3a0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a679a700 1 -- 172.21.15.76:0/641650272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3a006a850 msgr2=0x7fa3a019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a679a700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3a006a850 0x7fa3a019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a679a700 1 -- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 msgr2=0x7fa3a019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a679a700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 0x7fa3a019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a679a700 1 -- 172.21.15.76:0/641650272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3940095f0 con 0x7fa3a0108250 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a5f99700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3a0069eb0 0x7fa3a019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.071+0000 7fa3a5798700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3a006a850 0x7fa3a019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.072+0000 7fa3a679a700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a0108250 0x7fa3a0198590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa39800efc0 tx=0x7fa39800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.072+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398018070 con 0x7fa3a0108250 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.072+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a0199170 con 0x7fa3a0108250 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.072+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa39800f040 con 0x7fa3a0108250 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.072+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800cd10 con 0x7fa3a0108250 2024-03-23T15:42:02.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.072+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01a5470 con 0x7fa3a0108250 2024-03-23T15:42:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.072+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3a0112500 con 0x7fa3a0108250 2024-03-23T15:42:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.074+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fa398004ad0 con 0x7fa3a0108250 2024-03-23T15:42:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.074+0000 7fa39effd700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3880a21e0 0x7fa3880a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.074+0000 7fa3a5f99700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3880a21e0 0x7fa3880a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.075+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa3980ce1f0 con 0x7fa3a0108250 2024-03-23T15:42:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.075+0000 7fa39effd700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] conn(0x7fa3880ab9c0 0x7fa3880adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:02.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.075+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 --> [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3880ae490 con 0x7fa3880ab9c0 2024-03-23T15:42:02.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.075+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa3980ce610 con 0x7fa3a0108250 2024-03-23T15:42:02.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.075+0000 7fa3a5798700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] conn(0x7fa3880ab9c0 0x7fa3880adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:02.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.075+0000 7fa3a5f99700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3880a21e0 0x7fa3880a46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa390011550 tx=0x7fa390009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:02.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.075+0000 7fa3a5798700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] conn(0x7fa3880ab9c0 0x7fa3880adde0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:02.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.076+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== osd.0 v2:172.21.15.76:6802/4179784307 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3880ae490 con 0x7fa3880ab9c0 2024-03-23T15:42:02.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.096+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 --> [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3a0062910 con 0x7fa3880ab9c0 2024-03-23T15:42:02.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.096+0000 7fa39effd700 1 -- 172.21.15.76:0/641650272 <== osd.0 v2:172.21.15.76:6802/4179784307 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3a0062910 con 0x7fa3880ab9c0 2024-03-23T15:42:02.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] conn(0x7fa3880ab9c0 msgr2=0x7fa3880adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:02.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] conn(0x7fa3880ab9c0 0x7fa3880adde0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3880a21e0 msgr2=0x7fa3880a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3880a21e0 0x7fa3880a46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa390011550 tx=0x7fa390009260 comp rx=0 tx=0).stop 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a0108250 msgr2=0x7fa3a0198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 --2- 172.21.15.76:0/641650272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a0108250 0x7fa3a0198590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa39800efc0 tx=0x7fa39800c3a0 comp rx=0 tx=0).stop 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a679a700 1 -- 172.21.15.76:0/641650272 reap_dead start 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 shutdown_connections 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.097+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 >> 172.21.15.76:0/641650272 conn(0x7fa3a0078030 msgr2=0x7fa3a00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.098+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 shutdown_connections 2024-03-23T15:42:02.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:02.098+0000 7fa3a81fd700 1 -- 172.21.15.76:0/641650272 wait complete. 2024-03-23T15:42:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:02 smithi076 ceph-mon[32176]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:42:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:02 smithi183 ceph-mon[36296]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:42:03.020 INFO:teuthology.orchestra.run.smithi076.stdout:163208757301 2024-03-23T15:42:03.020 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-23T15:42:03.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:02 smithi192 ceph-mon[36653]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:42:03.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 -- 172.21.15.76:0/2915977251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 msgr2=0x7fd364101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 --2- 172.21.15.76:0/2915977251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 0x7fd364101730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd354009990 tx=0x7fd354009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 -- 172.21.15.76:0/2915977251 shutdown_connections 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 --2- 172.21.15.76:0/2915977251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd364106410 0x7fd36410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 --2- 172.21.15.76:0/2915977251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 0x7fd364101730 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 --2- 172.21.15.76:0/2915977251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364101d00 0x7fd364105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 -- 172.21.15.76:0/2915977251 >> 172.21.15.76:0/2915977251 conn(0x7fd364078070 msgr2=0x7fd364078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 -- 172.21.15.76:0/2915977251 shutdown_connections 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 -- 172.21.15.76:0/2915977251 wait complete. 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.447+0000 7fd36a333700 1 Processor -- start 2024-03-23T15:42:03.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd36a333700 1 -- start start 2024-03-23T15:42:03.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd36a333700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 0x7fd364198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:03.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd36a333700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364101d00 0x7fd364198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:03.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd36a333700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd364106410 0x7fd36419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:03.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd36a333700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd36419d5f0 con 0x7fd364101350 2024-03-23T15:42:03.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd36a333700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd36419d760 con 0x7fd364101d00 2024-03-23T15:42:03.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd36a333700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd36419d920 con 0x7fd364106410 2024-03-23T15:42:03.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 0x7fd364198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 0x7fd364198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37534/0 (socket says 172.21.15.76:37534) 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 -- 172.21.15.76:0/3745279415 learned_addr learned my addr 172.21.15.76:0/3745279415 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd3688d0700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd364106410 0x7fd36419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd3637fe700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364101d00 0x7fd364198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 -- 172.21.15.76:0/3745279415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd364106410 msgr2=0x7fd36419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd364106410 0x7fd36419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 -- 172.21.15.76:0/3745279415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364101d00 msgr2=0x7fd364198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd364101d00 0x7fd364198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.448+0000 7fd363fff700 1 -- 172.21.15.76:0/3745279415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3540095f0 con 0x7fd364101350 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.449+0000 7fd363fff700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 0x7fd364198710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd354000c00 tx=0x7fd354004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.449+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35401d070 con 0x7fd364101350 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.449+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35400b610 con 0x7fd364101350 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.449+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36419dba0 con 0x7fd364101350 2024-03-23T15:42:03.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.449+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3540173f0 con 0x7fd364101350 2024-03-23T15:42:03.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.449+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3641a5460 con 0x7fd364101350 2024-03-23T15:42:03.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.450+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd36419de70 con 0x7fd364101350 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.451+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fd354022670 con 0x7fd364101350 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.451+0000 7fd3617fa700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd34c0a21e0 0x7fd34c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.451+0000 7fd3637fe700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd34c0a21e0 0x7fd34c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.452+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd3540cf350 con 0x7fd364101350 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.452+0000 7fd3617fa700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] conn(0x7fd34c0ab9c0 0x7fd34c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.452+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 --> [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd34c0ae490 con 0x7fd34c0ab9c0 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.452+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd3540cf7f0 con 0x7fd364101350 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.452+0000 7fd3688d0700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] conn(0x7fd34c0ab9c0 0x7fd34c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.452+0000 7fd3637fe700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd34c0a21e0 0x7fd34c0a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3641079b0 tx=0x7fd358009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:03.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.452+0000 7fd3688d0700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] conn(0x7fd34c0ab9c0 0x7fd34c0adde0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:03.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.453+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== osd.9 v2:172.21.15.76:6826/1143697448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd34c0ae490 con 0x7fd34c0ab9c0 2024-03-23T15:42:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.469+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 --> [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd364111460 con 0x7fd34c0ab9c0 2024-03-23T15:42:03.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.469+0000 7fd3617fa700 1 -- 172.21.15.76:0/3745279415 <== osd.9 v2:172.21.15.76:6826/1143697448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd364111460 con 0x7fd34c0ab9c0 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] conn(0x7fd34c0ab9c0 msgr2=0x7fd34c0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] conn(0x7fd34c0ab9c0 0x7fd34c0adde0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd34c0a21e0 msgr2=0x7fd34c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd34c0a21e0 0x7fd34c0a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3641079b0 tx=0x7fd358009260 comp rx=0 tx=0).stop 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 msgr2=0x7fd364198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 --2- 172.21.15.76:0/3745279415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd364101350 0x7fd364198710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd354000c00 tx=0x7fd354004730 comp rx=0 tx=0).stop 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd3688d0700 1 -- 172.21.15.76:0/3745279415 reap_dead start 2024-03-23T15:42:03.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 shutdown_connections 2024-03-23T15:42:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 >> 172.21.15.76:0/3745279415 conn(0x7fd364078070 msgr2=0x7fd36406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 shutdown_connections 2024-03-23T15:42:03.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:03.470+0000 7fd36a333700 1 -- 172.21.15.76:0/3745279415 wait complete. 2024-03-23T15:42:04.146 INFO:teuthology.orchestra.run.smithi076.stdout:107374182461 2024-03-23T15:42:04.147 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-23T15:42:04.339 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:04.340 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:04.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:04 smithi076 ceph-mon[32176]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:42:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:04 smithi183 ceph-mon[36296]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:42:05.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:04 smithi192 ceph-mon[36653]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:42:05.272 INFO:teuthology.orchestra.run.smithi076.stdout:287762808865 2024-03-23T15:42:05.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-23T15:42:06.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 -- 172.21.15.76:0/422237902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04101d30 msgr2=0x7fcd04105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 --2- 172.21.15.76:0/422237902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04101d30 0x7fcd04105d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fccec009990 tx=0x7fccec009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 -- 172.21.15.76:0/422237902 shutdown_connections 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 --2- 172.21.15.76:0/422237902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd04106460 0x7fcd0410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 --2- 172.21.15.76:0/422237902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd04101380 0x7fcd04101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 --2- 172.21.15.76:0/422237902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04101d30 0x7fcd04105d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 -- 172.21.15.76:0/422237902 >> 172.21.15.76:0/422237902 conn(0x7fcd04078160 msgr2=0x7fcd04078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 -- 172.21.15.76:0/422237902 shutdown_connections 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 -- 172.21.15.76:0/422237902 wait complete. 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 Processor -- start 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.218+0000 7fcd098d9700 1 -- start start 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd098d9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd04101380 0x7fcd0419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd098d9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd04101d30 0x7fcd0419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd098d9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04106460 0x7fcd041989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:06.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd098d9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041991c0 con 0x7fcd04106460 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd098d9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd04199330 con 0x7fcd04101380 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd098d9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd041994f0 con 0x7fcd04101d30 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04106460 0x7fcd041989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04106460 0x7fcd041989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39944/0 (socket says 172.21.15.76:39944) 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 -- 172.21.15.76:0/441604796 learned_addr learned my addr 172.21.15.76:0/441604796 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd02ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd04101380 0x7fcd0419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd027fc700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd04101d30 0x7fcd0419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 -- 172.21.15.76:0/441604796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd04101d30 msgr2=0x7fcd0419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd04101d30 0x7fcd0419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 -- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd04101380 msgr2=0x7fcd0419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd04101380 0x7fcd0419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 -- 172.21.15.76:0/441604796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec0095f0 con 0x7fcd04106460 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd02ffd700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd04101380 0x7fcd0419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:06.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd037fe700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04106460 0x7fcd041989f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e940 tx=0x7fccfc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0097b0 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd041997d0 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc005020 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.219+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00ccf0 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.220+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd041a5470 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.220+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd04111460 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.221+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fccfc005490 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd088d7700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a21e0 0x7fccf00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd02ffd700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a21e0 0x7fccf00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fccfc014070 con 0x7fcd04106460 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd088d7700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] conn(0x7fccf00ab9c0 0x7fccf00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 --> [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccf00ae490 con 0x7fccf00ab9c0 2024-03-23T15:42:06.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fccfc0ce560 con 0x7fcd04106460 2024-03-23T15:42:06.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd02ffd700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a21e0 0x7fccf00a46a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd04078710 tx=0x7fccf400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:06.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.222+0000 7fcd027fc700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] conn(0x7fccf00ab9c0 0x7fccf00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:06.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.223+0000 7fcd027fc700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] conn(0x7fccf00ab9c0 0x7fccf00adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:06.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.224+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== osd.1 v2:172.21.15.183:6800/103132661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fccf00ae490 con 0x7fccf00ab9c0 2024-03-23T15:42:06.243 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:06.243 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:06.244 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:06.248 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:06.249 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:06.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.247+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 --> [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd04102860 con 0x7fccf00ab9c0 2024-03-23T15:42:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.248+0000 7fcd088d7700 1 -- 172.21.15.76:0/441604796 <== osd.1 v2:172.21.15.183:6800/103132661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd04102860 con 0x7fccf00ab9c0 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.249+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] conn(0x7fccf00ab9c0 msgr2=0x7fccf00adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.249+0000 7fcd098d9700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] conn(0x7fccf00ab9c0 0x7fccf00adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.249+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a21e0 msgr2=0x7fccf00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.250+0000 7fcd098d9700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fccf00a21e0 0x7fccf00a46a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd04078710 tx=0x7fccf400b040 comp rx=0 tx=0).stop 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.250+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04106460 msgr2=0x7fcd041989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.250+0000 7fcd098d9700 1 --2- 172.21.15.76:0/441604796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd04106460 0x7fcd041989f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e940 tx=0x7fccfc00ec50 comp rx=0 tx=0).stop 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.250+0000 7fcd037fe700 1 -- 172.21.15.76:0/441604796 reap_dead start 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.250+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 shutdown_connections 2024-03-23T15:42:06.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.250+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 >> 172.21.15.76:0/441604796 conn(0x7fcd04078160 msgr2=0x7fcd04104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.251+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 shutdown_connections 2024-03-23T15:42:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:06.251+0000 7fcd098d9700 1 -- 172.21.15.76:0/441604796 wait complete. 2024-03-23T15:42:06.706 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:06 smithi076 ceph-mon[32176]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:06.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:06 smithi183 ceph-mon[36296]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:07.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:06 smithi192 ceph-mon[36653]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:08.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:08 smithi183 ceph-mon[36296]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:09.031 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:08 smithi076 ceph-mon[32176]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:09.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:08 smithi192 ceph-mon[36653]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:10 smithi183 ceph-mon[36296]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:42:11.053 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:10 smithi076 ceph-mon[32176]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:42:11.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:10 smithi192 ceph-mon[36653]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:42:12.454 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:12.488 INFO:teuthology.orchestra.run.smithi076.stdout:34359738445 2024-03-23T15:42:12.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-23T15:42:12.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:12 smithi076 ceph-mon[32176]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:42:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:12 smithi183 ceph-mon[36296]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:42:13.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:12 smithi192 ceph-mon[36653]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:42:14.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:14 smithi183 ceph-mon[36296]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:14.923 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:14 smithi076 ceph-mon[32176]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:15.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:14 smithi192 ceph-mon[36653]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/677749001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 msgr2=0x7fb7b410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/677749001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b410df80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00b0a0 tx=0x7fb7ac00b3b0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/677749001 shutdown_connections 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/677749001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b410df80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/677749001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7b4107140 0x7fb7b4107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/677749001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7b4101100 0x7fb7b4101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/677749001 >> 172.21.15.76:0/677749001 conn(0x7fb7b4078030 msgr2=0x7fb7b4078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/677749001 shutdown_connections 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/677749001 wait complete. 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 Processor -- start 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.242+0000 7fb7bb8d5700 1 -- start start 2024-03-23T15:42:15.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7b4101100 0x7fb7b4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b4199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7b4107140 0x7fb7b419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41a4a60 con 0x7fb7b4101aa0 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41a4bd0 con 0x7fb7b4101100 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41a4d90 con 0x7fb7b4107140 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b4199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b9671700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7b4101100 0x7fb7b4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b4199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58066/0 (socket says 172.21.15.76:58066) 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 -- 172.21.15.76:0/1625121691 learned_addr learned my addr 172.21.15.76:0/1625121691 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:15.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b9e72700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7b4107140 0x7fb7b419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 -- 172.21.15.76:0/1625121691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7b4107140 msgr2=0x7fb7b419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7b4107140 0x7fb7b419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 -- 172.21.15.76:0/1625121691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7b4101100 msgr2=0x7fb7b4199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7b4101100 0x7fb7b4199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 -- 172.21.15.76:0/1625121691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac009d00 con 0x7fb7b4101aa0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b9671700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7b4101100 0x7fb7b4199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7b8e70700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b4199cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d950 tx=0x7fb7a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7aa7fc700 1 -- 172.21.15.76:0/1625121691 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400f890 con 0x7fb7b4101aa0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7aa7fc700 1 -- 172.21.15.76:0/1625121691 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4010460 con 0x7fb7b4101aa0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b41a5070 con 0x7fb7b4101aa0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7aa7fc700 1 -- 172.21.15.76:0/1625121691 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400e610 con 0x7fb7b4101aa0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.243+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b41a5570 con 0x7fb7b4101aa0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.244+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b4109660 con 0x7fb7b4101aa0 2024-03-23T15:42:15.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.245+0000 7fb7aa7fc700 1 -- 172.21.15.76:0/1625121691 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fb7a4010960 con 0x7fb7b4101aa0 2024-03-23T15:42:15.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.246+0000 7fb7aa7fc700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb7a00a21e0 0x7fb7a00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.246+0000 7fb7b9671700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb7a00a21e0 0x7fb7a00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.246+0000 7fb7aa7fc700 1 -- 172.21.15.76:0/1625121691 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb7a40cda20 con 0x7fb7b4101aa0 2024-03-23T15:42:15.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.246+0000 7fb7b9671700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb7a00a21e0 0x7fb7a00a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009fd0 tx=0x7fb7b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:15.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.251+0000 7fb7aa7fc700 1 -- 172.21.15.76:0/1625121691 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb7a4096a40 con 0x7fb7b4101aa0 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 -- 172.21.15.76:0/3710158467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 msgr2=0x7f47101028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 --2- 172.21.15.76:0/3710158467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 0x7f47101028c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47040099c0 tx=0x7f4704009cd0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 -- 172.21.15.76:0/3710158467 shutdown_connections 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 --2- 172.21.15.76:0/3710158467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710107590 0x7f471010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 --2- 172.21.15.76:0/3710158467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 0x7f47101028c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 --2- 172.21.15.76:0/3710158467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710102e90 0x7f4710106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 -- 172.21.15.76:0/3710158467 >> 172.21.15.76:0/3710158467 conn(0x7f47100792d0 msgr2=0x7f47100796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 -- 172.21.15.76:0/3710158467 shutdown_connections 2024-03-23T15:42:15.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.433+0000 7f4716f3f700 1 -- 172.21.15.76:0/3710158467 wait complete. 2024-03-23T15:42:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.434+0000 7f4716f3f700 1 Processor -- start 2024-03-23T15:42:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.434+0000 7f4716f3f700 1 -- start start 2024-03-23T15:42:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.434+0000 7f4716f3f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 0x7f47101986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.434+0000 7f4716f3f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710102e90 0x7f4710198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.434+0000 7f4716f3f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710107590 0x7f471019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.434+0000 7f4716f3f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f471019d580 con 0x7f4710107590 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.434+0000 7f4716f3f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f471019d6f0 con 0x7f47101024e0 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f4716f3f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f471019d8b0 con 0x7f4710102e90 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710107590 0x7f471019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710107590 0x7f471019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58078/0 (socket says 172.21.15.76:58078) 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 -- 172.21.15.76:0/587747048 learned_addr learned my addr 172.21.15.76:0/587747048 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f470ffff700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710102e90 0x7f4710198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f4714cdb700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 0x7f47101986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 -- 172.21.15.76:0/587747048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710102e90 msgr2=0x7f4710198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710102e90 0x7f4710198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 -- 172.21.15.76:0/587747048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 msgr2=0x7f47101986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 0x7f47101986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 -- 172.21.15.76:0/587747048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4708009c40 con 0x7f4710107590 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f47154dc700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710107590 0x7f471019cf70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f470800c370 tx=0x7f470800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f470dffb700 1 -- 172.21.15.76:0/587747048 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708017070 con 0x7f4710107590 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47040095f0 con 0x7f4710107590 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f470dffb700 1 -- 172.21.15.76:0/587747048 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f470800f040 con 0x7f4710107590 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.435+0000 7f470dffb700 1 -- 172.21.15.76:0/587747048 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470800e050 con 0x7f4710107590 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.436+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47101a5220 con 0x7f4710107590 2024-03-23T15:42:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.436+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471010a7d0 con 0x7f4710107590 2024-03-23T15:42:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.438+0000 7f470dffb700 1 -- 172.21.15.76:0/587747048 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f470800cb60 con 0x7f4710107590 2024-03-23T15:42:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.439+0000 7f470dffb700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47000a21e0 0x7f47000a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.439+0000 7f4714cdb700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47000a21e0 0x7f47000a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.439+0000 7f470dffb700 1 -- 172.21.15.76:0/587747048 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f47080cdbd0 con 0x7f4710107590 2024-03-23T15:42:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.439+0000 7f4714cdb700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47000a21e0 0x7f47000a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4704005850 tx=0x7f4704005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:15.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.442+0000 7f470dffb700 1 -- 172.21.15.76:0/587747048 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4708096bf0 con 0x7f4710107590 2024-03-23T15:42:15.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.453+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb7b406a470 con 0x7fb7b4101aa0 2024-03-23T15:42:15.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.454+0000 7fb7aa7fc700 1 -- 172.21.15.76:0/1625121691 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb7a409a100 con 0x7fb7b4101aa0 2024-03-23T15:42:15.457 INFO:teuthology.orchestra.run.smithi076.stdout:343597383704 2024-03-23T15:42:15.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb7a00a21e0 msgr2=0x7fb7a00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb7a00a21e0 0x7fb7a00a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009fd0 tx=0x7fb7b0009260 comp rx=0 tx=0).stop 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 msgr2=0x7fb7b4199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b4199cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d950 tx=0x7fb7a400dc60 comp rx=0 tx=0).stop 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 shutdown_connections 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb7a00a21e0 0x7fb7a00a46a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7b4107140 0x7fb7b419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7b4101100 0x7fb7b4199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 --2- 172.21.15.76:0/1625121691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b4101aa0 0x7fb7b4199cb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 >> 172.21.15.76:0/1625121691 conn(0x7fb7b4078030 msgr2=0x7fb7b40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 shutdown_connections 2024-03-23T15:42:15.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.456+0000 7fb7bb8d5700 1 -- 172.21.15.76:0/1625121691 wait complete. 2024-03-23T15:42:15.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.659+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4710199a00 con 0x7f4710107590 2024-03-23T15:42:15.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.659+0000 7f470dffb700 1 -- 172.21.15.76:0/587747048 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f470809a2b0 con 0x7f4710107590 2024-03-23T15:42:15.662 INFO:teuthology.orchestra.run.smithi076.stdout:197568495667 2024-03-23T15:42:15.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47000a21e0 msgr2=0x7f47000a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47000a21e0 0x7f47000a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4704005850 tx=0x7f4704005fb0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710107590 msgr2=0x7f471019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:15.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710107590 0x7f471019cf70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f470800c370 tx=0x7f470800c680 comp rx=0 tx=0).stop 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 shutdown_connections 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47000a21e0 0x7f47000a46a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4710107590 0x7f471019cf70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47101024e0 0x7f47101986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 --2- 172.21.15.76:0/587747048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4710102e90 0x7f4710198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 >> 172.21.15.76:0/587747048 conn(0x7f47100792d0 msgr2=0x7f47101002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 shutdown_connections 2024-03-23T15:42:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:15.661+0000 7f4716f3f700 1 -- 172.21.15.76:0/587747048 wait complete. 2024-03-23T15:42:15.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-03-23T15:42:15.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:15 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1625121691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T15:42:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-03-23T15:42:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-23T15:42:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-23T15:42:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-23T15:42:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-23T15:42:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-23T15:42:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-03-23T15:42:15.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-23T15:42:15.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-03-23T15:42:15.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-03-23T15:42:15.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:15 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1625121691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T15:42:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:55834574922 2024-03-23T15:42:15.890 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-23T15:42:16.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-03-23T15:42:16.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:15 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1625121691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-23T15:42:16.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 -- 172.21.15.76:0/3191852460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101d40 msgr2=0x7f5040105d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:16.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 --2- 172.21.15.76:0/3191852460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101d40 0x7f5040105d00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5034009990 tx=0x7f5034009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 -- 172.21.15.76:0/3191852460 shutdown_connections 2024-03-23T15:42:16.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 --2- 172.21.15.76:0/3191852460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50401063e0 0x7f504010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 --2- 172.21.15.76:0/3191852460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5040101390 0x7f5040101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 --2- 172.21.15.76:0/3191852460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101d40 0x7f5040105d00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 -- 172.21.15.76:0/3191852460 >> 172.21.15.76:0/3191852460 conn(0x7f5040077150 msgr2=0x7f5040077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 -- 172.21.15.76:0/3191852460 shutdown_connections 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 -- 172.21.15.76:0/3191852460 wait complete. 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.522+0000 7f5045408700 1 Processor -- start 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f5045408700 1 -- start start 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f5045408700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101390 0x7f5040102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f5045408700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5040101d40 0x7f50401032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f5045408700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50401063e0 0x7f5040103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f5045408700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f50401a6b10 con 0x7f5040101390 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f5045408700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50401a6c80 con 0x7f50401063e0 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f5045408700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f50401a6e40 con 0x7f5040101d40 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101390 0x7f5040102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:16.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101390 0x7f5040102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58100/0 (socket says 172.21.15.76:58100) 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 -- 172.21.15.76:0/269833173 learned_addr learned my addr 172.21.15.76:0/269833173 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503e7fc700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5040101d40 0x7f50401032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503f7fe700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50401063e0 0x7f5040103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 -- 172.21.15.76:0/269833173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5040101d40 msgr2=0x7f50401032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5040101d40 0x7f50401032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 -- 172.21.15.76:0/269833173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50401063e0 msgr2=0x7f5040103820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50401063e0 0x7f5040103820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.523+0000 7f503effd700 1 -- 172.21.15.76:0/269833173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50340095f0 con 0x7f5040101390 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.524+0000 7f503effd700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101390 0x7f5040102da0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f503000d980 tx=0x7f503000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.524+0000 7f5027fff700 1 -- 172.21.15.76:0/269833173 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030004220 con 0x7f5040101390 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.524+0000 7f5027fff700 1 -- 172.21.15.76:0/269833173 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5030007640 con 0x7f5040101390 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.524+0000 7f5027fff700 1 -- 172.21.15.76:0/269833173 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503000f690 con 0x7f5040101390 2024-03-23T15:42:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.524+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50401a7120 con 0x7f5040101390 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.524+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401a75f0 con 0x7f5040101390 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.525+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504010a710 con 0x7f5040101390 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.526+0000 7f5027fff700 1 -- 172.21.15.76:0/269833173 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f5030004410 con 0x7f5040101390 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.526+0000 7f5027fff700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50280a21e0 0x7f50280a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.527+0000 7f503e7fc700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50280a21e0 0x7f50280a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.527+0000 7f5027fff700 1 -- 172.21.15.76:0/269833173 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f50300cdf00 con 0x7f5040101390 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.527+0000 7f503e7fc700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50280a21e0 0x7f50280a46a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5034006010 tx=0x7f5034005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:16.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.535+0000 7f5027fff700 1 -- 172.21.15.76:0/269833173 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f5030096d50 con 0x7f5040101390 2024-03-23T15:42:16.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.747+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5040068ec0 con 0x7f5040101390 2024-03-23T15:42:16.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.748+0000 7f5027fff700 1 -- 172.21.15.76:0/269833173 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5030016090 con 0x7f5040101390 2024-03-23T15:42:16.751 INFO:teuthology.orchestra.run.smithi076.stdout:163208757304 2024-03-23T15:42:16.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50280a21e0 msgr2=0x7f50280a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:16.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50280a21e0 0x7f50280a46a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5034006010 tx=0x7f5034005960 comp rx=0 tx=0).stop 2024-03-23T15:42:16.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101390 msgr2=0x7f5040102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:16.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101390 0x7f5040102da0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f503000d980 tx=0x7f503000dc90 comp rx=0 tx=0).stop 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 shutdown_connections 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f50280a21e0 0x7f50280a46a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50401063e0 0x7f5040103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5040101390 0x7f5040102da0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 --2- 172.21.15.76:0/269833173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5040101d40 0x7f50401032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 >> 172.21.15.76:0/269833173 conn(0x7f5040077150 msgr2=0x7f504006aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 shutdown_connections 2024-03-23T15:42:16.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:16.750+0000 7f5045408700 1 -- 172.21.15.76:0/269833173 wait complete. 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-23T15:42:16.872 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-03-23T15:42:16.873 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-23T15:42:16.873 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-03-23T15:42:16.873 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-03-23T15:42:16.873 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T15:42:16.873 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/587747048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T15:42:16.873 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:16 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:42:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-03-23T15:42:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-23T15:42:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-23T15:42:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-23T15:42:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-23T15:42:16.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-23T15:42:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-03-23T15:42:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-23T15:42:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-03-23T15:42:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-03-23T15:42:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T15:42:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/587747048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T15:42:16.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:16 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:42:17.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:17.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-03-23T15:42:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-23T15:42:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-23T15:42:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-03-23T15:42:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-23T15:42:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-23T15:42:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-03-23T15:42:17.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-23T15:42:17.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-03-23T15:42:17.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-03-23T15:42:17.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: osdmap e86: 12 total, 12 up, 12 in 2024-03-23T15:42:17.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/587747048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-23T15:42:17.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:16 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:42:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:17 smithi183 ceph-mon[36296]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T15:42:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:17 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/269833173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T15:42:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:17 smithi183 ceph-mon[36296]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:17.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:17 smithi076 ceph-mon[32176]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T15:42:17.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:17 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/269833173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T15:42:17.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:17 smithi076 ceph-mon[32176]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:18.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:17 smithi192 ceph-mon[36653]: osdmap e87: 12 total, 12 up, 12 in 2024-03-23T15:42:18.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:17 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/269833173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T15:42:18.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:17 smithi192 ceph-mon[36653]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 -- 172.21.15.76:0/3839240678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76981013e0 msgr2=0x7f7698101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:18.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 --2- 172.21.15.76:0/3839240678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76981013e0 0x7f7698101860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f768c0099c0 tx=0x7f768c009cd0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 -- 172.21.15.76:0/3839240678 shutdown_connections 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 --2- 172.21.15.76:0/3839240678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7698101da0 0x7f769810dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 --2- 172.21.15.76:0/3839240678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7698107230 0x7f7698107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 --2- 172.21.15.76:0/3839240678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76981013e0 0x7f7698101860 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 -- 172.21.15.76:0/3839240678 >> 172.21.15.76:0/3839240678 conn(0x7f7698077fc0 msgr2=0x7f76980783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 -- 172.21.15.76:0/3839240678 shutdown_connections 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.668+0000 7f769dbab700 1 -- 172.21.15.76:0/3839240678 wait complete. 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.669+0000 7f769dbab700 1 Processor -- start 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.669+0000 7f769dbab700 1 -- start start 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76981013e0 0x7f7698199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7698101da0 0x7f7698199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:18.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7698107230 0x7f76981a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f769819a2b0 con 0x7f7698101da0 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76981a4ae0 con 0x7f7698107230 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76981a4c20 con 0x7f76981013e0 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7698101da0 0x7f7698199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7698101da0 0x7f7698199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58134/0 (socket says 172.21.15.76:58134) 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 -- 172.21.15.76:0/2566738096 learned_addr learned my addr 172.21.15.76:0/2566738096 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f76977fe700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76981013e0 0x7f7698199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7697fff700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7698107230 0x7f76981a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 -- 172.21.15.76:0/2566738096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76981013e0 msgr2=0x7f7698199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76981013e0 0x7f7698199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 -- 172.21.15.76:0/2566738096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7698107230 msgr2=0x7f76981a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7698107230 0x7f76981a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 -- 172.21.15.76:0/2566738096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c0095f0 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7696ffd700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7698101da0 0x7f7698199a60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f768c005910 tx=0x7f768c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7694ff9700 1 -- 172.21.15.76:0/2566738096 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c01d070 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7694ff9700 1 -- 172.21.15.76:0/2566738096 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768c003680 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76981a4e20 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f7694ff9700 1 -- 172.21.15.76:0/2566738096 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c022c90 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.670+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76981a51d0 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.671+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76981096a0 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.672+0000 7f7694ff9700 1 -- 172.21.15.76:0/2566738096 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f768c003a70 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.673+0000 7f7694ff9700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76800a21e0 0x7f76800a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.673+0000 7f76977fe700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76800a21e0 0x7f76800a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.673+0000 7f7694ff9700 1 -- 172.21.15.76:0/2566738096 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f768c0cf7c0 con 0x7f7698101da0 2024-03-23T15:42:18.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.673+0000 7f76977fe700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76800a21e0 0x7f76800a46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7688007b60 tx=0x7f76880073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:18.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.677+0000 7f7694ff9700 1 -- 172.21.15.76:0/2566738096 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f768c0986a0 con 0x7f7698101da0 2024-03-23T15:42:18.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.880+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7698111180 con 0x7f7698101da0 2024-03-23T15:42:18.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.880+0000 7f7694ff9700 1 -- 172.21.15.76:0/2566738096 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f768c09bd60 con 0x7f7698101da0 2024-03-23T15:42:18.883 INFO:teuthology.orchestra.run.smithi076.stdout:253403070505 2024-03-23T15:42:18.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76800a21e0 msgr2=0x7f76800a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:18.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76800a21e0 0x7f76800a46a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7688007b60 tx=0x7f76880073d0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7698101da0 msgr2=0x7f7698199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7698101da0 0x7f7698199a60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f768c005910 tx=0x7f768c004730 comp rx=0 tx=0).stop 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 shutdown_connections 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76800a21e0 0x7f76800a46a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7698107230 0x7f76981a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76981013e0 0x7f7698199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 --2- 172.21.15.76:0/2566738096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7698101da0 0x7f7698199a60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 >> 172.21.15.76:0/2566738096 conn(0x7f7698077fc0 msgr2=0x7f7698100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 shutdown_connections 2024-03-23T15:42:18.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.882+0000 7f769dbab700 1 -- 172.21.15.76:0/2566738096 wait complete. 2024-03-23T15:42:19.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757301 got 163208757304 for osd.5 2024-03-23T15:42:19.317 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:19.523 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.523 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.523 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.523 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:19 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.523 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1618010988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T15:42:19.523 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2566738096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T15:42:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:19 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:19 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:19 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:19 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1618010988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T15:42:19.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2566738096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T15:42:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:19 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:19 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:19 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:19 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1618010988' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T15:42:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2566738096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-23T15:42:20.392 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:20 smithi076 ceph-mon[32176]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:20.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:20 smithi192 ceph-mon[36653]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:20.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:20 smithi183 ceph-mon[36296]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:20.778 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495661 got 197568495667 for osd.6 2024-03-23T15:42:20.778 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:21.312 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:21 smithi076 ceph-mon[32176]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 -- 172.21.15.76:0/455722472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 msgr2=0x7f9b24106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 --2- 172.21.15.76:0/455722472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b24106e50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009990 tx=0x7f9b0c009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 -- 172.21.15.76:0/455722472 shutdown_connections 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 --2- 172.21.15.76:0/455722472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b24107580 0x7f9b2410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 --2- 172.21.15.76:0/455722472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b2406abc0 0x7f9b2406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 --2- 172.21.15.76:0/455722472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b24106e50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 -- 172.21.15.76:0/455722472 >> 172.21.15.76:0/455722472 conn(0x7f9b24077110 msgr2=0x7f9b24077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 -- 172.21.15.76:0/455722472 shutdown_connections 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.088+0000 7f9b2881e700 1 -- 172.21.15.76:0/455722472 wait complete. 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 Processor -- start 2024-03-23T15:42:21.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 -- start start 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b2406abc0 0x7f9b2419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b2419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b24107580 0x7f9b241985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24198c40 con 0x7f9b2406b570 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24198d80 con 0x7f9b2406abc0 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2881e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24198f40 con 0x7f9b24107580 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b2419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b2419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58116/0 (socket says 172.21.15.76:58116) 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 -- 172.21.15.76:0/1618010988 learned_addr learned my addr 172.21.15.76:0/1618010988 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b22d9d700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b24107580 0x7f9b241985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b2259c700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b2406abc0 0x7f9b2419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 -- 172.21.15.76:0/1618010988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b24107580 msgr2=0x7f9b241985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:21.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b24107580 0x7f9b241985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 -- 172.21.15.76:0/1618010988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b2406abc0 msgr2=0x7f9b2419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b2406abc0 0x7f9b2419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.089+0000 7f9b19d9b700 1 -- 172.21.15.76:0/1618010988 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c0095f0 con 0x7f9b2406b570 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.090+0000 7f9b19d9b700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b2419e960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c006010 tx=0x7f9b0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.090+0000 7f9b1bfff700 1 -- 172.21.15.76:0/1618010988 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c01d070 con 0x7f9b2406b570 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.090+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241991c0 con 0x7f9b2406b570 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stdout:141733920829 2024-03-23T15:42:21.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.090+0000 7f9b1bfff700 1 -- 172.21.15.76:0/1618010988 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0c00b610 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.090+0000 7f9b1bfff700 1 -- 172.21.15.76:0/1618010988 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0173f0 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.090+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241a54c0 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.091+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2410a730 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.092+0000 7f9b1bfff700 1 -- 172.21.15.76:0/1618010988 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f9b0c022590 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.092+0000 7f9b1bfff700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9b100a21e0 0x7f9b100a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.092+0000 7f9b2259c700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9b100a21e0 0x7f9b100a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.092+0000 7f9b1bfff700 1 -- 172.21.15.76:0/1618010988 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9b0c0cf2c0 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.093+0000 7f9b2259c700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9b100a21e0 0x7f9b100a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009a10 tx=0x7f9b14009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.097+0000 7f9b1bfff700 1 -- 172.21.15.76:0/1618010988 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9b0c0981a0 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.314+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9b24068ec0 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.314+0000 7f9b1bfff700 1 -- 172.21.15.76:0/1618010988 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9b0c09b860 con 0x7f9b2406b570 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9b100a21e0 msgr2=0x7f9b100a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:21.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9b100a21e0 0x7f9b100a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009a10 tx=0x7f9b14009190 comp rx=0 tx=0).stop 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 msgr2=0x7f9b2419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b2419e960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c006010 tx=0x7f9b0c004730 comp rx=0 tx=0).stop 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 shutdown_connections 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9b100a21e0 0x7f9b100a46a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b24107580 0x7f9b241985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b2406abc0 0x7f9b2419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 --2- 172.21.15.76:0/1618010988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b2406b570 0x7f9b2419e960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 >> 172.21.15.76:0/1618010988 conn(0x7f9b24077110 msgr2=0x7f9b240ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 shutdown_connections 2024-03-23T15:42:21.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:18.316+0000 7f9b2881e700 1 -- 172.21.15.76:0/1618010988 wait complete. 2024-03-23T15:42:21.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:21 smithi192 ceph-mon[36653]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-23T15:42:21.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:21 smithi183 ceph-mon[36296]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-23T15:42:21.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920829 for osd.4 2024-03-23T15:42:21.937 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:22.463 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:22 smithi076 ceph-mon[32176]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:22.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:22 smithi192 ceph-mon[36653]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:22.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:22 smithi183 ceph-mon[36296]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:23.118 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383699 got 343597383704 for osd.11 2024-03-23T15:42:23.119 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:23.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 -- 172.21.15.76:0/4069598763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8100d90 msgr2=0x7f42d8101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:23.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 --2- 172.21.15.76:0/4069598763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8100d90 0x7f42d8101170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009990 tx=0x7f42c8009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 -- 172.21.15.76:0/4069598763 shutdown_connections 2024-03-23T15:42:23.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 --2- 172.21.15.76:0/4069598763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d81075b0 0x7f42d810f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 --2- 172.21.15.76:0/4069598763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8100d90 0x7f42d8101170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 --2- 172.21.15.76:0/4069598763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8101740 0x7f42d8106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 -- 172.21.15.76:0/4069598763 >> 172.21.15.76:0/4069598763 conn(0x7f42d8076430 msgr2=0x7f42d8076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 -- 172.21.15.76:0/4069598763 shutdown_connections 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.299+0000 7f42e01d6700 1 -- 172.21.15.76:0/4069598763 wait complete. 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 Processor -- start 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 -- start start 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100d90 0x7f42d8101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8101740 0x7f42d8102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d81075b0 0x7f42d8102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42d81a6b40 con 0x7f42d8101740 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42d81a6cb0 con 0x7f42d81075b0 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42e01d6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f42d81a6e70 con 0x7f42d8100d90 2024-03-23T15:42:23.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8101740 0x7f42d8102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8101740 0x7f42d8102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58166/0 (socket says 172.21.15.76:58166) 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 -- 172.21.15.76:0/3176239074 learned_addr learned my addr 172.21.15.76:0/3176239074 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42ddf72700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100d90 0x7f42d8101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42de773700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d81075b0 0x7f42d8102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 -- 172.21.15.76:0/3176239074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100d90 msgr2=0x7f42d8101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100d90 0x7f42d8101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 -- 172.21.15.76:0/3176239074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d81075b0 msgr2=0x7f42d8102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d81075b0 0x7f42d8102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.300+0000 7f42dd771700 1 -- 172.21.15.76:0/3176239074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c80095f0 con 0x7f42d8101740 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.301+0000 7f42dd771700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8101740 0x7f42d8102250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d950 tx=0x7f42d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.301+0000 7f42ceffd700 1 -- 172.21.15.76:0/3176239074 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400f890 con 0x7f42d8101740 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.301+0000 7f42ceffd700 1 -- 172.21.15.76:0/3176239074 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4010460 con 0x7f42d8101740 2024-03-23T15:42:23.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.301+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d81a7150 con 0x7f42d8101740 2024-03-23T15:42:23.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.301+0000 7f42ceffd700 1 -- 172.21.15.76:0/3176239074 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400e610 con 0x7f42d8101740 2024-03-23T15:42:23.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.301+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d81a7650 con 0x7f42d8101740 2024-03-23T15:42:23.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.302+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d810a810 con 0x7f42d8101740 2024-03-23T15:42:23.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.303+0000 7f42ceffd700 1 -- 172.21.15.76:0/3176239074 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f42d40105d0 con 0x7f42d8101740 2024-03-23T15:42:23.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.303+0000 7f42ceffd700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f42c40a21e0 0x7f42c40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:23.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.304+0000 7f42ddf72700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f42c40a21e0 0x7f42c40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:23.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.304+0000 7f42ceffd700 1 -- 172.21.15.76:0/3176239074 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f42d40cdc80 con 0x7f42d8101740 2024-03-23T15:42:23.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.304+0000 7f42ddf72700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f42c40a21e0 0x7f42c40a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42c80098e0 tx=0x7f42c8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:23.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.308+0000 7f42ceffd700 1 -- 172.21.15.76:0/3176239074 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f42d409a600 con 0x7f42d8101740 2024-03-23T15:42:23.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.511+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f42d81122b0 con 0x7f42d8101740 2024-03-23T15:42:23.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.511+0000 7f42ceffd700 1 -- 172.21.15.76:0/3176239074 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42d409a190 con 0x7f42d8101740 2024-03-23T15:42:23.514 INFO:teuthology.orchestra.run.smithi076.stdout:77309411400 2024-03-23T15:42:23.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f42c40a21e0 msgr2=0x7f42c40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:23.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f42c40a21e0 0x7f42c40a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42c80098e0 tx=0x7f42c8005960 comp rx=0 tx=0).stop 2024-03-23T15:42:23.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8101740 msgr2=0x7f42d8102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:23.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8101740 0x7f42d8102250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d950 tx=0x7f42d400dc60 comp rx=0 tx=0).stop 2024-03-23T15:42:23.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 shutdown_connections 2024-03-23T15:42:23.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f42c40a21e0 0x7f42c40a46a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42d81075b0 0x7f42d8102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100d90 0x7f42d8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 --2- 172.21.15.76:0/3176239074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42d8101740 0x7f42d8102250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:23.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 >> 172.21.15.76:0/3176239074 conn(0x7f42d8076430 msgr2=0x7f42d80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:23.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 shutdown_connections 2024-03-23T15:42:23.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:23.513+0000 7f42e01d6700 1 -- 172.21.15.76:0/3176239074 wait complete. 2024-03-23T15:42:23.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:23 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3176239074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T15:42:23.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:23 smithi192 ceph-mon[36653]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-23T15:42:23.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:23 smithi192 ceph-mon[36653]: Cluster is now healthy 2024-03-23T15:42:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:23 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3176239074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T15:42:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:23 smithi076 ceph-mon[32176]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-23T15:42:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:23 smithi076 ceph-mon[32176]: Cluster is now healthy 2024-03-23T15:42:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:23 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3176239074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T15:42:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:23 smithi183 ceph-mon[36296]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-23T15:42:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:23 smithi183 ceph-mon[36296]: Cluster is now healthy 2024-03-23T15:42:24.240 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411400 for osd.2 2024-03-23T15:42:24.240 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 -- 172.21.15.76:0/4196952943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301007f0 msgr2=0x7f6430100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 --2- 172.21.15.76:0/4196952943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301007f0 0x7f6430100c70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6424009990 tx=0x7f6424009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 -- 172.21.15.76:0/4196952943 shutdown_connections 2024-03-23T15:42:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 --2- 172.21.15.76:0/4196952943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64301011b0 0x7f643010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 --2- 172.21.15.76:0/4196952943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6430107270 0x7f6430107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 --2- 172.21.15.76:0/4196952943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301007f0 0x7f6430100c70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 -- 172.21.15.76:0/4196952943 >> 172.21.15.76:0/4196952943 conn(0x7f6430077700 msgr2=0x7f64300feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 -- 172.21.15.76:0/4196952943 shutdown_connections 2024-03-23T15:42:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.432+0000 7f643638e700 1 -- 172.21.15.76:0/4196952943 wait complete. 2024-03-23T15:42:24.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 Processor -- start 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 -- start start 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64301007f0 0x7f64301a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301011b0 0x7f64301a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6430107270 0x7f643019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f643019af50 con 0x7f64301011b0 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f643019b090 con 0x7f64301007f0 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f643638e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f643019b250 con 0x7f6430107270 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f642f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301011b0 0x7f64301a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f642f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301011b0 0x7f64301a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58190/0 (socket says 172.21.15.76:58190) 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f642f7fe700 1 -- 172.21.15.76:0/856476802 learned_addr learned my addr 172.21.15.76:0/856476802 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.433+0000 7f642ffff700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64301007f0 0x7f64301a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f643492b700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6430107270 0x7f643019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:24.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642f7fe700 1 -- 172.21.15.76:0/856476802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6430107270 msgr2=0x7f643019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642f7fe700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6430107270 0x7f643019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642f7fe700 1 -- 172.21.15.76:0/856476802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64301007f0 msgr2=0x7f64301a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642f7fe700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64301007f0 0x7f64301a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642f7fe700 1 -- 172.21.15.76:0/856476802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64240095f0 con 0x7f64301011b0 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642f7fe700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301011b0 0x7f64301a0c70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6424000c00 tx=0x7f6424004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642d7fa700 1 -- 172.21.15.76:0/856476802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642401d070 con 0x7f64301011b0 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642d7fa700 1 -- 172.21.15.76:0/856476802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f642400b610 con 0x7f64301011b0 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643019b4d0 con 0x7f64301011b0 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f642d7fa700 1 -- 172.21.15.76:0/856476802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240173f0 con 0x7f64301011b0 2024-03-23T15:42:24.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.434+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64301a77d0 con 0x7f64301011b0 2024-03-23T15:42:24.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.435+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6430103b10 con 0x7f64301011b0 2024-03-23T15:42:24.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.436+0000 7f642d7fa700 1 -- 172.21.15.76:0/856476802 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f6424022670 con 0x7f64301011b0 2024-03-23T15:42:24.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.437+0000 7f642d7fa700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f64180a2260 0x7f64180a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:24.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.437+0000 7f642ffff700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f64180a2260 0x7f64180a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:24.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.437+0000 7f642d7fa700 1 -- 172.21.15.76:0/856476802 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f64240cef60 con 0x7f64301011b0 2024-03-23T15:42:24.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.437+0000 7f642ffff700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f64180a2260 0x7f64180a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f643019ba30 tx=0x7f6420009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:24.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.442+0000 7f642d7fa700 1 -- 172.21.15.76:0/856476802 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6424097e40 con 0x7f64301011b0 2024-03-23T15:42:24.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.653+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6430068ec0 con 0x7f64301011b0 2024-03-23T15:42:24.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.654+0000 7f642d7fa700 1 -- 172.21.15.76:0/856476802 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f642409b500 con 0x7f64301011b0 2024-03-23T15:42:24.657 INFO:teuthology.orchestra.run.smithi076.stdout:107374182466 2024-03-23T15:42:24.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f64180a2260 msgr2=0x7f64180a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:24.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f64180a2260 0x7f64180a4720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f643019ba30 tx=0x7f6420009260 comp rx=0 tx=0).stop 2024-03-23T15:42:24.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301011b0 msgr2=0x7f64301a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:24.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301011b0 0x7f64301a0c70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6424000c00 tx=0x7f6424004730 comp rx=0 tx=0).stop 2024-03-23T15:42:24.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 shutdown_connections 2024-03-23T15:42:24.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f64180a2260 0x7f64180a4720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6430107270 0x7f643019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64301007f0 0x7f64301a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 --2- 172.21.15.76:0/856476802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64301011b0 0x7f64301a0c70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:24.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 >> 172.21.15.76:0/856476802 conn(0x7f6430077700 msgr2=0x7f6430105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:24.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 shutdown_connections 2024-03-23T15:42:24.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:24.656+0000 7f643638e700 1 -- 172.21.15.76:0/856476802 wait complete. 2024-03-23T15:42:24.817 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:24 smithi076 ceph-mon[32176]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:24.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:24 smithi192 ceph-mon[36653]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:24.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:24 smithi183 ceph-mon[36296]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:25.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 -- 172.21.15.76:0/1668938374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640108230 msgr2=0x7fc640108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 --2- 172.21.15.76:0/1668938374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640108230 0x7fc640108610 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0099c0 tx=0x7fc63c009cd0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 -- 172.21.15.76:0/1668938374 shutdown_connections 2024-03-23T15:42:25.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 --2- 172.21.15.76:0/1668938374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc640102c10 0x7fc64010f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 --2- 172.21.15.76:0/1668938374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640108230 0x7fc640108610 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 --2- 172.21.15.76:0/1668938374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc640102250 0x7fc6401026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 -- 172.21.15.76:0/1668938374 >> 172.21.15.76:0/1668938374 conn(0x7fc640079330 msgr2=0x7fc640079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:25.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 -- 172.21.15.76:0/1668938374 shutdown_connections 2024-03-23T15:42:25.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.003+0000 7fc647320700 1 -- 172.21.15.76:0/1668938374 wait complete. 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 Processor -- start 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 -- start start 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640102250 0x7fc6401986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc640102c10 0x7fc640198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc640108230 0x7fc64019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc64019d530 con 0x7fc640102250 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc64019d6a0 con 0x7fc640102c10 2024-03-23T15:42:25.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.004+0000 7fc647320700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc64019d860 con 0x7fc640108230 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640102250 0x7fc6401986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640102250 0x7fc6401986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42546/0 (socket says 172.21.15.76:42546) 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 -- 172.21.15.76:0/4156731858 learned_addr learned my addr 172.21.15.76:0/4156731858 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6448bb700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc640102c10 0x7fc640198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6458bd700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc640108230 0x7fc64019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 -- 172.21.15.76:0/4156731858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc640108230 msgr2=0x7fc64019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc640108230 0x7fc64019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 -- 172.21.15.76:0/4156731858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc640102c10 msgr2=0x7fc640198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc640102c10 0x7fc640198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 -- 172.21.15.76:0/4156731858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c0095f0 con 0x7fc640102250 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6450bc700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640102250 0x7fc6401986a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006010 tx=0x7fc63c005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6367fc700 1 -- 172.21.15.76:0/4156731858 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c01d070 con 0x7fc640102250 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6367fc700 1 -- 172.21.15.76:0/4156731858 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc63c0056c0 con 0x7fc640102250 2024-03-23T15:42:25.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc6367fc700 1 -- 172.21.15.76:0/4156731858 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00fc20 con 0x7fc640102250 2024-03-23T15:42:25.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64019dae0 con 0x7fc640102250 2024-03-23T15:42:25.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.005+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc640074430 con 0x7fc640102250 2024-03-23T15:42:25.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.007+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc64010a710 con 0x7fc640102250 2024-03-23T15:42:25.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.007+0000 7fc6367fc700 1 -- 172.21.15.76:0/4156731858 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fc63c00b790 con 0x7fc640102250 2024-03-23T15:42:25.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.008+0000 7fc6367fc700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc62c0a21e0 0x7fc62c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.008+0000 7fc6448bb700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc62c0a21e0 0x7fc62c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.008+0000 7fc6367fc700 1 -- 172.21.15.76:0/4156731858 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc63c0ce970 con 0x7fc640102250 2024-03-23T15:42:25.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.009+0000 7fc6448bb700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc62c0a21e0 0x7fc62c0a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc640079910 tx=0x7fc6300073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:25.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.014+0000 7fc6367fc700 1 -- 172.21.15.76:0/4156731858 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc63c05d280 con 0x7fc640102250 2024-03-23T15:42:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.218+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc6401999b0 con 0x7fc640102250 2024-03-23T15:42:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.218+0000 7fc6367fc700 1 -- 172.21.15.76:0/4156731858 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc63c097850 con 0x7fc640102250 2024-03-23T15:42:25.220 INFO:teuthology.orchestra.run.smithi076.stdout:309237645344 2024-03-23T15:42:25.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc62c0a21e0 msgr2=0x7fc62c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc62c0a21e0 0x7fc62c0a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc640079910 tx=0x7fc6300073d0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640102250 msgr2=0x7fc6401986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640102250 0x7fc6401986a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006010 tx=0x7fc63c005280 comp rx=0 tx=0).stop 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 shutdown_connections 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc62c0a21e0 0x7fc62c0a46a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc640108230 0x7fc64019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc640102250 0x7fc6401986a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 --2- 172.21.15.76:0/4156731858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc640102c10 0x7fc640198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 >> 172.21.15.76:0/4156731858 conn(0x7fc640079330 msgr2=0x7fc640100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 shutdown_connections 2024-03-23T15:42:25.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.221+0000 7fc647320700 1 -- 172.21.15.76:0/4156731858 wait complete. 2024-03-23T15:42:25.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 -- 172.21.15.76:0/414163919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 msgr2=0x7f21e4106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 --2- 172.21.15.76:0/414163919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4106e40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009990 tx=0x7f21d4009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 -- 172.21.15.76:0/414163919 shutdown_connections 2024-03-23T15:42:25.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 --2- 172.21.15.76:0/414163919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e4107520 0x7f21e410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 --2- 172.21.15.76:0/414163919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102440 0x7f21e4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 --2- 172.21.15.76:0/414163919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4106e40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 -- 172.21.15.76:0/414163919 >> 172.21.15.76:0/414163919 conn(0x7f21e40fdcb0 msgr2=0x7f21e41000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:25.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 -- 172.21.15.76:0/414163919 shutdown_connections 2024-03-23T15:42:25.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.247+0000 7f21ebd87700 1 -- 172.21.15.76:0/414163919 wait complete. 2024-03-23T15:42:25.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 Processor -- start 2024-03-23T15:42:25.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 -- start start 2024-03-23T15:42:25.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102440 0x7f21e4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e4107520 0x7f21e419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f21e419d600 con 0x7f21e4102df0 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21e419d770 con 0x7f21e4107520 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ebd87700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f21e419d930 con 0x7f21e4102440 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42570/0 (socket says 172.21.15.76:42570) 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 -- 172.21.15.76:0/2871131572 learned_addr learned my addr 172.21.15.76:0/2871131572 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9b23700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102440 0x7f21e4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21ea324700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e4107520 0x7f21e419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 -- 172.21.15.76:0/2871131572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102440 msgr2=0x7f21e4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102440 0x7f21e4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 -- 172.21.15.76:0/2871131572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e4107520 msgr2=0x7f21e419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e4107520 0x7f21e419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9322700 1 -- 172.21.15.76:0/2871131572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e0009520 con 0x7f21e4102df0 2024-03-23T15:42:25.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.248+0000 7f21e9b23700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102440 0x7f21e4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:25.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.249+0000 7f21e9322700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4198c60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:25.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.249+0000 7f21daffd700 1 -- 172.21.15.76:0/2871131572 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d401d070 con 0x7f21e4102df0 2024-03-23T15:42:25.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.249+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc009c40 con 0x7f21e4102df0 2024-03-23T15:42:25.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.249+0000 7f21daffd700 1 -- 172.21.15.76:0/2871131572 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d400b610 con 0x7f21e4102df0 2024-03-23T15:42:25.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.249+0000 7f21daffd700 1 -- 172.21.15.76:0/2871131572 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40173f0 con 0x7f21e4102df0 2024-03-23T15:42:25.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.249+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d40095f0 con 0x7f21e4102df0 2024-03-23T15:42:25.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.250+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e410a840 con 0x7f21e4102df0 2024-03-23T15:42:25.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.251+0000 7f21daffd700 1 -- 172.21.15.76:0/2871131572 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f21d4021610 con 0x7f21e4102df0 2024-03-23T15:42:25.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.252+0000 7f21daffd700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f21d00a21e0 0x7f21d00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:25.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.252+0000 7f21e9b23700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f21d00a21e0 0x7f21d00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:25.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.252+0000 7f21daffd700 1 -- 172.21.15.76:0/2871131572 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f21d40cf630 con 0x7f21e4102df0 2024-03-23T15:42:25.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.252+0000 7f21e9b23700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f21d00a21e0 0x7f21d00a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009fd0 tx=0x7f21e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.259+0000 7f21daffd700 1 -- 172.21.15.76:0/2871131572 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f21d4098510 con 0x7f21e4102df0 2024-03-23T15:42:25.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.460+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f21e4199ae0 con 0x7f21e4102df0 2024-03-23T15:42:25.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.461+0000 7f21daffd700 1 -- 172.21.15.76:0/2871131572 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f21d4026020 con 0x7f21e4102df0 2024-03-23T15:42:25.463 INFO:teuthology.orchestra.run.smithi076.stdout:219043332142 2024-03-23T15:42:25.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f21d00a21e0 msgr2=0x7f21d00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f21d00a21e0 0x7f21d00a46a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009fd0 tx=0x7f21e0009260 comp rx=0 tx=0).stop 2024-03-23T15:42:25.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 msgr2=0x7f21e4198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4198c60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d40057c0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 shutdown_connections 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f21d00a21e0 0x7f21d00a46a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21e4107520 0x7f21e419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102440 0x7f21e4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 --2- 172.21.15.76:0/2871131572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21e4102df0 0x7f21e4198c60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 >> 172.21.15.76:0/2871131572 conn(0x7f21e40fdcb0 msgr2=0x7f21e40fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 shutdown_connections 2024-03-23T15:42:25.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:25.463+0000 7f21ebd87700 1 -- 172.21.15.76:0/2871131572 wait complete. 2024-03-23T15:42:25.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/856476802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T15:42:25.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/4156731858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T15:42:25.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2871131572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T15:42:25.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:25 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/856476802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T15:42:25.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:25 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/4156731858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T15:42:25.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:25 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2871131572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T15:42:25.897 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645337 got 309237645344 for osd.10 2024-03-23T15:42:25.897 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:25 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/856476802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T15:42:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:25 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/4156731858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-23T15:42:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:25 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2871131572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-23T15:42:26.096 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:26.553 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182466 for osd.3 2024-03-23T15:42:26.553 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:26.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:26 smithi076 ceph-mon[32176]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:26.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:26 smithi183 ceph-mon[36296]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:26.970 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:26 smithi192 ceph-mon[36653]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:26.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070499 got 253403070505 for osd.8 2024-03-23T15:42:26.982 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:27.831 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332136 got 219043332142 for osd.7 2024-03-23T15:42:27.831 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:28.082 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 -- 172.21.15.76:0/2835616362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b41068c0 msgr2=0x7f99b4106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 --2- 172.21.15.76:0/2835616362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b41068c0 0x7f99b4106ca0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f99a800aaa0 tx=0x7f99a800adb0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 -- 172.21.15.76:0/2835616362 shutdown_connections 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 --2- 172.21.15.76:0/2835616362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99b4101280 0x7f99b410d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 --2- 172.21.15.76:0/2835616362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b41068c0 0x7f99b4106ca0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 --2- 172.21.15.76:0/2835616362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b41008c0 0x7f99b4100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 -- 172.21.15.76:0/2835616362 >> 172.21.15.76:0/2835616362 conn(0x7f99b4077700 msgr2=0x7f99b40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 -- 172.21.15.76:0/2835616362 shutdown_connections 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.091+0000 7f99babad700 1 -- 172.21.15.76:0/2835616362 wait complete. 2024-03-23T15:42:28.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 Processor -- start 2024-03-23T15:42:28.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 -- start start 2024-03-23T15:42:28.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b41008c0 0x7f99b41963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:28.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b4101280 0x7f99b4196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:28.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99b41068c0 0x7f99b4196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:28.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41a6b40 con 0x7f99b4101280 2024-03-23T15:42:28.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41a6cb0 con 0x7f99b41008c0 2024-03-23T15:42:28.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99babad700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41a6e70 con 0x7f99b41068c0 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99b3fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b4101280 0x7f99b4196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99b3fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b4101280 0x7f99b4196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42592/0 (socket says 172.21.15.76:42592) 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99b3fff700 1 -- 172.21.15.76:0/3358024500 learned_addr learned my addr 172.21.15.76:0/3358024500 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99b8949700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b41008c0 0x7f99b41963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.092+0000 7f99b914a700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99b41068c0 0x7f99b4196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b8949700 1 -- 172.21.15.76:0/3358024500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99b41068c0 msgr2=0x7f99b4196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b8949700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99b41068c0 0x7f99b4196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b8949700 1 -- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b4101280 msgr2=0x7f99b4196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b8949700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b4101280 0x7f99b4196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b8949700 1 -- 172.21.15.76:0/3358024500 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a800a700 con 0x7f99b41008c0 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b3fff700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b4101280 0x7f99b4196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b914a700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99b41068c0 0x7f99b4196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b8949700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b41008c0 0x7f99b41963e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f99a8000c00 tx=0x7f99a800a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b1ffb700 1 -- 172.21.15.76:0/3358024500 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8003f60 con 0x7f99b41008c0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b41a70f0 con 0x7f99b41008c0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b1ffb700 1 -- 172.21.15.76:0/3358024500 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a80045a0 con 0x7f99b41008c0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99b1ffb700 1 -- 172.21.15.76:0/3358024500 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a800edb0 con 0x7f99b41008c0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.093+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b41a7560 con 0x7f99b41008c0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.094+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b4108ec0 con 0x7f99b41008c0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.095+0000 7f99b1ffb700 1 -- 172.21.15.76:0/3358024500 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f99a800e450 con 0x7f99b41008c0 2024-03-23T15:42:28.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.096+0000 7f99b1ffb700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f99a40a2240 0x7f99a40a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:28.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.096+0000 7f99b3fff700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f99a40a2240 0x7f99a40a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:28.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.096+0000 7f99b1ffb700 1 -- 172.21.15.76:0/3358024500 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f99a8012070 con 0x7f99b41008c0 2024-03-23T15:42:28.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.096+0000 7f99b3fff700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f99a40a2240 0x7f99a40a4700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99b40ffe40 tx=0x7f99a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.101+0000 7f99b1ffb700 1 -- 172.21.15.76:0/3358024500 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f99a80986d0 con 0x7f99b41008c0 2024-03-23T15:42:28.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.308+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f99b4068ec0 con 0x7f99b41008c0 2024-03-23T15:42:28.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.308+0000 7f99b1ffb700 1 -- 172.21.15.76:0/3358024500 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99a809bd90 con 0x7f99b41008c0 2024-03-23T15:42:28.310 INFO:teuthology.orchestra.run.smithi076.stdout:287762808870 2024-03-23T15:42:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f99a40a2240 msgr2=0x7f99a40a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f99a40a2240 0x7f99a40a4700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99b40ffe40 tx=0x7f99a0009260 comp rx=0 tx=0).stop 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b41008c0 msgr2=0x7f99b41963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b41008c0 0x7f99b41963e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f99a8000c00 tx=0x7f99a800a690 comp rx=0 tx=0).stop 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 shutdown_connections 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f99a40a2240 0x7f99a40a4700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99b41068c0 0x7f99b4196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b41008c0 0x7f99b41963e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 --2- 172.21.15.76:0/3358024500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99b4101280 0x7f99b4196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.310+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 >> 172.21.15.76:0/3358024500 conn(0x7f99b4077700 msgr2=0x7f99b41037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.311+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 shutdown_connections 2024-03-23T15:42:28.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:28.311+0000 7f99babad700 1 -- 172.21.15.76:0/3358024500 wait complete. 2024-03-23T15:42:28.691 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:28 smithi076 ceph-mon[32176]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:28.691 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:28 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3358024500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T15:42:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:28 smithi183 ceph-mon[36296]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:28 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3358024500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T15:42:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:28 smithi192 ceph-mon[36653]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:28 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3358024500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-23T15:42:30.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:30 smithi076 ceph-mon[32176]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:30.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:30 smithi183 ceph-mon[36296]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:30.909 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808870 for osd.9 2024-03-23T15:42:30.909 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:31.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:30 smithi192 ceph-mon[36653]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 -- 172.21.15.76:0/381973896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b0107510 msgr2=0x7f74b010f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 --2- 172.21.15.76:0/381973896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b0107510 0x7f74b010f0e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b0a0 tx=0x7f74a800b3b0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 -- 172.21.15.76:0/381973896 shutdown_connections 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 --2- 172.21.15.76:0/381973896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b0107510 0x7f74b010f0e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 --2- 172.21.15.76:0/381973896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b01024e0 0x7f74b01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 --2- 172.21.15.76:0/381973896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0102e90 0x7f74b0106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 -- 172.21.15.76:0/381973896 >> 172.21.15.76:0/381973896 conn(0x7f74b0079290 msgr2=0x7f74b00796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 -- 172.21.15.76:0/381973896 shutdown_connections 2024-03-23T15:42:31.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.385+0000 7f74b6816700 1 -- 172.21.15.76:0/381973896 wait complete. 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 Processor -- start 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 -- start start 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b01024e0 0x7f74b0074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b0102e90 0x7f74b0074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0107510 0x7f74b0074f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01a6c80 con 0x7f74b01024e0 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01a6df0 con 0x7f74b0102e90 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b6816700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01a6fb0 con 0x7f74b0107510 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b01024e0 0x7f74b0074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b01024e0 0x7f74b0074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42614/0 (socket says 172.21.15.76:42614) 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 -- 172.21.15.76:0/2164257346 learned_addr learned my addr 172.21.15.76:0/2164257346 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b4db3700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0107510 0x7f74b0074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74af7fe700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b0102e90 0x7f74b0074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:31.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 -- 172.21.15.76:0/2164257346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0107510 msgr2=0x7f74b0074f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0107510 0x7f74b0074f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 -- 172.21.15.76:0/2164257346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b0102e90 msgr2=0x7f74b0074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b0102e90 0x7f74b0074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74affff700 1 -- 172.21.15.76:0/2164257346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a8009d00 con 0x7f74b01024e0 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.386+0000 7f74b4db3700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0107510 0x7f74b0074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.387+0000 7f74affff700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b01024e0 0x7f74b0074500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f74a000e9a0 tx=0x7f74a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.387+0000 7f74ad7fa700 1 -- 172.21.15.76:0/2164257346 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0009790 con 0x7f74b01024e0 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.387+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b01a7290 con 0x7f74b01024e0 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.387+0000 7f74ad7fa700 1 -- 172.21.15.76:0/2164257346 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74a0010430 con 0x7f74b01024e0 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.387+0000 7f74ad7fa700 1 -- 172.21.15.76:0/2164257346 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0018680 con 0x7f74b01024e0 2024-03-23T15:42:31.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.387+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b01a7790 con 0x7f74b01024e0 2024-03-23T15:42:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.388+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b010a7c0 con 0x7f74b01024e0 2024-03-23T15:42:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.389+0000 7f74ad7fa700 1 -- 172.21.15.76:0/2164257346 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f74a0010910 con 0x7f74b01024e0 2024-03-23T15:42:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.390+0000 7f74ad7fa700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74980a21e0 0x7f74980a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.390+0000 7f74af7fe700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74980a21e0 0x7f74980a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.390+0000 7f74ad7fa700 1 -- 172.21.15.76:0/2164257346 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f74a0014070 con 0x7f74b01024e0 2024-03-23T15:42:31.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.390+0000 7f74af7fe700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74980a21e0 0x7f74980a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74b01088b0 tx=0x7f74a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:31.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.394+0000 7f74ad7fa700 1 -- 172.21.15.76:0/2164257346 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f74a00973a0 con 0x7f74b01024e0 2024-03-23T15:42:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.592+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f74b0075a30 con 0x7f74b01024e0 2024-03-23T15:42:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.593+0000 7f74ad7fa700 1 -- 172.21.15.76:0/2164257346 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74a00105a0 con 0x7f74b01024e0 2024-03-23T15:42:31.595 INFO:teuthology.orchestra.run.smithi076.stdout:55834574927 2024-03-23T15:42:31.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74980a21e0 msgr2=0x7f74980a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:31.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74980a21e0 0x7f74980a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74b01088b0 tx=0x7f74a4009260 comp rx=0 tx=0).stop 2024-03-23T15:42:31.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b01024e0 msgr2=0x7f74b0074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:31.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b01024e0 0x7f74b0074500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f74a000e9a0 tx=0x7f74a000ecb0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 shutdown_connections 2024-03-23T15:42:31.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f74980a21e0 0x7f74980a46a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74b0107510 0x7f74b0074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74b01024e0 0x7f74b0074500 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 --2- 172.21.15.76:0/2164257346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b0102e90 0x7f74b0074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:31.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 >> 172.21.15.76:0/2164257346 conn(0x7f74b0079290 msgr2=0x7f74b01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:31.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 shutdown_connections 2024-03-23T15:42:31.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:31.595+0000 7f74b6816700 1 -- 172.21.15.76:0/2164257346 wait complete. 2024-03-23T15:42:32.222 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574927 for osd.1 2024-03-23T15:42:32.222 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:32.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:32 smithi076 ceph-mon[32176]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:32.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:32 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2164257346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T15:42:32.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:32 smithi183 ceph-mon[36296]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:32.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:32 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2164257346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T15:42:33.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:32 smithi192 ceph-mon[36653]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-23T15:42:33.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:32 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2164257346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T15:42:33.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 -- 172.21.15.76:0/1888137089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8106130 msgr2=0x7fc7c8106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:33.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 --2- 172.21.15.76:0/1888137089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8106130 0x7fc7c8106510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400b0d0 tx=0x7fc7c400b3e0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 -- 172.21.15.76:0/1888137089 shutdown_connections 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 --2- 172.21.15.76:0/1888137089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c8102670 0x7fc7c810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 --2- 172.21.15.76:0/1888137089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8106130 0x7fc7c8106510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 --2- 172.21.15.76:0/1888137089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 0x7fc7c8102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 -- 172.21.15.76:0/1888137089 >> 172.21.15.76:0/1888137089 conn(0x7fc7c8076480 msgr2=0x7fc7c8076890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 -- 172.21.15.76:0/1888137089 shutdown_connections 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 -- 172.21.15.76:0/1888137089 wait complete. 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.083+0000 7fc7cf515700 1 Processor -- start 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cf515700 1 -- start start 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cf515700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 0x7fc7c8102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cf515700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8102670 0x7fc7c8103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cf515700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c8106130 0x7fc7c81038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cf515700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81a6c70 con 0x7fc7c8102670 2024-03-23T15:42:33.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cf515700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81a6de0 con 0x7fc7c8101cb0 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cf515700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c81a6fa0 con 0x7fc7c8106130 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7ccab0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8102670 0x7fc7c8103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 0x7fc7c8102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7ccab0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8102670 0x7fc7c8103370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42638/0 (socket says 172.21.15.76:42638) 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 0x7fc7c8102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:41442/0 (socket says 172.21.15.76:41442) 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7ccab0700 1 -- 172.21.15.76:0/2513798373 learned_addr learned my addr 172.21.15.76:0/2513798373 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cdab2700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c8106130 0x7fc7c81038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 -- 172.21.15.76:0/2513798373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c8106130 msgr2=0x7fc7c81038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c8106130 0x7fc7c81038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 -- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8102670 msgr2=0x7fc7c8103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8102670 0x7fc7c8103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 -- 172.21.15.76:0/2513798373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c4009d00 con 0x7fc7c8101cb0 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7cd2b1700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 0x7fc7c8102e30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4000f80 tx=0x7fc7c400bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:33.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.084+0000 7fc7be7fc700 1 -- 172.21.15.76:0/2513798373 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400e040 con 0x7fc7c8101cb0 2024-03-23T15:42:33.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.085+0000 7fc7be7fc700 1 -- 172.21.15.76:0/2513798373 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c40079d0 con 0x7fc7c8101cb0 2024-03-23T15:42:33.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.085+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c81a7220 con 0x7fc7c8101cb0 2024-03-23T15:42:33.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.085+0000 7fc7be7fc700 1 -- 172.21.15.76:0/2513798373 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c401d480 con 0x7fc7c8101cb0 2024-03-23T15:42:33.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.085+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c81a7630 con 0x7fc7c8101cb0 2024-03-23T15:42:33.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.085+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c810a710 con 0x7fc7c8101cb0 2024-03-23T15:42:33.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.087+0000 7fc7be7fc700 1 -- 172.21.15.76:0/2513798373 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fc7c4005190 con 0x7fc7c8101cb0 2024-03-23T15:42:33.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.087+0000 7fc7be7fc700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7b40a21e0 0x7fc7b40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:33.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.087+0000 7fc7ccab0700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7b40a21e0 0x7fc7b40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:33.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.087+0000 7fc7be7fc700 1 -- 172.21.15.76:0/2513798373 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc7c40ce890 con 0x7fc7c8101cb0 2024-03-23T15:42:33.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.088+0000 7fc7ccab0700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7b40a21e0 0x7fc7b40a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8076b30 tx=0x7fc7b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:33.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.091+0000 7fc7be7fc700 1 -- 172.21.15.76:0/2513798373 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc7c4097800 con 0x7fc7c8101cb0 2024-03-23T15:42:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.312+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc7c81111b0 con 0x7fc7c8101cb0 2024-03-23T15:42:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.313+0000 7fc7be7fc700 1 -- 172.21.15.76:0/2513798373 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7c409aec0 con 0x7fc7c8101cb0 2024-03-23T15:42:33.467 INFO:teuthology.orchestra.run.smithi076.stdout:34359738451 2024-03-23T15:42:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7b40a21e0 msgr2=0x7fc7b40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7b40a21e0 0x7fc7b40a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8076b30 tx=0x7fc7b8009260 comp rx=0 tx=0).stop 2024-03-23T15:42:33.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 msgr2=0x7fc7c8102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 0x7fc7c8102e30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4000f80 tx=0x7fc7c400bd50 comp rx=0 tx=0).stop 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 shutdown_connections 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7b40a21e0 0x7fc7b40a46a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c8106130 0x7fc7c81038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c8101cb0 0x7fc7c8102e30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 --2- 172.21.15.76:0/2513798373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7c8102670 0x7fc7c8103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 >> 172.21.15.76:0/2513798373 conn(0x7fc7c8076480 msgr2=0x7fc7c8100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 shutdown_connections 2024-03-23T15:42:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:33.316+0000 7fc7cf515700 1 -- 172.21.15.76:0/2513798373 wait complete. 2024-03-23T15:42:33.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:33 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2513798373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T15:42:33.904 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:33 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2513798373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T15:42:33.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738451 for osd.0 2024-03-23T15:42:33.936 DEBUG:teuthology.parallel:result is None 2024-03-23T15:42:33.936 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-23T15:42:33.936 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T15:42:34.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:33 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2513798373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T15:42:34.199 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:34 smithi076 ceph-mon[32176]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:34.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:34 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:34 smithi183 ceph-mon[36296]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:34.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:34 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:35.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:34 smithi192 ceph-mon[36653]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:35.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:35.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:34 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:35.662 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:35 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:42:35.662 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:35 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:42:35.662 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:35 smithi076 ceph-mon[32176]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:35.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:35 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:42:35.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:35 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:42:35.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:35 smithi183 ceph-mon[36296]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2668407475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 msgr2=0x7fa5c4108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2668407475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c4108610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80099c0 tx=0x7fa5b8009cd0 comp rx=0 tx=0).stop 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2668407475 shutdown_connections 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2668407475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 0x7fa5c410f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2668407475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c4108610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2668407475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c4102250 0x7fa5c41026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2668407475 >> 172.21.15.76:0/2668407475 conn(0x7fa5c4079330 msgr2=0x7fa5c4079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2668407475 shutdown_connections 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.886+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2668407475 wait complete. 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.887+0000 7fa5c9b51700 1 Processor -- start 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.887+0000 7fa5c9b51700 1 -- start start 2024-03-23T15:42:35.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c9b51700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c4102250 0x7fa5c4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c9b51700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 0x7fa5c4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c9b51700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c9b51700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c419d540 con 0x7fa5c4108230 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c9b51700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c419d6b0 con 0x7fa5c4102c10 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c9b51700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c419d870 con 0x7fa5c4102250 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 0x7fa5c4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43878/0 (socket says 172.21.15.76:43878) 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c2ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 0x7fa5c4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:57818/0 (socket says 172.21.15.76:57818) 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 -- 172.21.15.76:0/2923537299 learned_addr learned my addr 172.21.15.76:0/2923537299 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:35.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c37fe700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c4102250 0x7fa5c4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:35.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 -- 172.21.15.76:0/2923537299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c4102250 msgr2=0x7fa5c4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:35.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c4102250 0x7fa5c4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:35.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 -- 172.21.15.76:0/2923537299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 msgr2=0x7fa5c4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:35.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 0x7fa5c4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:35.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 -- 172.21.15.76:0/2923537299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b80095f0 con 0x7fa5c4108230 2024-03-23T15:42:35.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c2ffd700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 0x7fa5c4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:35.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c3fff700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c419cf30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc007ee0 tx=0x7fa5bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:35.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.888+0000 7fa5c0ff9700 1 -- 172.21.15.76:0/2923537299 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00dd20 con 0x7fa5c4108230 2024-03-23T15:42:35.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.889+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419db50 con 0x7fa5c4108230 2024-03-23T15:42:35.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.889+0000 7fa5c0ff9700 1 -- 172.21.15.76:0/2923537299 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5bc00f040 con 0x7fa5c4108230 2024-03-23T15:42:35.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.889+0000 7fa5c0ff9700 1 -- 172.21.15.76:0/2923537299 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0048e0 con 0x7fa5c4108230 2024-03-23T15:42:35.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.889+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a53b0 con 0x7fa5c4108230 2024-03-23T15:42:35.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.890+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c410a710 con 0x7fa5c4108230 2024-03-23T15:42:35.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.891+0000 7fa5c0ff9700 1 -- 172.21.15.76:0/2923537299 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fa5bc003a00 con 0x7fa5c4108230 2024-03-23T15:42:35.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.891+0000 7fa5c0ff9700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa5ac0a2260 0x7fa5ac0a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:35.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.891+0000 7fa5c37fe700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa5ac0a2260 0x7fa5ac0a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:35.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.891+0000 7fa5c0ff9700 1 -- 172.21.15.76:0/2923537299 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa5bc0cddd0 con 0x7fa5c4108230 2024-03-23T15:42:35.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.892+0000 7fa5c37fe700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa5ac0a2260 0x7fa5ac0a4720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009990 tx=0x7fa5b8005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:35.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:35.895+0000 7fa5c0ff9700 1 -- 172.21.15.76:0/2923537299 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa5bc096c20 con 0x7fa5c4108230 2024-03-23T15:42:36.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:35 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:42:36.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:35 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:42:36.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:35 smithi192 ceph-mon[36653]: from='mgr.14150 172.21.15.76:0/1532181894' entity='mgr.x' 2024-03-23T15:42:36.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.085+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5c4112230 con 0x7fa5ac0a2260 2024-03-23T15:42:36.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.093+0000 7fa5c0ff9700 1 -- 172.21.15.76:0/2923537299 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258978 (secure 0 0 0) 0x7fa5c4112230 con 0x7fa5ac0a2260 2024-03-23T15:42:36.099 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa5ac0a2260 msgr2=0x7fa5ac0a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa5ac0a2260 0x7fa5ac0a4720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009990 tx=0x7fa5b8005bc0 comp rx=0 tx=0).stop 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 msgr2=0x7fa5c419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c419cf30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc007ee0 tx=0x7fa5bc00d3b0 comp rx=0 tx=0).stop 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 shutdown_connections 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa5ac0a2260 0x7fa5ac0a4720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5c4108230 0x7fa5c419cf30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c4102250 0x7fa5c4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 --2- 172.21.15.76:0/2923537299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4102c10 0x7fa5c4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 >> 172.21.15.76:0/2923537299 conn(0x7fa5c4079330 msgr2=0x7fa5c4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 shutdown_connections 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:36.096+0000 7fa5c9b51700 1 -- 172.21.15.76:0/2923537299 wait complete. 2024-03-23T15:42:36.101 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-23T15:42:36.777 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:36 smithi076 ceph-mon[32176]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:36.804 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-03-23T15:42:35.495119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":593,"num_read_kb":416,"num_write":339,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369108,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572716,"statfs":{"total":1151940427776,"available":1151562461184,"internally_reserved":0,"allocated":11833344,"data_stored":7436703,"data_compressed":182127,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002955"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622644+0000","last_change":"2024-03-23T15:41:18.651731+0000","last_active":"2024-03-23T15:42:16.622644+0000","last_peered":"2024-03-23T15:42:16.622644+0000","last_clean":"2024-03-23T15:42:16.622644+0000","last_became_active":"2024-03-23T15:41:18.651312+0000","last_became_peered":"2024-03-23T15:41:18.651312+0000","last_unstale":"2024-03-23T15:42:16.622644+0000","last_undegraded":"2024-03-23T15:42:16.622644+0000","last_fullsized":"2024-03-23T15:42:16.622644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:39:20.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622436+0000","last_change":"2024-03-23T15:41:18.655736+0000","last_active":"2024-03-23T15:42:16.622436+0000","last_peered":"2024-03-23T15:42:16.622436+0000","last_clean":"2024-03-23T15:42:16.622436+0000","last_became_active":"2024-03-23T15:41:18.655389+0000","last_became_peered":"2024-03-23T15:41:18.655389+0000","last_unstale":"2024-03-23T15:42:16.622436+0000","last_undegraded":"2024-03-23T15:42:16.622436+0000","last_fullsized":"2024-03-23T15:42:16.622436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:35:56.260056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622442+0000","last_change":"2024-03-23T15:41:18.650709+0000","last_active":"2024-03-23T15:42:16.622442+0000","last_peered":"2024-03-23T15:42:16.622442+0000","last_clean":"2024-03-23T15:42:16.622442+0000","last_became_active":"2024-03-23T15:41:18.650316+0000","last_became_peered":"2024-03-23T15:41:18.650316+0000","last_unstale":"2024-03-23T15:42:16.622442+0000","last_undegraded":"2024-03-23T15:42:16.622442+0000","last_fullsized":"2024-03-23T15:42:16.622442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:50:57.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689194+0000","last_change":"2024-03-23T15:41:18.652216+0000","last_active":"2024-03-23T15:42:16.689194+0000","last_peered":"2024-03-23T15:42:16.689194+0000","last_clean":"2024-03-23T15:42:16.689194+0000","last_became_active":"2024-03-23T15:41:18.650525+0000","last_became_peered":"2024-03-23T15:41:18.650525+0000","last_unstale":"2024-03-23T15:42:16.689194+0000","last_undegraded":"2024-03-23T15:42:16.689194+0000","last_fullsized":"2024-03-23T15:42:16.689194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:16.458257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620128+0000","last_change":"2024-03-23T15:41:18.649146+0000","last_active":"2024-03-23T15:42:16.620128+0000","last_peered":"2024-03-23T15:42:16.620128+0000","last_clean":"2024-03-23T15:42:16.620128+0000","last_became_active":"2024-03-23T15:41:18.648728+0000","last_became_peered":"2024-03-23T15:41:18.648728+0000","last_unstale":"2024-03-23T15:42:16.620128+0000","last_undegraded":"2024-03-23T15:42:16.620128+0000","last_fullsized":"2024-03-23T15:42:16.620128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:56.251250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625127+0000","last_change":"2024-03-23T15:41:18.652957+0000","last_active":"2024-03-23T15:42:16.625127+0000","last_peered":"2024-03-23T15:42:16.625127+0000","last_clean":"2024-03-23T15:42:16.625127+0000","last_became_active":"2024-03-23T15:41:18.652578+0000","last_became_peered":"2024-03-23T15:41:18.652578+0000","last_unstale":"2024-03-23T15:42:16.625127+0000","last_undegraded":"2024-03-23T15:42:16.625127+0000","last_fullsized":"2024-03-23T15:42:16.625127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:46:38.961554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620222+0000","last_change":"2024-03-23T15:41:18.646134+0000","last_active":"2024-03-23T15:42:16.620222+0000","last_peered":"2024-03-23T15:42:16.620222+0000","last_clean":"2024-03-23T15:42:16.620222+0000","last_became_active":"2024-03-23T15:41:18.645680+0000","last_became_peered":"2024-03-23T15:41:18.645680+0000","last_unstale":"2024-03-23T15:42:16.620222+0000","last_undegraded":"2024-03-23T15:42:16.620222+0000","last_fullsized":"2024-03-23T15:42:16.620222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:34:31.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627918+0000","last_change":"2024-03-23T15:41:18.646143+0000","last_active":"2024-03-23T15:42:16.627918+0000","last_peered":"2024-03-23T15:42:16.627918+0000","last_clean":"2024-03-23T15:42:16.627918+0000","last_became_active":"2024-03-23T15:41:18.645771+0000","last_became_peered":"2024-03-23T15:41:18.645771+0000","last_unstale":"2024-03-23T15:42:16.627918+0000","last_undegraded":"2024-03-23T15:42:16.627918+0000","last_fullsized":"2024-03-23T15:42:16.627918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:39.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623687+0000","last_change":"2024-03-23T15:41:18.665083+0000","last_active":"2024-03-23T15:42:16.623687+0000","last_peered":"2024-03-23T15:42:16.623687+0000","last_clean":"2024-03-23T15:42:16.623687+0000","last_became_active":"2024-03-23T15:41:18.663143+0000","last_became_peered":"2024-03-23T15:41:18.663143+0000","last_unstale":"2024-03-23T15:42:16.623687+0000","last_undegraded":"2024-03-23T15:42:16.623687+0000","last_fullsized":"2024-03-23T15:42:16.623687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:52:23.421645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625458+0000","last_change":"2024-03-23T15:41:18.650014+0000","last_active":"2024-03-23T15:42:16.625458+0000","last_peered":"2024-03-23T15:42:16.625458+0000","last_clean":"2024-03-23T15:42:16.625458+0000","last_became_active":"2024-03-23T15:41:18.649673+0000","last_became_peered":"2024-03-23T15:41:18.649673+0000","last_unstale":"2024-03-23T15:42:16.625458+0000","last_undegraded":"2024-03-23T15:42:16.625458+0000","last_fullsized":"2024-03-23T15:42:16.625458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:30.387993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620880+0000","last_change":"2024-03-23T15:41:18.665622+0000","last_active":"2024-03-23T15:42:16.620880+0000","last_peered":"2024-03-23T15:42:16.620880+0000","last_clean":"2024-03-23T15:42:16.620880+0000","last_became_active":"2024-03-23T15:41:18.665274+0000","last_became_peered":"2024-03-23T15:41:18.665274+0000","last_unstale":"2024-03-23T15:42:16.620880+0000","last_undegraded":"2024-03-23T15:42:16.620880+0000","last_fullsized":"2024-03-23T15:42:16.620880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:12:18.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689095+0000","last_change":"2024-03-23T15:41:18.650888+0000","last_active":"2024-03-23T15:42:16.689095+0000","last_peered":"2024-03-23T15:42:16.689095+0000","last_clean":"2024-03-23T15:42:16.689095+0000","last_became_active":"2024-03-23T15:41:18.650363+0000","last_became_peered":"2024-03-23T15:41:18.650363+0000","last_unstale":"2024-03-23T15:42:16.689095+0000","last_undegraded":"2024-03-23T15:42:16.689095+0000","last_fullsized":"2024-03-23T15:42:16.689095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:41.289697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625765+0000","last_change":"2024-03-23T15:41:18.634183+0000","last_active":"2024-03-23T15:42:16.625765+0000","last_peered":"2024-03-23T15:42:16.625765+0000","last_clean":"2024-03-23T15:42:16.625765+0000","last_became_active":"2024-03-23T15:41:18.633849+0000","last_became_peered":"2024-03-23T15:41:18.633849+0000","last_unstale":"2024-03-23T15:42:16.625765+0000","last_undegraded":"2024-03-23T15:42:16.625765+0000","last_fullsized":"2024-03-23T15:42:16.625765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:13.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622197+0000","last_change":"2024-03-23T15:41:18.645269+0000","last_active":"2024-03-23T15:42:16.622197+0000","last_peered":"2024-03-23T15:42:16.622197+0000","last_clean":"2024-03-23T15:42:16.622197+0000","last_became_active":"2024-03-23T15:41:18.644940+0000","last_became_peered":"2024-03-23T15:41:18.644940+0000","last_unstale":"2024-03-23T15:42:16.622197+0000","last_undegraded":"2024-03-23T15:42:16.622197+0000","last_fullsized":"2024-03-23T15:42:16.622197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:26.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622000+0000","last_change":"2024-03-23T15:41:18.652789+0000","last_active":"2024-03-23T15:42:16.622000+0000","last_peered":"2024-03-23T15:42:16.622000+0000","last_clean":"2024-03-23T15:42:16.622000+0000","last_became_active":"2024-03-23T15:41:18.651520+0000","last_became_peered":"2024-03-23T15:41:18.651520+0000","last_unstale":"2024-03-23T15:42:16.622000+0000","last_undegraded":"2024-03-23T15:42:16.622000+0000","last_fullsized":"2024-03-23T15:42:16.622000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:00.388217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622362+0000","last_change":"2024-03-23T15:41:18.660398+0000","last_active":"2024-03-23T15:42:16.622362+0000","last_peered":"2024-03-23T15:42:16.622362+0000","last_clean":"2024-03-23T15:42:16.622362+0000","last_became_active":"2024-03-23T15:41:18.660084+0000","last_became_peered":"2024-03-23T15:41:18.660084+0000","last_unstale":"2024-03-23T15:42:16.622362+0000","last_undegraded":"2024-03-23T15:42:16.622362+0000","last_fullsized":"2024-03-23T15:42:16.622362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:04:33.115173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622743+0000","last_change":"2024-03-23T15:41:18.663192+0000","last_active":"2024-03-23T15:42:16.622743+0000","last_peered":"2024-03-23T15:42:16.622743+0000","last_clean":"2024-03-23T15:42:16.622743+0000","last_became_active":"2024-03-23T15:41:18.662844+0000","last_became_peered":"2024-03-23T15:41:18.662844+0000","last_unstale":"2024-03-23T15:42:16.622743+0000","last_undegraded":"2024-03-23T15:42:16.622743+0000","last_fullsized":"2024-03-23T15:42:16.622743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:24.260706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841865+0000","last_change":"2024-03-23T15:41:18.646405+0000","last_active":"2024-03-23T15:42:16.841865+0000","last_peered":"2024-03-23T15:42:16.841865+0000","last_clean":"2024-03-23T15:42:16.841865+0000","last_became_active":"2024-03-23T15:41:18.645516+0000","last_became_peered":"2024-03-23T15:41:18.645516+0000","last_unstale":"2024-03-23T15:42:16.841865+0000","last_undegraded":"2024-03-23T15:42:16.841865+0000","last_fullsized":"2024-03-23T15:42:16.841865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:29:18.588190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843159+0000","last_change":"2024-03-23T15:42:16.843159+0000","last_active":"2024-03-23T15:42:16.843159+0000","last_peered":"2024-03-23T15:42:16.843159+0000","last_clean":"2024-03-23T15:42:16.843159+0000","last_became_active":"2024-03-23T15:42:16.842729+0000","last_became_peered":"2024-03-23T15:42:16.842729+0000","last_unstale":"2024-03-23T15:42:16.843159+0000","last_undegraded":"2024-03-23T15:42:16.843159+0000","last_fullsized":"2024-03-23T15:42:16.843159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:57:18.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841913+0000","last_change":"2024-03-23T15:41:17.655443+0000","last_active":"2024-03-23T15:42:16.841913+0000","last_peered":"2024-03-23T15:42:16.841913+0000","last_clean":"2024-03-23T15:42:16.841913+0000","last_became_active":"2024-03-23T15:41:17.654834+0000","last_became_peered":"2024-03-23T15:41:17.654834+0000","last_unstale":"2024-03-23T15:42:16.841913+0000","last_undegraded":"2024-03-23T15:42:16.841913+0000","last_fullsized":"2024-03-23T15:42:16.841913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:49:43.215644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628289+0000","last_change":"2024-03-23T15:41:18.650241+0000","last_active":"2024-03-23T15:42:16.628289+0000","last_peered":"2024-03-23T15:42:16.628289+0000","last_clean":"2024-03-23T15:42:16.628289+0000","last_became_active":"2024-03-23T15:41:18.649914+0000","last_became_peered":"2024-03-23T15:41:18.649914+0000","last_unstale":"2024-03-23T15:42:16.628289+0000","last_undegraded":"2024-03-23T15:42:16.628289+0000","last_fullsized":"2024-03-23T15:42:16.628289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:35:18.892649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628447+0000","last_change":"2024-03-23T15:41:17.663308+0000","last_active":"2024-03-23T15:42:16.628447+0000","last_peered":"2024-03-23T15:42:16.628447+0000","last_clean":"2024-03-23T15:42:16.628447+0000","last_became_active":"2024-03-23T15:41:17.662958+0000","last_became_peered":"2024-03-23T15:41:17.662958+0000","last_unstale":"2024-03-23T15:42:16.628447+0000","last_undegraded":"2024-03-23T15:42:16.628447+0000","last_fullsized":"2024-03-23T15:42:16.628447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:43:37.512893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622314+0000","last_change":"2024-03-23T15:41:18.651225+0000","last_active":"2024-03-23T15:42:16.622314+0000","last_peered":"2024-03-23T15:42:16.622314+0000","last_clean":"2024-03-23T15:42:16.622314+0000","last_became_active":"2024-03-23T15:41:18.650884+0000","last_became_peered":"2024-03-23T15:41:18.650884+0000","last_unstale":"2024-03-23T15:42:16.622314+0000","last_undegraded":"2024-03-23T15:42:16.622314+0000","last_fullsized":"2024-03-23T15:42:16.622314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:00.445203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620673+0000","last_change":"2024-03-23T15:41:17.662753+0000","last_active":"2024-03-23T15:42:16.620673+0000","last_peered":"2024-03-23T15:42:16.620673+0000","last_clean":"2024-03-23T15:42:16.620673+0000","last_became_active":"2024-03-23T15:41:17.662378+0000","last_became_peered":"2024-03-23T15:41:17.662378+0000","last_unstale":"2024-03-23T15:42:16.620673+0000","last_undegraded":"2024-03-23T15:42:16.620673+0000","last_fullsized":"2024-03-23T15:42:16.620673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:58:55.674739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622672+0000","last_change":"2024-03-23T15:41:18.659171+0000","last_active":"2024-03-23T15:42:16.622672+0000","last_peered":"2024-03-23T15:42:16.622672+0000","last_clean":"2024-03-23T15:42:16.622672+0000","last_became_active":"2024-03-23T15:41:18.658836+0000","last_became_peered":"2024-03-23T15:41:18.658836+0000","last_unstale":"2024-03-23T15:42:16.622672+0000","last_undegraded":"2024-03-23T15:42:16.622672+0000","last_fullsized":"2024-03-23T15:42:16.622672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:15.056528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.632016+0000","last_change":"2024-03-23T15:42:16.631598+0000","last_active":"2024-03-23T15:42:16.632016+0000","last_peered":"2024-03-23T15:42:16.632016+0000","last_clean":"2024-03-23T15:42:16.632016+0000","last_became_active":"2024-03-23T15:42:16.626457+0000","last_became_peered":"2024-03-23T15:42:16.626457+0000","last_unstale":"2024-03-23T15:42:16.632016+0000","last_undegraded":"2024-03-23T15:42:16.632016+0000","last_fullsized":"2024-03-23T15:42:16.632016+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:29.776204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623174+0000","last_change":"2024-03-23T15:41:18.648500+0000","last_active":"2024-03-23T15:42:16.623174+0000","last_peered":"2024-03-23T15:42:16.623174+0000","last_clean":"2024-03-23T15:42:16.623174+0000","last_became_active":"2024-03-23T15:41:18.648139+0000","last_became_peered":"2024-03-23T15:41:18.648139+0000","last_unstale":"2024-03-23T15:42:16.623174+0000","last_undegraded":"2024-03-23T15:42:16.623174+0000","last_fullsized":"2024-03-23T15:42:16.623174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:38.071057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:32.776274+0000","last_change":"2024-03-23T15:41:17.658471+0000","last_active":"2024-03-23T15:42:32.776274+0000","last_peered":"2024-03-23T15:42:32.776274+0000","last_clean":"2024-03-23T15:42:32.776274+0000","last_became_active":"2024-03-23T15:41:17.658142+0000","last_became_peered":"2024-03-23T15:41:17.658142+0000","last_unstale":"2024-03-23T15:42:32.776274+0000","last_undegraded":"2024-03-23T15:42:32.776274+0000","last_fullsized":"2024-03-23T15:42:32.776274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:23.224790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620645+0000","last_change":"2024-03-23T15:41:18.646046+0000","last_active":"2024-03-23T15:42:16.620645+0000","last_peered":"2024-03-23T15:42:16.620645+0000","last_clean":"2024-03-23T15:42:16.620645+0000","last_became_active":"2024-03-23T15:41:18.645727+0000","last_became_peered":"2024-03-23T15:41:18.645727+0000","last_unstale":"2024-03-23T15:42:16.620645+0000","last_undegraded":"2024-03-23T15:42:16.620645+0000","last_fullsized":"2024-03-23T15:42:16.620645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:55.552079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625081+0000","last_change":"2024-03-23T15:41:17.657285+0000","last_active":"2024-03-23T15:42:16.625081+0000","last_peered":"2024-03-23T15:42:16.625081+0000","last_clean":"2024-03-23T15:42:16.625081+0000","last_became_active":"2024-03-23T15:41:17.656942+0000","last_became_peered":"2024-03-23T15:41:17.656942+0000","last_unstale":"2024-03-23T15:42:16.625081+0000","last_undegraded":"2024-03-23T15:42:16.625081+0000","last_fullsized":"2024-03-23T15:42:16.625081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:18.107467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623937+0000","last_change":"2024-03-23T15:41:18.655175+0000","last_active":"2024-03-23T15:42:16.623937+0000","last_peered":"2024-03-23T15:42:16.623937+0000","last_clean":"2024-03-23T15:42:16.623937+0000","last_became_active":"2024-03-23T15:41:18.654846+0000","last_became_peered":"2024-03-23T15:41:18.654846+0000","last_unstale":"2024-03-23T15:42:16.623937+0000","last_undegraded":"2024-03-23T15:42:16.623937+0000","last_fullsized":"2024-03-23T15:42:16.623937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:43:10.008538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625643+0000","last_change":"2024-03-23T15:41:17.657391+0000","last_active":"2024-03-23T15:42:16.625643+0000","last_peered":"2024-03-23T15:42:16.625643+0000","last_clean":"2024-03-23T15:42:16.625643+0000","last_became_active":"2024-03-23T15:41:17.657081+0000","last_became_peered":"2024-03-23T15:41:17.657081+0000","last_unstale":"2024-03-23T15:42:16.625643+0000","last_undegraded":"2024-03-23T15:42:16.625643+0000","last_fullsized":"2024-03-23T15:42:16.625643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:08.744477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628032+0000","last_change":"2024-03-23T15:41:18.656268+0000","last_active":"2024-03-23T15:42:16.628032+0000","last_peered":"2024-03-23T15:42:16.628032+0000","last_clean":"2024-03-23T15:42:16.628032+0000","last_became_active":"2024-03-23T15:41:18.655910+0000","last_became_peered":"2024-03-23T15:41:18.655910+0000","last_unstale":"2024-03-23T15:42:16.628032+0000","last_undegraded":"2024-03-23T15:42:16.628032+0000","last_fullsized":"2024-03-23T15:42:16.628032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:38:57.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623201+0000","last_change":"2024-03-23T15:41:17.656555+0000","last_active":"2024-03-23T15:42:16.623201+0000","last_peered":"2024-03-23T15:42:16.623201+0000","last_clean":"2024-03-23T15:42:16.623201+0000","last_became_active":"2024-03-23T15:41:17.655732+0000","last_became_peered":"2024-03-23T15:41:17.655732+0000","last_unstale":"2024-03-23T15:42:16.623201+0000","last_undegraded":"2024-03-23T15:42:16.623201+0000","last_fullsized":"2024-03-23T15:42:16.623201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:53.839841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841655+0000","last_change":"2024-03-23T15:41:18.647334+0000","last_active":"2024-03-23T15:42:16.841655+0000","last_peered":"2024-03-23T15:42:16.841655+0000","last_clean":"2024-03-23T15:42:16.841655+0000","last_became_active":"2024-03-23T15:41:18.646679+0000","last_became_peered":"2024-03-23T15:41:18.646679+0000","last_unstale":"2024-03-23T15:42:16.841655+0000","last_undegraded":"2024-03-23T15:42:16.841655+0000","last_fullsized":"2024-03-23T15:42:16.841655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.710759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:31.774809+0000","last_change":"2024-03-23T15:41:17.655400+0000","last_active":"2024-03-23T15:42:31.774809+0000","last_peered":"2024-03-23T15:42:31.774809+0000","last_clean":"2024-03-23T15:42:31.774809+0000","last_became_active":"2024-03-23T15:41:17.655034+0000","last_became_peered":"2024-03-23T15:41:17.655034+0000","last_unstale":"2024-03-23T15:42:31.774809+0000","last_undegraded":"2024-03-23T15:42:31.774809+0000","last_fullsized":"2024-03-23T15:42:31.774809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:30.973048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623424+0000","last_change":"2024-03-23T15:41:18.660248+0000","last_active":"2024-03-23T15:42:16.623424+0000","last_peered":"2024-03-23T15:42:16.623424+0000","last_clean":"2024-03-23T15:42:16.623424+0000","last_became_active":"2024-03-23T15:41:18.659728+0000","last_became_peered":"2024-03-23T15:41:18.659728+0000","last_unstale":"2024-03-23T15:42:16.623424+0000","last_undegraded":"2024-03-23T15:42:16.623424+0000","last_fullsized":"2024-03-23T15:42:16.623424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:03.681499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620473+0000","last_change":"2024-03-23T15:41:17.659784+0000","last_active":"2024-03-23T15:42:16.620473+0000","last_peered":"2024-03-23T15:42:16.620473+0000","last_clean":"2024-03-23T15:42:16.620473+0000","last_became_active":"2024-03-23T15:41:17.659422+0000","last_became_peered":"2024-03-23T15:41:17.659422+0000","last_unstale":"2024-03-23T15:42:16.620473+0000","last_undegraded":"2024-03-23T15:42:16.620473+0000","last_fullsized":"2024-03-23T15:42:16.620473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:48.607487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622219+0000","last_change":"2024-03-23T15:41:18.651777+0000","last_active":"2024-03-23T15:42:16.622219+0000","last_peered":"2024-03-23T15:42:16.622219+0000","last_clean":"2024-03-23T15:42:16.622219+0000","last_became_active":"2024-03-23T15:41:18.651457+0000","last_became_peered":"2024-03-23T15:41:18.651457+0000","last_unstale":"2024-03-23T15:42:16.622219+0000","last_undegraded":"2024-03-23T15:42:16.622219+0000","last_fullsized":"2024-03-23T15:42:16.622219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:25:28.831493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623195+0000","last_change":"2024-03-23T15:41:17.656357+0000","last_active":"2024-03-23T15:42:16.623195+0000","last_peered":"2024-03-23T15:42:16.623195+0000","last_clean":"2024-03-23T15:42:16.623195+0000","last_became_active":"2024-03-23T15:41:17.655977+0000","last_became_peered":"2024-03-23T15:41:17.655977+0000","last_unstale":"2024-03-23T15:42:16.623195+0000","last_undegraded":"2024-03-23T15:42:16.623195+0000","last_fullsized":"2024-03-23T15:42:16.623195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:07:45.248954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626757+0000","last_change":"2024-03-23T15:42:16.626757+0000","last_active":"2024-03-23T15:42:16.626757+0000","last_peered":"2024-03-23T15:42:16.626757+0000","last_clean":"2024-03-23T15:42:16.626757+0000","last_became_active":"2024-03-23T15:42:16.626409+0000","last_became_peered":"2024-03-23T15:42:16.626409+0000","last_unstale":"2024-03-23T15:42:16.626757+0000","last_undegraded":"2024-03-23T15:42:16.626757+0000","last_fullsized":"2024-03-23T15:42:16.626757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:46.969160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":197,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:32.775903+0000","last_change":"2024-03-23T15:41:17.659838+0000","last_active":"2024-03-23T15:42:32.775903+0000","last_peered":"2024-03-23T15:42:32.775903+0000","last_clean":"2024-03-23T15:42:32.775903+0000","last_became_active":"2024-03-23T15:41:17.659326+0000","last_became_peered":"2024-03-23T15:41:17.659326+0000","last_unstale":"2024-03-23T15:42:32.775903+0000","last_undegraded":"2024-03-23T15:42:32.775903+0000","last_fullsized":"2024-03-23T15:42:32.775903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:07.651340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":150,"num_read_kb":134,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620436+0000","last_change":"2024-03-23T15:41:18.656629+0000","last_active":"2024-03-23T15:42:16.620436+0000","last_peered":"2024-03-23T15:42:16.620436+0000","last_clean":"2024-03-23T15:42:16.620436+0000","last_became_active":"2024-03-23T15:41:18.656196+0000","last_became_peered":"2024-03-23T15:41:18.656196+0000","last_unstale":"2024-03-23T15:42:16.620436+0000","last_undegraded":"2024-03-23T15:42:16.620436+0000","last_fullsized":"2024-03-23T15:42:16.620436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:30.013411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626655+0000","last_change":"2024-03-23T15:41:17.663999+0000","last_active":"2024-03-23T15:42:16.626655+0000","last_peered":"2024-03-23T15:42:16.626655+0000","last_clean":"2024-03-23T15:42:16.626655+0000","last_became_active":"2024-03-23T15:41:17.663039+0000","last_became_peered":"2024-03-23T15:41:17.663039+0000","last_unstale":"2024-03-23T15:42:16.626655+0000","last_undegraded":"2024-03-23T15:42:16.626655+0000","last_fullsized":"2024-03-23T15:42:16.626655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:00.271821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625635+0000","last_change":"2024-03-23T15:41:18.646822+0000","last_active":"2024-03-23T15:42:16.625635+0000","last_peered":"2024-03-23T15:42:16.625635+0000","last_clean":"2024-03-23T15:42:16.625635+0000","last_became_active":"2024-03-23T15:41:18.646476+0000","last_became_peered":"2024-03-23T15:41:18.646476+0000","last_unstale":"2024-03-23T15:42:16.625635+0000","last_undegraded":"2024-03-23T15:42:16.625635+0000","last_fullsized":"2024-03-23T15:42:16.625635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:18.352065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623150+0000","last_change":"2024-03-23T15:41:17.663507+0000","last_active":"2024-03-23T15:42:16.623150+0000","last_peered":"2024-03-23T15:42:16.623150+0000","last_clean":"2024-03-23T15:42:16.623150+0000","last_became_active":"2024-03-23T15:41:17.663122+0000","last_became_peered":"2024-03-23T15:41:17.663122+0000","last_unstale":"2024-03-23T15:42:16.623150+0000","last_undegraded":"2024-03-23T15:42:16.623150+0000","last_fullsized":"2024-03-23T15:42:16.623150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:32.202576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620544+0000","last_change":"2024-03-23T15:41:18.650110+0000","last_active":"2024-03-23T15:42:16.620544+0000","last_peered":"2024-03-23T15:42:16.620544+0000","last_clean":"2024-03-23T15:42:16.620544+0000","last_became_active":"2024-03-23T15:41:18.649737+0000","last_became_peered":"2024-03-23T15:41:18.649737+0000","last_unstale":"2024-03-23T15:42:16.620544+0000","last_undegraded":"2024-03-23T15:42:16.620544+0000","last_fullsized":"2024-03-23T15:42:16.620544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:31.610961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:32.777708+0000","last_change":"2024-03-23T15:41:17.653930+0000","last_active":"2024-03-23T15:42:32.777708+0000","last_peered":"2024-03-23T15:42:32.777708+0000","last_clean":"2024-03-23T15:42:32.777708+0000","last_became_active":"2024-03-23T15:41:17.652677+0000","last_became_peered":"2024-03-23T15:41:17.652677+0000","last_unstale":"2024-03-23T15:42:32.777708+0000","last_undegraded":"2024-03-23T15:42:32.777708+0000","last_fullsized":"2024-03-23T15:42:32.777708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:47.167052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843231+0000","last_change":"2024-03-23T15:42:16.843231+0000","last_active":"2024-03-23T15:42:16.843231+0000","last_peered":"2024-03-23T15:42:16.843231+0000","last_clean":"2024-03-23T15:42:16.843231+0000","last_became_active":"2024-03-23T15:42:16.842785+0000","last_became_peered":"2024-03-23T15:42:16.842785+0000","last_unstale":"2024-03-23T15:42:16.843231+0000","last_undegraded":"2024-03-23T15:42:16.843231+0000","last_fullsized":"2024-03-23T15:42:16.843231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:24:07.377828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620049+0000","last_change":"2024-03-23T15:41:17.662091+0000","last_active":"2024-03-23T15:42:16.620049+0000","last_peered":"2024-03-23T15:42:16.620049+0000","last_clean":"2024-03-23T15:42:16.620049+0000","last_became_active":"2024-03-23T15:41:17.661747+0000","last_became_peered":"2024-03-23T15:41:17.661747+0000","last_unstale":"2024-03-23T15:42:16.620049+0000","last_undegraded":"2024-03-23T15:42:16.620049+0000","last_fullsized":"2024-03-23T15:42:16.620049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:08.661262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627776+0000","last_change":"2024-03-23T15:42:16.627776+0000","last_active":"2024-03-23T15:42:16.627776+0000","last_peered":"2024-03-23T15:42:16.627776+0000","last_clean":"2024-03-23T15:42:16.627776+0000","last_became_active":"2024-03-23T15:42:16.627331+0000","last_became_peered":"2024-03-23T15:42:16.627331+0000","last_unstale":"2024-03-23T15:42:16.627776+0000","last_undegraded":"2024-03-23T15:42:16.627776+0000","last_fullsized":"2024-03-23T15:42:16.627776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:02:57.288827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620552+0000","last_change":"2024-03-23T15:41:17.660898+0000","last_active":"2024-03-23T15:42:16.620552+0000","last_peered":"2024-03-23T15:42:16.620552+0000","last_clean":"2024-03-23T15:42:16.620552+0000","last_became_active":"2024-03-23T15:41:17.660515+0000","last_became_peered":"2024-03-23T15:41:17.660515+0000","last_unstale":"2024-03-23T15:42:16.620552+0000","last_undegraded":"2024-03-23T15:42:16.620552+0000","last_fullsized":"2024-03-23T15:42:16.620552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:58.911321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622448+0000","last_change":"2024-03-23T15:41:18.653582+0000","last_active":"2024-03-23T15:42:16.622448+0000","last_peered":"2024-03-23T15:42:16.622448+0000","last_clean":"2024-03-23T15:42:16.622448+0000","last_became_active":"2024-03-23T15:41:18.653184+0000","last_became_peered":"2024-03-23T15:41:18.653184+0000","last_unstale":"2024-03-23T15:42:16.622448+0000","last_undegraded":"2024-03-23T15:42:16.622448+0000","last_fullsized":"2024-03-23T15:42:16.622448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:59:16.726981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841716+0000","last_change":"2024-03-23T15:41:17.651886+0000","last_active":"2024-03-23T15:42:16.841716+0000","last_peered":"2024-03-23T15:42:16.841716+0000","last_clean":"2024-03-23T15:42:16.841716+0000","last_became_active":"2024-03-23T15:41:17.651472+0000","last_became_peered":"2024-03-23T15:41:17.651472+0000","last_unstale":"2024-03-23T15:42:16.841716+0000","last_undegraded":"2024-03-23T15:42:16.841716+0000","last_fullsized":"2024-03-23T15:42:16.841716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:37.098333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625275+0000","last_change":"2024-03-23T15:41:18.644205+0000","last_active":"2024-03-23T15:42:16.625275+0000","last_peered":"2024-03-23T15:42:16.625275+0000","last_clean":"2024-03-23T15:42:16.625275+0000","last_became_active":"2024-03-23T15:41:18.643850+0000","last_became_peered":"2024-03-23T15:41:18.643850+0000","last_unstale":"2024-03-23T15:42:16.625275+0000","last_undegraded":"2024-03-23T15:42:16.625275+0000","last_fullsized":"2024-03-23T15:42:16.625275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:48:10.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620920+0000","last_change":"2024-03-23T15:41:17.661051+0000","last_active":"2024-03-23T15:42:16.620920+0000","last_peered":"2024-03-23T15:42:16.620920+0000","last_clean":"2024-03-23T15:42:16.620920+0000","last_became_active":"2024-03-23T15:41:17.659673+0000","last_became_peered":"2024-03-23T15:41:17.659673+0000","last_unstale":"2024-03-23T15:42:16.620920+0000","last_undegraded":"2024-03-23T15:42:16.620920+0000","last_fullsized":"2024-03-23T15:42:16.620920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.028104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843248+0000","last_change":"2024-03-23T15:42:16.843248+0000","last_active":"2024-03-23T15:42:16.843248+0000","last_peered":"2024-03-23T15:42:16.843248+0000","last_clean":"2024-03-23T15:42:16.843248+0000","last_became_active":"2024-03-23T15:42:16.842800+0000","last_became_peered":"2024-03-23T15:42:16.842800+0000","last_unstale":"2024-03-23T15:42:16.843248+0000","last_undegraded":"2024-03-23T15:42:16.843248+0000","last_fullsized":"2024-03-23T15:42:16.843248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623438+0000","last_change":"2024-03-23T15:41:18.653560+0000","last_active":"2024-03-23T15:42:16.623438+0000","last_peered":"2024-03-23T15:42:16.623438+0000","last_clean":"2024-03-23T15:42:16.623438+0000","last_became_active":"2024-03-23T15:41:18.653249+0000","last_became_peered":"2024-03-23T15:41:18.653249+0000","last_unstale":"2024-03-23T15:42:16.623438+0000","last_undegraded":"2024-03-23T15:42:16.623438+0000","last_fullsized":"2024-03-23T15:42:16.623438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:22.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:33.778347+0000","last_change":"2024-03-23T15:41:17.656318+0000","last_active":"2024-03-23T15:42:33.778347+0000","last_peered":"2024-03-23T15:42:33.778347+0000","last_clean":"2024-03-23T15:42:33.778347+0000","last_became_active":"2024-03-23T15:41:17.655958+0000","last_became_peered":"2024-03-23T15:41:17.655958+0000","last_unstale":"2024-03-23T15:42:33.778347+0000","last_undegraded":"2024-03-23T15:42:33.778347+0000","last_fullsized":"2024-03-23T15:42:33.778347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:12.333504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628493+0000","last_change":"2024-03-23T15:41:18.657616+0000","last_active":"2024-03-23T15:42:16.628493+0000","last_peered":"2024-03-23T15:42:16.628493+0000","last_clean":"2024-03-23T15:42:16.628493+0000","last_became_active":"2024-03-23T15:41:18.657271+0000","last_became_peered":"2024-03-23T15:41:18.657271+0000","last_unstale":"2024-03-23T15:42:16.628493+0000","last_undegraded":"2024-03-23T15:42:16.628493+0000","last_fullsized":"2024-03-23T15:42:16.628493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:02.241087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689107+0000","last_change":"2024-03-23T15:41:17.654358+0000","last_active":"2024-03-23T15:42:16.689107+0000","last_peered":"2024-03-23T15:42:16.689107+0000","last_clean":"2024-03-23T15:42:16.689107+0000","last_became_active":"2024-03-23T15:41:17.653558+0000","last_became_peered":"2024-03-23T15:41:17.653558+0000","last_unstale":"2024-03-23T15:42:16.689107+0000","last_undegraded":"2024-03-23T15:42:16.689107+0000","last_fullsized":"2024-03-23T15:42:16.689107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:32.377883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620261+0000","last_change":"2024-03-23T15:40:05.871801+0000","last_active":"2024-03-23T15:42:16.620261+0000","last_peered":"2024-03-23T15:42:16.620261+0000","last_clean":"2024-03-23T15:42:16.620261+0000","last_became_active":"2024-03-23T15:40:05.871417+0000","last_became_peered":"2024-03-23T15:40:05.871417+0000","last_unstale":"2024-03-23T15:42:16.620261+0000","last_undegraded":"2024-03-23T15:42:16.620261+0000","last_fullsized":"2024-03-23T15:42:16.620261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_clean_scrub_stamp":"2024-03-23T15:36:39.745753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:55.235176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625401+0000","last_change":"2024-03-23T15:41:18.652251+0000","last_active":"2024-03-23T15:42:16.625401+0000","last_peered":"2024-03-23T15:42:16.625401+0000","last_clean":"2024-03-23T15:42:16.625401+0000","last_became_active":"2024-03-23T15:41:18.651876+0000","last_became_peered":"2024-03-23T15:41:18.651876+0000","last_unstale":"2024-03-23T15:42:16.625401+0000","last_undegraded":"2024-03-23T15:42:16.625401+0000","last_fullsized":"2024-03-23T15:42:16.625401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:45.235243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.692783+0000","last_change":"2024-03-23T15:42:16.692783+0000","last_active":"2024-03-23T15:42:16.692783+0000","last_peered":"2024-03-23T15:42:16.692783+0000","last_clean":"2024-03-23T15:42:16.692783+0000","last_became_active":"2024-03-23T15:42:16.690169+0000","last_became_peered":"2024-03-23T15:42:16.690169+0000","last_unstale":"2024-03-23T15:42:16.692783+0000","last_undegraded":"2024-03-23T15:42:16.692783+0000","last_fullsized":"2024-03-23T15:42:16.692783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:20.679040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620537+0000","last_change":"2024-03-23T15:41:17.660703+0000","last_active":"2024-03-23T15:42:16.620537+0000","last_peered":"2024-03-23T15:42:16.620537+0000","last_clean":"2024-03-23T15:42:16.620537+0000","last_became_active":"2024-03-23T15:41:17.660343+0000","last_became_peered":"2024-03-23T15:41:17.660343+0000","last_unstale":"2024-03-23T15:42:16.620537+0000","last_undegraded":"2024-03-23T15:42:16.620537+0000","last_fullsized":"2024-03-23T15:42:16.620537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:14:03.063397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625621+0000","last_change":"2024-03-23T15:41:18.644624+0000","last_active":"2024-03-23T15:42:16.625621+0000","last_peered":"2024-03-23T15:42:16.625621+0000","last_clean":"2024-03-23T15:42:16.625621+0000","last_became_active":"2024-03-23T15:41:18.644275+0000","last_became_peered":"2024-03-23T15:41:18.644275+0000","last_unstale":"2024-03-23T15:42:16.625621+0000","last_undegraded":"2024-03-23T15:42:16.625621+0000","last_fullsized":"2024-03-23T15:42:16.625621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:25.901935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841905+0000","last_change":"2024-03-23T15:41:17.650068+0000","last_active":"2024-03-23T15:42:16.841905+0000","last_peered":"2024-03-23T15:42:16.841905+0000","last_clean":"2024-03-23T15:42:16.841905+0000","last_became_active":"2024-03-23T15:41:17.649331+0000","last_became_peered":"2024-03-23T15:41:17.649331+0000","last_unstale":"2024-03-23T15:42:16.841905+0000","last_undegraded":"2024-03-23T15:42:16.841905+0000","last_fullsized":"2024-03-23T15:42:16.841905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:47.578645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628165+0000","last_change":"2024-03-23T15:41:18.656413+0000","last_active":"2024-03-23T15:42:16.628165+0000","last_peered":"2024-03-23T15:42:16.628165+0000","last_clean":"2024-03-23T15:42:16.628165+0000","last_became_active":"2024-03-23T15:41:18.656043+0000","last_became_peered":"2024-03-23T15:41:18.656043+0000","last_unstale":"2024-03-23T15:42:16.628165+0000","last_undegraded":"2024-03-23T15:42:16.628165+0000","last_fullsized":"2024-03-23T15:42:16.628165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:49:25.453420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689057+0000","last_change":"2024-03-23T15:41:17.654492+0000","last_active":"2024-03-23T15:42:16.689057+0000","last_peered":"2024-03-23T15:42:16.689057+0000","last_clean":"2024-03-23T15:42:16.689057+0000","last_became_active":"2024-03-23T15:41:17.654132+0000","last_became_peered":"2024-03-23T15:41:17.654132+0000","last_unstale":"2024-03-23T15:42:16.689057+0000","last_undegraded":"2024-03-23T15:42:16.689057+0000","last_fullsized":"2024-03-23T15:42:16.689057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:39:58.210703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622330+0000","last_change":"2024-03-23T15:41:18.652659+0000","last_active":"2024-03-23T15:42:16.622330+0000","last_peered":"2024-03-23T15:42:16.622330+0000","last_clean":"2024-03-23T15:42:16.622330+0000","last_became_active":"2024-03-23T15:41:18.652246+0000","last_became_peered":"2024-03-23T15:41:18.652246+0000","last_unstale":"2024-03-23T15:42:16.622330+0000","last_undegraded":"2024-03-23T15:42:16.622330+0000","last_fullsized":"2024-03-23T15:42:16.622330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:50.868694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.632166+0000","last_change":"2024-03-23T15:42:16.632166+0000","last_active":"2024-03-23T15:42:16.632166+0000","last_peered":"2024-03-23T15:42:16.632166+0000","last_clean":"2024-03-23T15:42:16.632166+0000","last_became_active":"2024-03-23T15:42:16.626587+0000","last_became_peered":"2024-03-23T15:42:16.626587+0000","last_unstale":"2024-03-23T15:42:16.632166+0000","last_undegraded":"2024-03-23T15:42:16.632166+0000","last_fullsized":"2024-03-23T15:42:16.632166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.514597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620060+0000","last_change":"2024-03-23T15:41:18.654049+0000","last_active":"2024-03-23T15:42:16.620060+0000","last_peered":"2024-03-23T15:42:16.620060+0000","last_clean":"2024-03-23T15:42:16.620060+0000","last_became_active":"2024-03-23T15:41:18.653765+0000","last_became_peered":"2024-03-23T15:41:18.653765+0000","last_unstale":"2024-03-23T15:42:16.620060+0000","last_undegraded":"2024-03-23T15:42:16.620060+0000","last_fullsized":"2024-03-23T15:42:16.620060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:12:11.467744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627573+0000","last_change":"2024-03-23T15:41:17.656679+0000","last_active":"2024-03-23T15:42:16.627573+0000","last_peered":"2024-03-23T15:42:16.627573+0000","last_clean":"2024-03-23T15:42:16.627573+0000","last_became_active":"2024-03-23T15:41:17.656368+0000","last_became_peered":"2024-03-23T15:41:17.656368+0000","last_unstale":"2024-03-23T15:42:16.627573+0000","last_undegraded":"2024-03-23T15:42:16.627573+0000","last_fullsized":"2024-03-23T15:42:16.627573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:14.176288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622132+0000","last_change":"2024-03-23T15:41:18.653637+0000","last_active":"2024-03-23T15:42:16.622132+0000","last_peered":"2024-03-23T15:42:16.622132+0000","last_clean":"2024-03-23T15:42:16.622132+0000","last_became_active":"2024-03-23T15:41:18.652864+0000","last_became_peered":"2024-03-23T15:41:18.652864+0000","last_unstale":"2024-03-23T15:42:16.622132+0000","last_undegraded":"2024-03-23T15:42:16.622132+0000","last_fullsized":"2024-03-23T15:42:16.622132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:32.104901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626421+0000","last_change":"2024-03-23T15:41:17.664417+0000","last_active":"2024-03-23T15:42:16.626421+0000","last_peered":"2024-03-23T15:42:16.626421+0000","last_clean":"2024-03-23T15:42:16.626421+0000","last_became_active":"2024-03-23T15:41:17.663454+0000","last_became_peered":"2024-03-23T15:41:17.663454+0000","last_unstale":"2024-03-23T15:42:16.626421+0000","last_undegraded":"2024-03-23T15:42:16.626421+0000","last_fullsized":"2024-03-23T15:42:16.626421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:14.947767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627594+0000","last_change":"2024-03-23T15:41:18.647857+0000","last_active":"2024-03-23T15:42:16.627594+0000","last_peered":"2024-03-23T15:42:16.627594+0000","last_clean":"2024-03-23T15:42:16.627594+0000","last_became_active":"2024-03-23T15:41:18.646924+0000","last_became_peered":"2024-03-23T15:41:18.646924+0000","last_unstale":"2024-03-23T15:42:16.627594+0000","last_undegraded":"2024-03-23T15:42:16.627594+0000","last_fullsized":"2024-03-23T15:42:16.627594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:19.494901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620229+0000","last_change":"2024-03-23T15:41:17.661995+0000","last_active":"2024-03-23T15:42:16.620229+0000","last_peered":"2024-03-23T15:42:16.620229+0000","last_clean":"2024-03-23T15:42:16.620229+0000","last_became_active":"2024-03-23T15:41:17.661632+0000","last_became_peered":"2024-03-23T15:41:17.661632+0000","last_unstale":"2024-03-23T15:42:16.620229+0000","last_undegraded":"2024-03-23T15:42:16.620229+0000","last_fullsized":"2024-03-23T15:42:16.620229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:12.540038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620831+0000","last_change":"2024-03-23T15:41:18.651258+0000","last_active":"2024-03-23T15:42:16.620831+0000","last_peered":"2024-03-23T15:42:16.620831+0000","last_clean":"2024-03-23T15:42:16.620831+0000","last_became_active":"2024-03-23T15:41:18.650910+0000","last_became_peered":"2024-03-23T15:41:18.650910+0000","last_unstale":"2024-03-23T15:42:16.620831+0000","last_undegraded":"2024-03-23T15:42:16.620831+0000","last_fullsized":"2024-03-23T15:42:16.620831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:06:50.139644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626748+0000","last_change":"2024-03-23T15:41:17.654232+0000","last_active":"2024-03-23T15:42:16.626748+0000","last_peered":"2024-03-23T15:42:16.626748+0000","last_clean":"2024-03-23T15:42:16.626748+0000","last_became_active":"2024-03-23T15:41:17.653832+0000","last_became_peered":"2024-03-23T15:41:17.653832+0000","last_unstale":"2024-03-23T15:42:16.626748+0000","last_undegraded":"2024-03-23T15:42:16.626748+0000","last_fullsized":"2024-03-23T15:42:16.626748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:05.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620932+0000","last_change":"2024-03-23T15:41:18.664444+0000","last_active":"2024-03-23T15:42:16.620932+0000","last_peered":"2024-03-23T15:42:16.620932+0000","last_clean":"2024-03-23T15:42:16.620932+0000","last_became_active":"2024-03-23T15:41:18.664127+0000","last_became_peered":"2024-03-23T15:41:18.664127+0000","last_unstale":"2024-03-23T15:42:16.620932+0000","last_undegraded":"2024-03-23T15:42:16.620932+0000","last_fullsized":"2024-03-23T15:42:16.620932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:25.942230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623064+0000","last_change":"2024-03-23T15:41:17.664834+0000","last_active":"2024-03-23T15:42:16.623064+0000","last_peered":"2024-03-23T15:42:16.623064+0000","last_clean":"2024-03-23T15:42:16.623064+0000","last_became_active":"2024-03-23T15:41:17.664286+0000","last_became_peered":"2024-03-23T15:41:17.664286+0000","last_unstale":"2024-03-23T15:42:16.623064+0000","last_undegraded":"2024-03-23T15:42:16.623064+0000","last_fullsized":"2024-03-23T15:42:16.623064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:50.595420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620354+0000","last_change":"2024-03-23T15:41:18.646341+0000","last_active":"2024-03-23T15:42:16.620354+0000","last_peered":"2024-03-23T15:42:16.620354+0000","last_clean":"2024-03-23T15:42:16.620354+0000","last_became_active":"2024-03-23T15:41:18.645995+0000","last_became_peered":"2024-03-23T15:41:18.645995+0000","last_unstale":"2024-03-23T15:42:16.620354+0000","last_undegraded":"2024-03-23T15:42:16.620354+0000","last_fullsized":"2024-03-23T15:42:16.620354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:56.408957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.694643+0000","last_change":"2024-03-23T15:42:16.694643+0000","last_active":"2024-03-23T15:42:16.694643+0000","last_peered":"2024-03-23T15:42:16.694643+0000","last_clean":"2024-03-23T15:42:16.694643+0000","last_became_active":"2024-03-23T15:42:16.690063+0000","last_became_peered":"2024-03-23T15:42:16.690063+0000","last_unstale":"2024-03-23T15:42:16.694643+0000","last_undegraded":"2024-03-23T15:42:16.694643+0000","last_fullsized":"2024-03-23T15:42:16.694643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:54.873573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620756+0000","last_change":"2024-03-23T15:41:18.655250+0000","last_active":"2024-03-23T15:42:16.620756+0000","last_peered":"2024-03-23T15:42:16.620756+0000","last_clean":"2024-03-23T15:42:16.620756+0000","last_became_active":"2024-03-23T15:41:18.654888+0000","last_became_peered":"2024-03-23T15:41:18.654888+0000","last_unstale":"2024-03-23T15:42:16.620756+0000","last_undegraded":"2024-03-23T15:42:16.620756+0000","last_fullsized":"2024-03-23T15:42:16.620756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:36.730600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622152+0000","last_change":"2024-03-23T15:41:18.654309+0000","last_active":"2024-03-23T15:42:16.622152+0000","last_peered":"2024-03-23T15:42:16.622152+0000","last_clean":"2024-03-23T15:42:16.622152+0000","last_became_active":"2024-03-23T15:41:18.653319+0000","last_became_peered":"2024-03-23T15:41:18.653319+0000","last_unstale":"2024-03-23T15:42:16.622152+0000","last_undegraded":"2024-03-23T15:42:16.622152+0000","last_fullsized":"2024-03-23T15:42:16.622152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:02:05.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626359+0000","last_change":"2024-03-23T15:41:18.655179+0000","last_active":"2024-03-23T15:42:16.626359+0000","last_peered":"2024-03-23T15:42:16.626359+0000","last_clean":"2024-03-23T15:42:16.626359+0000","last_became_active":"2024-03-23T15:41:18.654830+0000","last_became_peered":"2024-03-23T15:41:18.654830+0000","last_unstale":"2024-03-23T15:42:16.626359+0000","last_undegraded":"2024-03-23T15:42:16.626359+0000","last_fullsized":"2024-03-23T15:42:16.626359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:29.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626300+0000","last_change":"2024-03-23T15:41:18.650734+0000","last_active":"2024-03-23T15:42:16.626300+0000","last_peered":"2024-03-23T15:42:16.626300+0000","last_clean":"2024-03-23T15:42:16.626300+0000","last_became_active":"2024-03-23T15:41:18.649993+0000","last_became_peered":"2024-03-23T15:41:18.649993+0000","last_unstale":"2024-03-23T15:42:16.626300+0000","last_undegraded":"2024-03-23T15:42:16.626300+0000","last_fullsized":"2024-03-23T15:42:16.626300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:06.463525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622098+0000","last_change":"2024-03-23T15:41:18.655741+0000","last_active":"2024-03-23T15:42:16.622098+0000","last_peered":"2024-03-23T15:42:16.622098+0000","last_clean":"2024-03-23T15:42:16.622098+0000","last_became_active":"2024-03-23T15:41:18.653387+0000","last_became_peered":"2024-03-23T15:41:18.653387+0000","last_unstale":"2024-03-23T15:42:16.622098+0000","last_undegraded":"2024-03-23T15:42:16.622098+0000","last_fullsized":"2024-03-23T15:42:16.622098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:20:26.758549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625725+0000","last_change":"2024-03-23T15:41:18.654739+0000","last_active":"2024-03-23T15:42:16.625725+0000","last_peered":"2024-03-23T15:42:16.625725+0000","last_clean":"2024-03-23T15:42:16.625725+0000","last_became_active":"2024-03-23T15:41:18.654151+0000","last_became_peered":"2024-03-23T15:41:18.654151+0000","last_unstale":"2024-03-23T15:42:16.625725+0000","last_undegraded":"2024-03-23T15:42:16.625725+0000","last_fullsized":"2024-03-23T15:42:16.625725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:01.358959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622422+0000","last_change":"2024-03-23T15:41:18.654081+0000","last_active":"2024-03-23T15:42:16.622422+0000","last_peered":"2024-03-23T15:42:16.622422+0000","last_clean":"2024-03-23T15:42:16.622422+0000","last_became_active":"2024-03-23T15:41:18.653696+0000","last_became_peered":"2024-03-23T15:41:18.653696+0000","last_unstale":"2024-03-23T15:42:16.622422+0000","last_undegraded":"2024-03-23T15:42:16.622422+0000","last_fullsized":"2024-03-23T15:42:16.622422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:23:19.741645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627099+0000","last_change":"2024-03-23T15:41:18.652356+0000","last_active":"2024-03-23T15:42:16.627099+0000","last_peered":"2024-03-23T15:42:16.627099+0000","last_clean":"2024-03-23T15:42:16.627099+0000","last_became_active":"2024-03-23T15:41:18.651976+0000","last_became_peered":"2024-03-23T15:41:18.651976+0000","last_unstale":"2024-03-23T15:42:16.627099+0000","last_undegraded":"2024-03-23T15:42:16.627099+0000","last_fullsized":"2024-03-23T15:42:16.627099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:38.972186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628124+0000","last_change":"2024-03-23T15:41:18.653775+0000","last_active":"2024-03-23T15:42:16.628124+0000","last_peered":"2024-03-23T15:42:16.628124+0000","last_clean":"2024-03-23T15:42:16.628124+0000","last_became_active":"2024-03-23T15:41:18.653404+0000","last_became_peered":"2024-03-23T15:41:18.653404+0000","last_unstale":"2024-03-23T15:42:16.628124+0000","last_undegraded":"2024-03-23T15:42:16.628124+0000","last_fullsized":"2024-03-23T15:42:16.628124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:28:57.534731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843115+0000","last_change":"2024-03-23T15:42:16.843115+0000","last_active":"2024-03-23T15:42:16.843115+0000","last_peered":"2024-03-23T15:42:16.843115+0000","last_clean":"2024-03-23T15:42:16.843115+0000","last_became_active":"2024-03-23T15:42:16.842743+0000","last_became_peered":"2024-03-23T15:42:16.842743+0000","last_unstale":"2024-03-23T15:42:16.843115+0000","last_undegraded":"2024-03-23T15:42:16.843115+0000","last_fullsized":"2024-03-23T15:42:16.843115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:36:11.377398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627449+0000","last_change":"2024-03-23T15:41:18.652487+0000","last_active":"2024-03-23T15:42:16.627449+0000","last_peered":"2024-03-23T15:42:16.627449+0000","last_clean":"2024-03-23T15:42:16.627449+0000","last_became_active":"2024-03-23T15:41:18.652151+0000","last_became_peered":"2024-03-23T15:41:18.652151+0000","last_unstale":"2024-03-23T15:42:16.627449+0000","last_undegraded":"2024-03-23T15:42:16.627449+0000","last_fullsized":"2024-03-23T15:42:16.627449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:39.133585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622943+0000","last_change":"2024-03-23T15:41:18.657265+0000","last_active":"2024-03-23T15:42:16.622943+0000","last_peered":"2024-03-23T15:42:16.622943+0000","last_clean":"2024-03-23T15:42:16.622943+0000","last_became_active":"2024-03-23T15:41:18.656970+0000","last_became_peered":"2024-03-23T15:41:18.656970+0000","last_unstale":"2024-03-23T15:42:16.622943+0000","last_undegraded":"2024-03-23T15:42:16.622943+0000","last_fullsized":"2024-03-23T15:42:16.622943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:22.808034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625658+0000","last_change":"2024-03-23T15:41:18.655254+0000","last_active":"2024-03-23T15:42:16.625658+0000","last_peered":"2024-03-23T15:42:16.625658+0000","last_clean":"2024-03-23T15:42:16.625658+0000","last_became_active":"2024-03-23T15:41:18.654879+0000","last_became_peered":"2024-03-23T15:41:18.654879+0000","last_unstale":"2024-03-23T15:42:16.625658+0000","last_undegraded":"2024-03-23T15:42:16.625658+0000","last_fullsized":"2024-03-23T15:42:16.625658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:36:18.923723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622784+0000","last_change":"2024-03-23T15:41:18.658652+0000","last_active":"2024-03-23T15:42:16.622784+0000","last_peered":"2024-03-23T15:42:16.622784+0000","last_clean":"2024-03-23T15:42:16.622784+0000","last_became_active":"2024-03-23T15:41:18.658261+0000","last_became_peered":"2024-03-23T15:41:18.658261+0000","last_unstale":"2024-03-23T15:42:16.622784+0000","last_undegraded":"2024-03-23T15:42:16.622784+0000","last_fullsized":"2024-03-23T15:42:16.622784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:45.145101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245950,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":457,"num_read_kb":297,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":195141,"data_compressed":5607,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":456395,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":1108922,"data_compressed":43282,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456631,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456551,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1048481,"data_compressed":41413,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":456020,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":516859,"data_compressed":7678,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457291,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":518600,"data_compressed":7678,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":456331,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":456457,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":1048165,"data_compressed":41413,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62787,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62624,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T15:42:36.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T15:42:36.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:36 smithi183 ceph-mon[36296]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:37.053 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:37.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:36 smithi192 ceph-mon[36653]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:37 smithi076 ceph-mon[32176]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:42:37.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:37 smithi183 ceph-mon[36296]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:42:38.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:37 smithi192 ceph-mon[36653]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:42:38.810 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:38 smithi076 ceph-mon[32176]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:38.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.804+0000 7f766c672700 1 -- 172.21.15.76:0/2007941034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 msgr2=0x7f766406ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.804+0000 7f766c672700 1 --2- 172.21.15.76:0/2007941034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f766406ae00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7660009990 tx=0x7f7660009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.804+0000 7f766c672700 1 -- 172.21.15.76:0/2007941034 shutdown_connections 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.804+0000 7f766c672700 1 --2- 172.21.15.76:0/2007941034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f766406b340 0x7f766410f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.804+0000 7f766c672700 1 --2- 172.21.15.76:0/2007941034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7664107210 0x7f76641075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.804+0000 7f766c672700 1 --2- 172.21.15.76:0/2007941034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f766406ae00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.804+0000 7f766c672700 1 -- 172.21.15.76:0/2007941034 >> 172.21.15.76:0/2007941034 conn(0x7f7664078160 msgr2=0x7f7664078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 -- 172.21.15.76:0/2007941034 shutdown_connections 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 -- 172.21.15.76:0/2007941034 wait complete. 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 Processor -- start 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 -- start start 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f76641011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f766406b340 0x7f76641016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:38.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7664107210 0x7f7664101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f76641a6be0 con 0x7f766406a980 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76641a6d50 con 0x7f766406b340 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766c672700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76641a6f10 con 0x7f7664107210 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766a40e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f76641011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766a40e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f76641011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43904/0 (socket says 172.21.15.76:43904) 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f766a40e700 1 -- 172.21.15.76:0/2504334362 learned_addr learned my addr 172.21.15.76:0/2504334362 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.805+0000 7f7669c0d700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f766406b340 0x7f76641016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766ac0f700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7664107210 0x7f7664101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766a40e700 1 -- 172.21.15.76:0/2504334362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7664107210 msgr2=0x7f7664101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766a40e700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7664107210 0x7f7664101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766a40e700 1 -- 172.21.15.76:0/2504334362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f766406b340 msgr2=0x7f76641016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766a40e700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f766406b340 0x7f76641016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:38.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766a40e700 1 -- 172.21.15.76:0/2504334362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76600095f0 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766a40e700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f76641011b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f765400d980 tx=0x7f765400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f765b7fe700 1 -- 172.21.15.76:0/2504334362 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654004220 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f765b7fe700 1 -- 172.21.15.76:0/2504334362 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7654007640 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76641a71f0 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f765b7fe700 1 -- 172.21.15.76:0/2504334362 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765400f690 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.806+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76641a76c0 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.807+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f766410a770 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.808+0000 7f765b7fe700 1 -- 172.21.15.76:0/2504334362 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f7654004410 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.809+0000 7f765b7fe700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76500a21e0 0x7f76500a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.809+0000 7f7669c0d700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76500a21e0 0x7f76500a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.809+0000 7f765b7fe700 1 -- 172.21.15.76:0/2504334362 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f76540cdb00 con 0x7f766406a980 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.809+0000 7f7669c0d700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76500a21e0 0x7f76500a46a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76600098e0 tx=0x7f7660005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:38.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:38.813+0000 7f765b7fe700 1 -- 172.21.15.76:0/2504334362 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7654096950 con 0x7f766406a980 2024-03-23T15:42:38.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:38 smithi183 ceph-mon[36296]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.004+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76641026e0 con 0x7f76500a21e0 2024-03-23T15:42:39.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.011+0000 7f765b7fe700 1 -- 172.21.15.76:0/2504334362 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258978 (secure 0 0 0) 0x7f76641026e0 con 0x7f76500a21e0 2024-03-23T15:42:39.017 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:39.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76500a21e0 msgr2=0x7f76500a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76500a21e0 0x7f76500a46a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76600098e0 tx=0x7f7660005960 comp rx=0 tx=0).stop 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 msgr2=0x7f76641011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f76641011b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f765400d980 tx=0x7f765400dc90 comp rx=0 tx=0).stop 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 shutdown_connections 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76500a21e0 0x7f76500a46a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7664107210 0x7f7664101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f766406a980 0x7f76641011b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 --2- 172.21.15.76:0/2504334362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f766406b340 0x7f76641016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 >> 172.21.15.76:0/2504334362 conn(0x7f7664078160 msgr2=0x7f76640ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 shutdown_connections 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:39.014+0000 7f766c672700 1 -- 172.21.15.76:0/2504334362 wait complete. 2024-03-23T15:42:39.019 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-23T15:42:39.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:38 smithi192 ceph-mon[36653]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:42:39.678 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-03-23T15:42:37.495743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":593,"num_read_kb":416,"num_write":339,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369108,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572716,"statfs":{"total":1151940427776,"available":1151562461184,"internally_reserved":0,"allocated":11833344,"data_stored":7436703,"data_compressed":182127,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003194"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622644+0000","last_change":"2024-03-23T15:41:18.651731+0000","last_active":"2024-03-23T15:42:16.622644+0000","last_peered":"2024-03-23T15:42:16.622644+0000","last_clean":"2024-03-23T15:42:16.622644+0000","last_became_active":"2024-03-23T15:41:18.651312+0000","last_became_peered":"2024-03-23T15:41:18.651312+0000","last_unstale":"2024-03-23T15:42:16.622644+0000","last_undegraded":"2024-03-23T15:42:16.622644+0000","last_fullsized":"2024-03-23T15:42:16.622644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:39:20.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622436+0000","last_change":"2024-03-23T15:41:18.655736+0000","last_active":"2024-03-23T15:42:16.622436+0000","last_peered":"2024-03-23T15:42:16.622436+0000","last_clean":"2024-03-23T15:42:16.622436+0000","last_became_active":"2024-03-23T15:41:18.655389+0000","last_became_peered":"2024-03-23T15:41:18.655389+0000","last_unstale":"2024-03-23T15:42:16.622436+0000","last_undegraded":"2024-03-23T15:42:16.622436+0000","last_fullsized":"2024-03-23T15:42:16.622436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:35:56.260056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622442+0000","last_change":"2024-03-23T15:41:18.650709+0000","last_active":"2024-03-23T15:42:16.622442+0000","last_peered":"2024-03-23T15:42:16.622442+0000","last_clean":"2024-03-23T15:42:16.622442+0000","last_became_active":"2024-03-23T15:41:18.650316+0000","last_became_peered":"2024-03-23T15:41:18.650316+0000","last_unstale":"2024-03-23T15:42:16.622442+0000","last_undegraded":"2024-03-23T15:42:16.622442+0000","last_fullsized":"2024-03-23T15:42:16.622442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:50:57.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689194+0000","last_change":"2024-03-23T15:41:18.652216+0000","last_active":"2024-03-23T15:42:16.689194+0000","last_peered":"2024-03-23T15:42:16.689194+0000","last_clean":"2024-03-23T15:42:16.689194+0000","last_became_active":"2024-03-23T15:41:18.650525+0000","last_became_peered":"2024-03-23T15:41:18.650525+0000","last_unstale":"2024-03-23T15:42:16.689194+0000","last_undegraded":"2024-03-23T15:42:16.689194+0000","last_fullsized":"2024-03-23T15:42:16.689194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:16.458257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620128+0000","last_change":"2024-03-23T15:41:18.649146+0000","last_active":"2024-03-23T15:42:16.620128+0000","last_peered":"2024-03-23T15:42:16.620128+0000","last_clean":"2024-03-23T15:42:16.620128+0000","last_became_active":"2024-03-23T15:41:18.648728+0000","last_became_peered":"2024-03-23T15:41:18.648728+0000","last_unstale":"2024-03-23T15:42:16.620128+0000","last_undegraded":"2024-03-23T15:42:16.620128+0000","last_fullsized":"2024-03-23T15:42:16.620128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:56.251250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625127+0000","last_change":"2024-03-23T15:41:18.652957+0000","last_active":"2024-03-23T15:42:16.625127+0000","last_peered":"2024-03-23T15:42:16.625127+0000","last_clean":"2024-03-23T15:42:16.625127+0000","last_became_active":"2024-03-23T15:41:18.652578+0000","last_became_peered":"2024-03-23T15:41:18.652578+0000","last_unstale":"2024-03-23T15:42:16.625127+0000","last_undegraded":"2024-03-23T15:42:16.625127+0000","last_fullsized":"2024-03-23T15:42:16.625127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:46:38.961554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620222+0000","last_change":"2024-03-23T15:41:18.646134+0000","last_active":"2024-03-23T15:42:16.620222+0000","last_peered":"2024-03-23T15:42:16.620222+0000","last_clean":"2024-03-23T15:42:16.620222+0000","last_became_active":"2024-03-23T15:41:18.645680+0000","last_became_peered":"2024-03-23T15:41:18.645680+0000","last_unstale":"2024-03-23T15:42:16.620222+0000","last_undegraded":"2024-03-23T15:42:16.620222+0000","last_fullsized":"2024-03-23T15:42:16.620222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:34:31.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627918+0000","last_change":"2024-03-23T15:41:18.646143+0000","last_active":"2024-03-23T15:42:16.627918+0000","last_peered":"2024-03-23T15:42:16.627918+0000","last_clean":"2024-03-23T15:42:16.627918+0000","last_became_active":"2024-03-23T15:41:18.645771+0000","last_became_peered":"2024-03-23T15:41:18.645771+0000","last_unstale":"2024-03-23T15:42:16.627918+0000","last_undegraded":"2024-03-23T15:42:16.627918+0000","last_fullsized":"2024-03-23T15:42:16.627918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:39.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623687+0000","last_change":"2024-03-23T15:41:18.665083+0000","last_active":"2024-03-23T15:42:16.623687+0000","last_peered":"2024-03-23T15:42:16.623687+0000","last_clean":"2024-03-23T15:42:16.623687+0000","last_became_active":"2024-03-23T15:41:18.663143+0000","last_became_peered":"2024-03-23T15:41:18.663143+0000","last_unstale":"2024-03-23T15:42:16.623687+0000","last_undegraded":"2024-03-23T15:42:16.623687+0000","last_fullsized":"2024-03-23T15:42:16.623687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:52:23.421645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625458+0000","last_change":"2024-03-23T15:41:18.650014+0000","last_active":"2024-03-23T15:42:16.625458+0000","last_peered":"2024-03-23T15:42:16.625458+0000","last_clean":"2024-03-23T15:42:16.625458+0000","last_became_active":"2024-03-23T15:41:18.649673+0000","last_became_peered":"2024-03-23T15:41:18.649673+0000","last_unstale":"2024-03-23T15:42:16.625458+0000","last_undegraded":"2024-03-23T15:42:16.625458+0000","last_fullsized":"2024-03-23T15:42:16.625458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:30.387993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620880+0000","last_change":"2024-03-23T15:41:18.665622+0000","last_active":"2024-03-23T15:42:16.620880+0000","last_peered":"2024-03-23T15:42:16.620880+0000","last_clean":"2024-03-23T15:42:16.620880+0000","last_became_active":"2024-03-23T15:41:18.665274+0000","last_became_peered":"2024-03-23T15:41:18.665274+0000","last_unstale":"2024-03-23T15:42:16.620880+0000","last_undegraded":"2024-03-23T15:42:16.620880+0000","last_fullsized":"2024-03-23T15:42:16.620880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:12:18.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689095+0000","last_change":"2024-03-23T15:41:18.650888+0000","last_active":"2024-03-23T15:42:16.689095+0000","last_peered":"2024-03-23T15:42:16.689095+0000","last_clean":"2024-03-23T15:42:16.689095+0000","last_became_active":"2024-03-23T15:41:18.650363+0000","last_became_peered":"2024-03-23T15:41:18.650363+0000","last_unstale":"2024-03-23T15:42:16.689095+0000","last_undegraded":"2024-03-23T15:42:16.689095+0000","last_fullsized":"2024-03-23T15:42:16.689095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:41.289697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625765+0000","last_change":"2024-03-23T15:41:18.634183+0000","last_active":"2024-03-23T15:42:16.625765+0000","last_peered":"2024-03-23T15:42:16.625765+0000","last_clean":"2024-03-23T15:42:16.625765+0000","last_became_active":"2024-03-23T15:41:18.633849+0000","last_became_peered":"2024-03-23T15:41:18.633849+0000","last_unstale":"2024-03-23T15:42:16.625765+0000","last_undegraded":"2024-03-23T15:42:16.625765+0000","last_fullsized":"2024-03-23T15:42:16.625765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:13.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622197+0000","last_change":"2024-03-23T15:41:18.645269+0000","last_active":"2024-03-23T15:42:16.622197+0000","last_peered":"2024-03-23T15:42:16.622197+0000","last_clean":"2024-03-23T15:42:16.622197+0000","last_became_active":"2024-03-23T15:41:18.644940+0000","last_became_peered":"2024-03-23T15:41:18.644940+0000","last_unstale":"2024-03-23T15:42:16.622197+0000","last_undegraded":"2024-03-23T15:42:16.622197+0000","last_fullsized":"2024-03-23T15:42:16.622197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:26.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622000+0000","last_change":"2024-03-23T15:41:18.652789+0000","last_active":"2024-03-23T15:42:16.622000+0000","last_peered":"2024-03-23T15:42:16.622000+0000","last_clean":"2024-03-23T15:42:16.622000+0000","last_became_active":"2024-03-23T15:41:18.651520+0000","last_became_peered":"2024-03-23T15:41:18.651520+0000","last_unstale":"2024-03-23T15:42:16.622000+0000","last_undegraded":"2024-03-23T15:42:16.622000+0000","last_fullsized":"2024-03-23T15:42:16.622000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:00.388217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622362+0000","last_change":"2024-03-23T15:41:18.660398+0000","last_active":"2024-03-23T15:42:16.622362+0000","last_peered":"2024-03-23T15:42:16.622362+0000","last_clean":"2024-03-23T15:42:16.622362+0000","last_became_active":"2024-03-23T15:41:18.660084+0000","last_became_peered":"2024-03-23T15:41:18.660084+0000","last_unstale":"2024-03-23T15:42:16.622362+0000","last_undegraded":"2024-03-23T15:42:16.622362+0000","last_fullsized":"2024-03-23T15:42:16.622362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:04:33.115173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622743+0000","last_change":"2024-03-23T15:41:18.663192+0000","last_active":"2024-03-23T15:42:16.622743+0000","last_peered":"2024-03-23T15:42:16.622743+0000","last_clean":"2024-03-23T15:42:16.622743+0000","last_became_active":"2024-03-23T15:41:18.662844+0000","last_became_peered":"2024-03-23T15:41:18.662844+0000","last_unstale":"2024-03-23T15:42:16.622743+0000","last_undegraded":"2024-03-23T15:42:16.622743+0000","last_fullsized":"2024-03-23T15:42:16.622743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:24.260706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841865+0000","last_change":"2024-03-23T15:41:18.646405+0000","last_active":"2024-03-23T15:42:16.841865+0000","last_peered":"2024-03-23T15:42:16.841865+0000","last_clean":"2024-03-23T15:42:16.841865+0000","last_became_active":"2024-03-23T15:41:18.645516+0000","last_became_peered":"2024-03-23T15:41:18.645516+0000","last_unstale":"2024-03-23T15:42:16.841865+0000","last_undegraded":"2024-03-23T15:42:16.841865+0000","last_fullsized":"2024-03-23T15:42:16.841865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:29:18.588190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843159+0000","last_change":"2024-03-23T15:42:16.843159+0000","last_active":"2024-03-23T15:42:16.843159+0000","last_peered":"2024-03-23T15:42:16.843159+0000","last_clean":"2024-03-23T15:42:16.843159+0000","last_became_active":"2024-03-23T15:42:16.842729+0000","last_became_peered":"2024-03-23T15:42:16.842729+0000","last_unstale":"2024-03-23T15:42:16.843159+0000","last_undegraded":"2024-03-23T15:42:16.843159+0000","last_fullsized":"2024-03-23T15:42:16.843159+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:57:18.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841913+0000","last_change":"2024-03-23T15:41:17.655443+0000","last_active":"2024-03-23T15:42:16.841913+0000","last_peered":"2024-03-23T15:42:16.841913+0000","last_clean":"2024-03-23T15:42:16.841913+0000","last_became_active":"2024-03-23T15:41:17.654834+0000","last_became_peered":"2024-03-23T15:41:17.654834+0000","last_unstale":"2024-03-23T15:42:16.841913+0000","last_undegraded":"2024-03-23T15:42:16.841913+0000","last_fullsized":"2024-03-23T15:42:16.841913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:49:43.215644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628289+0000","last_change":"2024-03-23T15:41:18.650241+0000","last_active":"2024-03-23T15:42:16.628289+0000","last_peered":"2024-03-23T15:42:16.628289+0000","last_clean":"2024-03-23T15:42:16.628289+0000","last_became_active":"2024-03-23T15:41:18.649914+0000","last_became_peered":"2024-03-23T15:41:18.649914+0000","last_unstale":"2024-03-23T15:42:16.628289+0000","last_undegraded":"2024-03-23T15:42:16.628289+0000","last_fullsized":"2024-03-23T15:42:16.628289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:35:18.892649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628447+0000","last_change":"2024-03-23T15:41:17.663308+0000","last_active":"2024-03-23T15:42:16.628447+0000","last_peered":"2024-03-23T15:42:16.628447+0000","last_clean":"2024-03-23T15:42:16.628447+0000","last_became_active":"2024-03-23T15:41:17.662958+0000","last_became_peered":"2024-03-23T15:41:17.662958+0000","last_unstale":"2024-03-23T15:42:16.628447+0000","last_undegraded":"2024-03-23T15:42:16.628447+0000","last_fullsized":"2024-03-23T15:42:16.628447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:43:37.512893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622314+0000","last_change":"2024-03-23T15:41:18.651225+0000","last_active":"2024-03-23T15:42:16.622314+0000","last_peered":"2024-03-23T15:42:16.622314+0000","last_clean":"2024-03-23T15:42:16.622314+0000","last_became_active":"2024-03-23T15:41:18.650884+0000","last_became_peered":"2024-03-23T15:41:18.650884+0000","last_unstale":"2024-03-23T15:42:16.622314+0000","last_undegraded":"2024-03-23T15:42:16.622314+0000","last_fullsized":"2024-03-23T15:42:16.622314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:00.445203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620673+0000","last_change":"2024-03-23T15:41:17.662753+0000","last_active":"2024-03-23T15:42:16.620673+0000","last_peered":"2024-03-23T15:42:16.620673+0000","last_clean":"2024-03-23T15:42:16.620673+0000","last_became_active":"2024-03-23T15:41:17.662378+0000","last_became_peered":"2024-03-23T15:41:17.662378+0000","last_unstale":"2024-03-23T15:42:16.620673+0000","last_undegraded":"2024-03-23T15:42:16.620673+0000","last_fullsized":"2024-03-23T15:42:16.620673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:58:55.674739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622672+0000","last_change":"2024-03-23T15:41:18.659171+0000","last_active":"2024-03-23T15:42:16.622672+0000","last_peered":"2024-03-23T15:42:16.622672+0000","last_clean":"2024-03-23T15:42:16.622672+0000","last_became_active":"2024-03-23T15:41:18.658836+0000","last_became_peered":"2024-03-23T15:41:18.658836+0000","last_unstale":"2024-03-23T15:42:16.622672+0000","last_undegraded":"2024-03-23T15:42:16.622672+0000","last_fullsized":"2024-03-23T15:42:16.622672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:15.056528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.632016+0000","last_change":"2024-03-23T15:42:16.631598+0000","last_active":"2024-03-23T15:42:16.632016+0000","last_peered":"2024-03-23T15:42:16.632016+0000","last_clean":"2024-03-23T15:42:16.632016+0000","last_became_active":"2024-03-23T15:42:16.626457+0000","last_became_peered":"2024-03-23T15:42:16.626457+0000","last_unstale":"2024-03-23T15:42:16.632016+0000","last_undegraded":"2024-03-23T15:42:16.632016+0000","last_fullsized":"2024-03-23T15:42:16.632016+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:29.776204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623174+0000","last_change":"2024-03-23T15:41:18.648500+0000","last_active":"2024-03-23T15:42:16.623174+0000","last_peered":"2024-03-23T15:42:16.623174+0000","last_clean":"2024-03-23T15:42:16.623174+0000","last_became_active":"2024-03-23T15:41:18.648139+0000","last_became_peered":"2024-03-23T15:41:18.648139+0000","last_unstale":"2024-03-23T15:42:16.623174+0000","last_undegraded":"2024-03-23T15:42:16.623174+0000","last_fullsized":"2024-03-23T15:42:16.623174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:38.071057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:32.776274+0000","last_change":"2024-03-23T15:41:17.658471+0000","last_active":"2024-03-23T15:42:32.776274+0000","last_peered":"2024-03-23T15:42:32.776274+0000","last_clean":"2024-03-23T15:42:32.776274+0000","last_became_active":"2024-03-23T15:41:17.658142+0000","last_became_peered":"2024-03-23T15:41:17.658142+0000","last_unstale":"2024-03-23T15:42:32.776274+0000","last_undegraded":"2024-03-23T15:42:32.776274+0000","last_fullsized":"2024-03-23T15:42:32.776274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:23.224790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620645+0000","last_change":"2024-03-23T15:41:18.646046+0000","last_active":"2024-03-23T15:42:16.620645+0000","last_peered":"2024-03-23T15:42:16.620645+0000","last_clean":"2024-03-23T15:42:16.620645+0000","last_became_active":"2024-03-23T15:41:18.645727+0000","last_became_peered":"2024-03-23T15:41:18.645727+0000","last_unstale":"2024-03-23T15:42:16.620645+0000","last_undegraded":"2024-03-23T15:42:16.620645+0000","last_fullsized":"2024-03-23T15:42:16.620645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:55.552079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625081+0000","last_change":"2024-03-23T15:41:17.657285+0000","last_active":"2024-03-23T15:42:16.625081+0000","last_peered":"2024-03-23T15:42:16.625081+0000","last_clean":"2024-03-23T15:42:16.625081+0000","last_became_active":"2024-03-23T15:41:17.656942+0000","last_became_peered":"2024-03-23T15:41:17.656942+0000","last_unstale":"2024-03-23T15:42:16.625081+0000","last_undegraded":"2024-03-23T15:42:16.625081+0000","last_fullsized":"2024-03-23T15:42:16.625081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:18.107467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623937+0000","last_change":"2024-03-23T15:41:18.655175+0000","last_active":"2024-03-23T15:42:16.623937+0000","last_peered":"2024-03-23T15:42:16.623937+0000","last_clean":"2024-03-23T15:42:16.623937+0000","last_became_active":"2024-03-23T15:41:18.654846+0000","last_became_peered":"2024-03-23T15:41:18.654846+0000","last_unstale":"2024-03-23T15:42:16.623937+0000","last_undegraded":"2024-03-23T15:42:16.623937+0000","last_fullsized":"2024-03-23T15:42:16.623937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:43:10.008538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625643+0000","last_change":"2024-03-23T15:41:17.657391+0000","last_active":"2024-03-23T15:42:16.625643+0000","last_peered":"2024-03-23T15:42:16.625643+0000","last_clean":"2024-03-23T15:42:16.625643+0000","last_became_active":"2024-03-23T15:41:17.657081+0000","last_became_peered":"2024-03-23T15:41:17.657081+0000","last_unstale":"2024-03-23T15:42:16.625643+0000","last_undegraded":"2024-03-23T15:42:16.625643+0000","last_fullsized":"2024-03-23T15:42:16.625643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:08.744477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628032+0000","last_change":"2024-03-23T15:41:18.656268+0000","last_active":"2024-03-23T15:42:16.628032+0000","last_peered":"2024-03-23T15:42:16.628032+0000","last_clean":"2024-03-23T15:42:16.628032+0000","last_became_active":"2024-03-23T15:41:18.655910+0000","last_became_peered":"2024-03-23T15:41:18.655910+0000","last_unstale":"2024-03-23T15:42:16.628032+0000","last_undegraded":"2024-03-23T15:42:16.628032+0000","last_fullsized":"2024-03-23T15:42:16.628032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:38:57.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623201+0000","last_change":"2024-03-23T15:41:17.656555+0000","last_active":"2024-03-23T15:42:16.623201+0000","last_peered":"2024-03-23T15:42:16.623201+0000","last_clean":"2024-03-23T15:42:16.623201+0000","last_became_active":"2024-03-23T15:41:17.655732+0000","last_became_peered":"2024-03-23T15:41:17.655732+0000","last_unstale":"2024-03-23T15:42:16.623201+0000","last_undegraded":"2024-03-23T15:42:16.623201+0000","last_fullsized":"2024-03-23T15:42:16.623201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:53.839841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841655+0000","last_change":"2024-03-23T15:41:18.647334+0000","last_active":"2024-03-23T15:42:16.841655+0000","last_peered":"2024-03-23T15:42:16.841655+0000","last_clean":"2024-03-23T15:42:16.841655+0000","last_became_active":"2024-03-23T15:41:18.646679+0000","last_became_peered":"2024-03-23T15:41:18.646679+0000","last_unstale":"2024-03-23T15:42:16.841655+0000","last_undegraded":"2024-03-23T15:42:16.841655+0000","last_fullsized":"2024-03-23T15:42:16.841655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.710759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:31.774809+0000","last_change":"2024-03-23T15:41:17.655400+0000","last_active":"2024-03-23T15:42:31.774809+0000","last_peered":"2024-03-23T15:42:31.774809+0000","last_clean":"2024-03-23T15:42:31.774809+0000","last_became_active":"2024-03-23T15:41:17.655034+0000","last_became_peered":"2024-03-23T15:41:17.655034+0000","last_unstale":"2024-03-23T15:42:31.774809+0000","last_undegraded":"2024-03-23T15:42:31.774809+0000","last_fullsized":"2024-03-23T15:42:31.774809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:30.973048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623424+0000","last_change":"2024-03-23T15:41:18.660248+0000","last_active":"2024-03-23T15:42:16.623424+0000","last_peered":"2024-03-23T15:42:16.623424+0000","last_clean":"2024-03-23T15:42:16.623424+0000","last_became_active":"2024-03-23T15:41:18.659728+0000","last_became_peered":"2024-03-23T15:41:18.659728+0000","last_unstale":"2024-03-23T15:42:16.623424+0000","last_undegraded":"2024-03-23T15:42:16.623424+0000","last_fullsized":"2024-03-23T15:42:16.623424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:03.681499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620473+0000","last_change":"2024-03-23T15:41:17.659784+0000","last_active":"2024-03-23T15:42:16.620473+0000","last_peered":"2024-03-23T15:42:16.620473+0000","last_clean":"2024-03-23T15:42:16.620473+0000","last_became_active":"2024-03-23T15:41:17.659422+0000","last_became_peered":"2024-03-23T15:41:17.659422+0000","last_unstale":"2024-03-23T15:42:16.620473+0000","last_undegraded":"2024-03-23T15:42:16.620473+0000","last_fullsized":"2024-03-23T15:42:16.620473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:48.607487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622219+0000","last_change":"2024-03-23T15:41:18.651777+0000","last_active":"2024-03-23T15:42:16.622219+0000","last_peered":"2024-03-23T15:42:16.622219+0000","last_clean":"2024-03-23T15:42:16.622219+0000","last_became_active":"2024-03-23T15:41:18.651457+0000","last_became_peered":"2024-03-23T15:41:18.651457+0000","last_unstale":"2024-03-23T15:42:16.622219+0000","last_undegraded":"2024-03-23T15:42:16.622219+0000","last_fullsized":"2024-03-23T15:42:16.622219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:25:28.831493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623195+0000","last_change":"2024-03-23T15:41:17.656357+0000","last_active":"2024-03-23T15:42:16.623195+0000","last_peered":"2024-03-23T15:42:16.623195+0000","last_clean":"2024-03-23T15:42:16.623195+0000","last_became_active":"2024-03-23T15:41:17.655977+0000","last_became_peered":"2024-03-23T15:41:17.655977+0000","last_unstale":"2024-03-23T15:42:16.623195+0000","last_undegraded":"2024-03-23T15:42:16.623195+0000","last_fullsized":"2024-03-23T15:42:16.623195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:07:45.248954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626757+0000","last_change":"2024-03-23T15:42:16.626757+0000","last_active":"2024-03-23T15:42:16.626757+0000","last_peered":"2024-03-23T15:42:16.626757+0000","last_clean":"2024-03-23T15:42:16.626757+0000","last_became_active":"2024-03-23T15:42:16.626409+0000","last_became_peered":"2024-03-23T15:42:16.626409+0000","last_unstale":"2024-03-23T15:42:16.626757+0000","last_undegraded":"2024-03-23T15:42:16.626757+0000","last_fullsized":"2024-03-23T15:42:16.626757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:46.969160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":197,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:32.775903+0000","last_change":"2024-03-23T15:41:17.659838+0000","last_active":"2024-03-23T15:42:32.775903+0000","last_peered":"2024-03-23T15:42:32.775903+0000","last_clean":"2024-03-23T15:42:32.775903+0000","last_became_active":"2024-03-23T15:41:17.659326+0000","last_became_peered":"2024-03-23T15:41:17.659326+0000","last_unstale":"2024-03-23T15:42:32.775903+0000","last_undegraded":"2024-03-23T15:42:32.775903+0000","last_fullsized":"2024-03-23T15:42:32.775903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:07.651340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":150,"num_read_kb":134,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620436+0000","last_change":"2024-03-23T15:41:18.656629+0000","last_active":"2024-03-23T15:42:16.620436+0000","last_peered":"2024-03-23T15:42:16.620436+0000","last_clean":"2024-03-23T15:42:16.620436+0000","last_became_active":"2024-03-23T15:41:18.656196+0000","last_became_peered":"2024-03-23T15:41:18.656196+0000","last_unstale":"2024-03-23T15:42:16.620436+0000","last_undegraded":"2024-03-23T15:42:16.620436+0000","last_fullsized":"2024-03-23T15:42:16.620436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:30.013411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626655+0000","last_change":"2024-03-23T15:41:17.663999+0000","last_active":"2024-03-23T15:42:16.626655+0000","last_peered":"2024-03-23T15:42:16.626655+0000","last_clean":"2024-03-23T15:42:16.626655+0000","last_became_active":"2024-03-23T15:41:17.663039+0000","last_became_peered":"2024-03-23T15:41:17.663039+0000","last_unstale":"2024-03-23T15:42:16.626655+0000","last_undegraded":"2024-03-23T15:42:16.626655+0000","last_fullsized":"2024-03-23T15:42:16.626655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:00.271821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625635+0000","last_change":"2024-03-23T15:41:18.646822+0000","last_active":"2024-03-23T15:42:16.625635+0000","last_peered":"2024-03-23T15:42:16.625635+0000","last_clean":"2024-03-23T15:42:16.625635+0000","last_became_active":"2024-03-23T15:41:18.646476+0000","last_became_peered":"2024-03-23T15:41:18.646476+0000","last_unstale":"2024-03-23T15:42:16.625635+0000","last_undegraded":"2024-03-23T15:42:16.625635+0000","last_fullsized":"2024-03-23T15:42:16.625635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:18.352065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623150+0000","last_change":"2024-03-23T15:41:17.663507+0000","last_active":"2024-03-23T15:42:16.623150+0000","last_peered":"2024-03-23T15:42:16.623150+0000","last_clean":"2024-03-23T15:42:16.623150+0000","last_became_active":"2024-03-23T15:41:17.663122+0000","last_became_peered":"2024-03-23T15:41:17.663122+0000","last_unstale":"2024-03-23T15:42:16.623150+0000","last_undegraded":"2024-03-23T15:42:16.623150+0000","last_fullsized":"2024-03-23T15:42:16.623150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:32.202576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620544+0000","last_change":"2024-03-23T15:41:18.650110+0000","last_active":"2024-03-23T15:42:16.620544+0000","last_peered":"2024-03-23T15:42:16.620544+0000","last_clean":"2024-03-23T15:42:16.620544+0000","last_became_active":"2024-03-23T15:41:18.649737+0000","last_became_peered":"2024-03-23T15:41:18.649737+0000","last_unstale":"2024-03-23T15:42:16.620544+0000","last_undegraded":"2024-03-23T15:42:16.620544+0000","last_fullsized":"2024-03-23T15:42:16.620544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:31.610961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:32.777708+0000","last_change":"2024-03-23T15:41:17.653930+0000","last_active":"2024-03-23T15:42:32.777708+0000","last_peered":"2024-03-23T15:42:32.777708+0000","last_clean":"2024-03-23T15:42:32.777708+0000","last_became_active":"2024-03-23T15:41:17.652677+0000","last_became_peered":"2024-03-23T15:41:17.652677+0000","last_unstale":"2024-03-23T15:42:32.777708+0000","last_undegraded":"2024-03-23T15:42:32.777708+0000","last_fullsized":"2024-03-23T15:42:32.777708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:47.167052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843231+0000","last_change":"2024-03-23T15:42:16.843231+0000","last_active":"2024-03-23T15:42:16.843231+0000","last_peered":"2024-03-23T15:42:16.843231+0000","last_clean":"2024-03-23T15:42:16.843231+0000","last_became_active":"2024-03-23T15:42:16.842785+0000","last_became_peered":"2024-03-23T15:42:16.842785+0000","last_unstale":"2024-03-23T15:42:16.843231+0000","last_undegraded":"2024-03-23T15:42:16.843231+0000","last_fullsized":"2024-03-23T15:42:16.843231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:24:07.377828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620049+0000","last_change":"2024-03-23T15:41:17.662091+0000","last_active":"2024-03-23T15:42:16.620049+0000","last_peered":"2024-03-23T15:42:16.620049+0000","last_clean":"2024-03-23T15:42:16.620049+0000","last_became_active":"2024-03-23T15:41:17.661747+0000","last_became_peered":"2024-03-23T15:41:17.661747+0000","last_unstale":"2024-03-23T15:42:16.620049+0000","last_undegraded":"2024-03-23T15:42:16.620049+0000","last_fullsized":"2024-03-23T15:42:16.620049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:08.661262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627776+0000","last_change":"2024-03-23T15:42:16.627776+0000","last_active":"2024-03-23T15:42:16.627776+0000","last_peered":"2024-03-23T15:42:16.627776+0000","last_clean":"2024-03-23T15:42:16.627776+0000","last_became_active":"2024-03-23T15:42:16.627331+0000","last_became_peered":"2024-03-23T15:42:16.627331+0000","last_unstale":"2024-03-23T15:42:16.627776+0000","last_undegraded":"2024-03-23T15:42:16.627776+0000","last_fullsized":"2024-03-23T15:42:16.627776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:02:57.288827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620552+0000","last_change":"2024-03-23T15:41:17.660898+0000","last_active":"2024-03-23T15:42:16.620552+0000","last_peered":"2024-03-23T15:42:16.620552+0000","last_clean":"2024-03-23T15:42:16.620552+0000","last_became_active":"2024-03-23T15:41:17.660515+0000","last_became_peered":"2024-03-23T15:41:17.660515+0000","last_unstale":"2024-03-23T15:42:16.620552+0000","last_undegraded":"2024-03-23T15:42:16.620552+0000","last_fullsized":"2024-03-23T15:42:16.620552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:58.911321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622448+0000","last_change":"2024-03-23T15:41:18.653582+0000","last_active":"2024-03-23T15:42:16.622448+0000","last_peered":"2024-03-23T15:42:16.622448+0000","last_clean":"2024-03-23T15:42:16.622448+0000","last_became_active":"2024-03-23T15:41:18.653184+0000","last_became_peered":"2024-03-23T15:41:18.653184+0000","last_unstale":"2024-03-23T15:42:16.622448+0000","last_undegraded":"2024-03-23T15:42:16.622448+0000","last_fullsized":"2024-03-23T15:42:16.622448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:59:16.726981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841716+0000","last_change":"2024-03-23T15:41:17.651886+0000","last_active":"2024-03-23T15:42:16.841716+0000","last_peered":"2024-03-23T15:42:16.841716+0000","last_clean":"2024-03-23T15:42:16.841716+0000","last_became_active":"2024-03-23T15:41:17.651472+0000","last_became_peered":"2024-03-23T15:41:17.651472+0000","last_unstale":"2024-03-23T15:42:16.841716+0000","last_undegraded":"2024-03-23T15:42:16.841716+0000","last_fullsized":"2024-03-23T15:42:16.841716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:37.098333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625275+0000","last_change":"2024-03-23T15:41:18.644205+0000","last_active":"2024-03-23T15:42:16.625275+0000","last_peered":"2024-03-23T15:42:16.625275+0000","last_clean":"2024-03-23T15:42:16.625275+0000","last_became_active":"2024-03-23T15:41:18.643850+0000","last_became_peered":"2024-03-23T15:41:18.643850+0000","last_unstale":"2024-03-23T15:42:16.625275+0000","last_undegraded":"2024-03-23T15:42:16.625275+0000","last_fullsized":"2024-03-23T15:42:16.625275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:48:10.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620920+0000","last_change":"2024-03-23T15:41:17.661051+0000","last_active":"2024-03-23T15:42:16.620920+0000","last_peered":"2024-03-23T15:42:16.620920+0000","last_clean":"2024-03-23T15:42:16.620920+0000","last_became_active":"2024-03-23T15:41:17.659673+0000","last_became_peered":"2024-03-23T15:41:17.659673+0000","last_unstale":"2024-03-23T15:42:16.620920+0000","last_undegraded":"2024-03-23T15:42:16.620920+0000","last_fullsized":"2024-03-23T15:42:16.620920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.028104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843248+0000","last_change":"2024-03-23T15:42:16.843248+0000","last_active":"2024-03-23T15:42:16.843248+0000","last_peered":"2024-03-23T15:42:16.843248+0000","last_clean":"2024-03-23T15:42:16.843248+0000","last_became_active":"2024-03-23T15:42:16.842800+0000","last_became_peered":"2024-03-23T15:42:16.842800+0000","last_unstale":"2024-03-23T15:42:16.843248+0000","last_undegraded":"2024-03-23T15:42:16.843248+0000","last_fullsized":"2024-03-23T15:42:16.843248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623438+0000","last_change":"2024-03-23T15:41:18.653560+0000","last_active":"2024-03-23T15:42:16.623438+0000","last_peered":"2024-03-23T15:42:16.623438+0000","last_clean":"2024-03-23T15:42:16.623438+0000","last_became_active":"2024-03-23T15:41:18.653249+0000","last_became_peered":"2024-03-23T15:41:18.653249+0000","last_unstale":"2024-03-23T15:42:16.623438+0000","last_undegraded":"2024-03-23T15:42:16.623438+0000","last_fullsized":"2024-03-23T15:42:16.623438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:22.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:33.778347+0000","last_change":"2024-03-23T15:41:17.656318+0000","last_active":"2024-03-23T15:42:33.778347+0000","last_peered":"2024-03-23T15:42:33.778347+0000","last_clean":"2024-03-23T15:42:33.778347+0000","last_became_active":"2024-03-23T15:41:17.655958+0000","last_became_peered":"2024-03-23T15:41:17.655958+0000","last_unstale":"2024-03-23T15:42:33.778347+0000","last_undegraded":"2024-03-23T15:42:33.778347+0000","last_fullsized":"2024-03-23T15:42:33.778347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:12.333504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628493+0000","last_change":"2024-03-23T15:41:18.657616+0000","last_active":"2024-03-23T15:42:16.628493+0000","last_peered":"2024-03-23T15:42:16.628493+0000","last_clean":"2024-03-23T15:42:16.628493+0000","last_became_active":"2024-03-23T15:41:18.657271+0000","last_became_peered":"2024-03-23T15:41:18.657271+0000","last_unstale":"2024-03-23T15:42:16.628493+0000","last_undegraded":"2024-03-23T15:42:16.628493+0000","last_fullsized":"2024-03-23T15:42:16.628493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:02.241087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689107+0000","last_change":"2024-03-23T15:41:17.654358+0000","last_active":"2024-03-23T15:42:16.689107+0000","last_peered":"2024-03-23T15:42:16.689107+0000","last_clean":"2024-03-23T15:42:16.689107+0000","last_became_active":"2024-03-23T15:41:17.653558+0000","last_became_peered":"2024-03-23T15:41:17.653558+0000","last_unstale":"2024-03-23T15:42:16.689107+0000","last_undegraded":"2024-03-23T15:42:16.689107+0000","last_fullsized":"2024-03-23T15:42:16.689107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:32.377883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620261+0000","last_change":"2024-03-23T15:40:05.871801+0000","last_active":"2024-03-23T15:42:16.620261+0000","last_peered":"2024-03-23T15:42:16.620261+0000","last_clean":"2024-03-23T15:42:16.620261+0000","last_became_active":"2024-03-23T15:40:05.871417+0000","last_became_peered":"2024-03-23T15:40:05.871417+0000","last_unstale":"2024-03-23T15:42:16.620261+0000","last_undegraded":"2024-03-23T15:42:16.620261+0000","last_fullsized":"2024-03-23T15:42:16.620261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_clean_scrub_stamp":"2024-03-23T15:36:39.745753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:55.235176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625401+0000","last_change":"2024-03-23T15:41:18.652251+0000","last_active":"2024-03-23T15:42:16.625401+0000","last_peered":"2024-03-23T15:42:16.625401+0000","last_clean":"2024-03-23T15:42:16.625401+0000","last_became_active":"2024-03-23T15:41:18.651876+0000","last_became_peered":"2024-03-23T15:41:18.651876+0000","last_unstale":"2024-03-23T15:42:16.625401+0000","last_undegraded":"2024-03-23T15:42:16.625401+0000","last_fullsized":"2024-03-23T15:42:16.625401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:45.235243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.692783+0000","last_change":"2024-03-23T15:42:16.692783+0000","last_active":"2024-03-23T15:42:16.692783+0000","last_peered":"2024-03-23T15:42:16.692783+0000","last_clean":"2024-03-23T15:42:16.692783+0000","last_became_active":"2024-03-23T15:42:16.690169+0000","last_became_peered":"2024-03-23T15:42:16.690169+0000","last_unstale":"2024-03-23T15:42:16.692783+0000","last_undegraded":"2024-03-23T15:42:16.692783+0000","last_fullsized":"2024-03-23T15:42:16.692783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:20.679040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620537+0000","last_change":"2024-03-23T15:41:17.660703+0000","last_active":"2024-03-23T15:42:16.620537+0000","last_peered":"2024-03-23T15:42:16.620537+0000","last_clean":"2024-03-23T15:42:16.620537+0000","last_became_active":"2024-03-23T15:41:17.660343+0000","last_became_peered":"2024-03-23T15:41:17.660343+0000","last_unstale":"2024-03-23T15:42:16.620537+0000","last_undegraded":"2024-03-23T15:42:16.620537+0000","last_fullsized":"2024-03-23T15:42:16.620537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:14:03.063397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625621+0000","last_change":"2024-03-23T15:41:18.644624+0000","last_active":"2024-03-23T15:42:16.625621+0000","last_peered":"2024-03-23T15:42:16.625621+0000","last_clean":"2024-03-23T15:42:16.625621+0000","last_became_active":"2024-03-23T15:41:18.644275+0000","last_became_peered":"2024-03-23T15:41:18.644275+0000","last_unstale":"2024-03-23T15:42:16.625621+0000","last_undegraded":"2024-03-23T15:42:16.625621+0000","last_fullsized":"2024-03-23T15:42:16.625621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:25.901935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.841905+0000","last_change":"2024-03-23T15:41:17.650068+0000","last_active":"2024-03-23T15:42:16.841905+0000","last_peered":"2024-03-23T15:42:16.841905+0000","last_clean":"2024-03-23T15:42:16.841905+0000","last_became_active":"2024-03-23T15:41:17.649331+0000","last_became_peered":"2024-03-23T15:41:17.649331+0000","last_unstale":"2024-03-23T15:42:16.841905+0000","last_undegraded":"2024-03-23T15:42:16.841905+0000","last_fullsized":"2024-03-23T15:42:16.841905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:47.578645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628165+0000","last_change":"2024-03-23T15:41:18.656413+0000","last_active":"2024-03-23T15:42:16.628165+0000","last_peered":"2024-03-23T15:42:16.628165+0000","last_clean":"2024-03-23T15:42:16.628165+0000","last_became_active":"2024-03-23T15:41:18.656043+0000","last_became_peered":"2024-03-23T15:41:18.656043+0000","last_unstale":"2024-03-23T15:42:16.628165+0000","last_undegraded":"2024-03-23T15:42:16.628165+0000","last_fullsized":"2024-03-23T15:42:16.628165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:49:25.453420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.689057+0000","last_change":"2024-03-23T15:41:17.654492+0000","last_active":"2024-03-23T15:42:16.689057+0000","last_peered":"2024-03-23T15:42:16.689057+0000","last_clean":"2024-03-23T15:42:16.689057+0000","last_became_active":"2024-03-23T15:41:17.654132+0000","last_became_peered":"2024-03-23T15:41:17.654132+0000","last_unstale":"2024-03-23T15:42:16.689057+0000","last_undegraded":"2024-03-23T15:42:16.689057+0000","last_fullsized":"2024-03-23T15:42:16.689057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:39:58.210703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622330+0000","last_change":"2024-03-23T15:41:18.652659+0000","last_active":"2024-03-23T15:42:16.622330+0000","last_peered":"2024-03-23T15:42:16.622330+0000","last_clean":"2024-03-23T15:42:16.622330+0000","last_became_active":"2024-03-23T15:41:18.652246+0000","last_became_peered":"2024-03-23T15:41:18.652246+0000","last_unstale":"2024-03-23T15:42:16.622330+0000","last_undegraded":"2024-03-23T15:42:16.622330+0000","last_fullsized":"2024-03-23T15:42:16.622330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:50.868694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.632166+0000","last_change":"2024-03-23T15:42:16.632166+0000","last_active":"2024-03-23T15:42:16.632166+0000","last_peered":"2024-03-23T15:42:16.632166+0000","last_clean":"2024-03-23T15:42:16.632166+0000","last_became_active":"2024-03-23T15:42:16.626587+0000","last_became_peered":"2024-03-23T15:42:16.626587+0000","last_unstale":"2024-03-23T15:42:16.632166+0000","last_undegraded":"2024-03-23T15:42:16.632166+0000","last_fullsized":"2024-03-23T15:42:16.632166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.514597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620060+0000","last_change":"2024-03-23T15:41:18.654049+0000","last_active":"2024-03-23T15:42:16.620060+0000","last_peered":"2024-03-23T15:42:16.620060+0000","last_clean":"2024-03-23T15:42:16.620060+0000","last_became_active":"2024-03-23T15:41:18.653765+0000","last_became_peered":"2024-03-23T15:41:18.653765+0000","last_unstale":"2024-03-23T15:42:16.620060+0000","last_undegraded":"2024-03-23T15:42:16.620060+0000","last_fullsized":"2024-03-23T15:42:16.620060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:12:11.467744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627573+0000","last_change":"2024-03-23T15:41:17.656679+0000","last_active":"2024-03-23T15:42:16.627573+0000","last_peered":"2024-03-23T15:42:16.627573+0000","last_clean":"2024-03-23T15:42:16.627573+0000","last_became_active":"2024-03-23T15:41:17.656368+0000","last_became_peered":"2024-03-23T15:41:17.656368+0000","last_unstale":"2024-03-23T15:42:16.627573+0000","last_undegraded":"2024-03-23T15:42:16.627573+0000","last_fullsized":"2024-03-23T15:42:16.627573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:14.176288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622132+0000","last_change":"2024-03-23T15:41:18.653637+0000","last_active":"2024-03-23T15:42:16.622132+0000","last_peered":"2024-03-23T15:42:16.622132+0000","last_clean":"2024-03-23T15:42:16.622132+0000","last_became_active":"2024-03-23T15:41:18.652864+0000","last_became_peered":"2024-03-23T15:41:18.652864+0000","last_unstale":"2024-03-23T15:42:16.622132+0000","last_undegraded":"2024-03-23T15:42:16.622132+0000","last_fullsized":"2024-03-23T15:42:16.622132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:32.104901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626421+0000","last_change":"2024-03-23T15:41:17.664417+0000","last_active":"2024-03-23T15:42:16.626421+0000","last_peered":"2024-03-23T15:42:16.626421+0000","last_clean":"2024-03-23T15:42:16.626421+0000","last_became_active":"2024-03-23T15:41:17.663454+0000","last_became_peered":"2024-03-23T15:41:17.663454+0000","last_unstale":"2024-03-23T15:42:16.626421+0000","last_undegraded":"2024-03-23T15:42:16.626421+0000","last_fullsized":"2024-03-23T15:42:16.626421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:14.947767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627594+0000","last_change":"2024-03-23T15:41:18.647857+0000","last_active":"2024-03-23T15:42:16.627594+0000","last_peered":"2024-03-23T15:42:16.627594+0000","last_clean":"2024-03-23T15:42:16.627594+0000","last_became_active":"2024-03-23T15:41:18.646924+0000","last_became_peered":"2024-03-23T15:41:18.646924+0000","last_unstale":"2024-03-23T15:42:16.627594+0000","last_undegraded":"2024-03-23T15:42:16.627594+0000","last_fullsized":"2024-03-23T15:42:16.627594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:19.494901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620229+0000","last_change":"2024-03-23T15:41:17.661995+0000","last_active":"2024-03-23T15:42:16.620229+0000","last_peered":"2024-03-23T15:42:16.620229+0000","last_clean":"2024-03-23T15:42:16.620229+0000","last_became_active":"2024-03-23T15:41:17.661632+0000","last_became_peered":"2024-03-23T15:41:17.661632+0000","last_unstale":"2024-03-23T15:42:16.620229+0000","last_undegraded":"2024-03-23T15:42:16.620229+0000","last_fullsized":"2024-03-23T15:42:16.620229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:12.540038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620831+0000","last_change":"2024-03-23T15:41:18.651258+0000","last_active":"2024-03-23T15:42:16.620831+0000","last_peered":"2024-03-23T15:42:16.620831+0000","last_clean":"2024-03-23T15:42:16.620831+0000","last_became_active":"2024-03-23T15:41:18.650910+0000","last_became_peered":"2024-03-23T15:41:18.650910+0000","last_unstale":"2024-03-23T15:42:16.620831+0000","last_undegraded":"2024-03-23T15:42:16.620831+0000","last_fullsized":"2024-03-23T15:42:16.620831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:06:50.139644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626748+0000","last_change":"2024-03-23T15:41:17.654232+0000","last_active":"2024-03-23T15:42:16.626748+0000","last_peered":"2024-03-23T15:42:16.626748+0000","last_clean":"2024-03-23T15:42:16.626748+0000","last_became_active":"2024-03-23T15:41:17.653832+0000","last_became_peered":"2024-03-23T15:41:17.653832+0000","last_unstale":"2024-03-23T15:42:16.626748+0000","last_undegraded":"2024-03-23T15:42:16.626748+0000","last_fullsized":"2024-03-23T15:42:16.626748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:05.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620932+0000","last_change":"2024-03-23T15:41:18.664444+0000","last_active":"2024-03-23T15:42:16.620932+0000","last_peered":"2024-03-23T15:42:16.620932+0000","last_clean":"2024-03-23T15:42:16.620932+0000","last_became_active":"2024-03-23T15:41:18.664127+0000","last_became_peered":"2024-03-23T15:41:18.664127+0000","last_unstale":"2024-03-23T15:42:16.620932+0000","last_undegraded":"2024-03-23T15:42:16.620932+0000","last_fullsized":"2024-03-23T15:42:16.620932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:25.942230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.623064+0000","last_change":"2024-03-23T15:41:17.664834+0000","last_active":"2024-03-23T15:42:16.623064+0000","last_peered":"2024-03-23T15:42:16.623064+0000","last_clean":"2024-03-23T15:42:16.623064+0000","last_became_active":"2024-03-23T15:41:17.664286+0000","last_became_peered":"2024-03-23T15:41:17.664286+0000","last_unstale":"2024-03-23T15:42:16.623064+0000","last_undegraded":"2024-03-23T15:42:16.623064+0000","last_fullsized":"2024-03-23T15:42:16.623064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:50.595420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620354+0000","last_change":"2024-03-23T15:41:18.646341+0000","last_active":"2024-03-23T15:42:16.620354+0000","last_peered":"2024-03-23T15:42:16.620354+0000","last_clean":"2024-03-23T15:42:16.620354+0000","last_became_active":"2024-03-23T15:41:18.645995+0000","last_became_peered":"2024-03-23T15:41:18.645995+0000","last_unstale":"2024-03-23T15:42:16.620354+0000","last_undegraded":"2024-03-23T15:42:16.620354+0000","last_fullsized":"2024-03-23T15:42:16.620354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:56.408957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.694643+0000","last_change":"2024-03-23T15:42:16.694643+0000","last_active":"2024-03-23T15:42:16.694643+0000","last_peered":"2024-03-23T15:42:16.694643+0000","last_clean":"2024-03-23T15:42:16.694643+0000","last_became_active":"2024-03-23T15:42:16.690063+0000","last_became_peered":"2024-03-23T15:42:16.690063+0000","last_unstale":"2024-03-23T15:42:16.694643+0000","last_undegraded":"2024-03-23T15:42:16.694643+0000","last_fullsized":"2024-03-23T15:42:16.694643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:54.873573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.620756+0000","last_change":"2024-03-23T15:41:18.655250+0000","last_active":"2024-03-23T15:42:16.620756+0000","last_peered":"2024-03-23T15:42:16.620756+0000","last_clean":"2024-03-23T15:42:16.620756+0000","last_became_active":"2024-03-23T15:41:18.654888+0000","last_became_peered":"2024-03-23T15:41:18.654888+0000","last_unstale":"2024-03-23T15:42:16.620756+0000","last_undegraded":"2024-03-23T15:42:16.620756+0000","last_fullsized":"2024-03-23T15:42:16.620756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:36.730600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622152+0000","last_change":"2024-03-23T15:41:18.654309+0000","last_active":"2024-03-23T15:42:16.622152+0000","last_peered":"2024-03-23T15:42:16.622152+0000","last_clean":"2024-03-23T15:42:16.622152+0000","last_became_active":"2024-03-23T15:41:18.653319+0000","last_became_peered":"2024-03-23T15:41:18.653319+0000","last_unstale":"2024-03-23T15:42:16.622152+0000","last_undegraded":"2024-03-23T15:42:16.622152+0000","last_fullsized":"2024-03-23T15:42:16.622152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:02:05.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626359+0000","last_change":"2024-03-23T15:41:18.655179+0000","last_active":"2024-03-23T15:42:16.626359+0000","last_peered":"2024-03-23T15:42:16.626359+0000","last_clean":"2024-03-23T15:42:16.626359+0000","last_became_active":"2024-03-23T15:41:18.654830+0000","last_became_peered":"2024-03-23T15:41:18.654830+0000","last_unstale":"2024-03-23T15:42:16.626359+0000","last_undegraded":"2024-03-23T15:42:16.626359+0000","last_fullsized":"2024-03-23T15:42:16.626359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:29.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.626300+0000","last_change":"2024-03-23T15:41:18.650734+0000","last_active":"2024-03-23T15:42:16.626300+0000","last_peered":"2024-03-23T15:42:16.626300+0000","last_clean":"2024-03-23T15:42:16.626300+0000","last_became_active":"2024-03-23T15:41:18.649993+0000","last_became_peered":"2024-03-23T15:41:18.649993+0000","last_unstale":"2024-03-23T15:42:16.626300+0000","last_undegraded":"2024-03-23T15:42:16.626300+0000","last_fullsized":"2024-03-23T15:42:16.626300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:06.463525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622098+0000","last_change":"2024-03-23T15:41:18.655741+0000","last_active":"2024-03-23T15:42:16.622098+0000","last_peered":"2024-03-23T15:42:16.622098+0000","last_clean":"2024-03-23T15:42:16.622098+0000","last_became_active":"2024-03-23T15:41:18.653387+0000","last_became_peered":"2024-03-23T15:41:18.653387+0000","last_unstale":"2024-03-23T15:42:16.622098+0000","last_undegraded":"2024-03-23T15:42:16.622098+0000","last_fullsized":"2024-03-23T15:42:16.622098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:20:26.758549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625725+0000","last_change":"2024-03-23T15:41:18.654739+0000","last_active":"2024-03-23T15:42:16.625725+0000","last_peered":"2024-03-23T15:42:16.625725+0000","last_clean":"2024-03-23T15:42:16.625725+0000","last_became_active":"2024-03-23T15:41:18.654151+0000","last_became_peered":"2024-03-23T15:41:18.654151+0000","last_unstale":"2024-03-23T15:42:16.625725+0000","last_undegraded":"2024-03-23T15:42:16.625725+0000","last_fullsized":"2024-03-23T15:42:16.625725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:01.358959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622422+0000","last_change":"2024-03-23T15:41:18.654081+0000","last_active":"2024-03-23T15:42:16.622422+0000","last_peered":"2024-03-23T15:42:16.622422+0000","last_clean":"2024-03-23T15:42:16.622422+0000","last_became_active":"2024-03-23T15:41:18.653696+0000","last_became_peered":"2024-03-23T15:41:18.653696+0000","last_unstale":"2024-03-23T15:42:16.622422+0000","last_undegraded":"2024-03-23T15:42:16.622422+0000","last_fullsized":"2024-03-23T15:42:16.622422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:23:19.741645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627099+0000","last_change":"2024-03-23T15:41:18.652356+0000","last_active":"2024-03-23T15:42:16.627099+0000","last_peered":"2024-03-23T15:42:16.627099+0000","last_clean":"2024-03-23T15:42:16.627099+0000","last_became_active":"2024-03-23T15:41:18.651976+0000","last_became_peered":"2024-03-23T15:41:18.651976+0000","last_unstale":"2024-03-23T15:42:16.627099+0000","last_undegraded":"2024-03-23T15:42:16.627099+0000","last_fullsized":"2024-03-23T15:42:16.627099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:38.972186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.628124+0000","last_change":"2024-03-23T15:41:18.653775+0000","last_active":"2024-03-23T15:42:16.628124+0000","last_peered":"2024-03-23T15:42:16.628124+0000","last_clean":"2024-03-23T15:42:16.628124+0000","last_became_active":"2024-03-23T15:41:18.653404+0000","last_became_peered":"2024-03-23T15:41:18.653404+0000","last_unstale":"2024-03-23T15:42:16.628124+0000","last_undegraded":"2024-03-23T15:42:16.628124+0000","last_fullsized":"2024-03-23T15:42:16.628124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:28:57.534731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.843115+0000","last_change":"2024-03-23T15:42:16.843115+0000","last_active":"2024-03-23T15:42:16.843115+0000","last_peered":"2024-03-23T15:42:16.843115+0000","last_clean":"2024-03-23T15:42:16.843115+0000","last_became_active":"2024-03-23T15:42:16.842743+0000","last_became_peered":"2024-03-23T15:42:16.842743+0000","last_unstale":"2024-03-23T15:42:16.843115+0000","last_undegraded":"2024-03-23T15:42:16.843115+0000","last_fullsized":"2024-03-23T15:42:16.843115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:36:11.377398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.627449+0000","last_change":"2024-03-23T15:41:18.652487+0000","last_active":"2024-03-23T15:42:16.627449+0000","last_peered":"2024-03-23T15:42:16.627449+0000","last_clean":"2024-03-23T15:42:16.627449+0000","last_became_active":"2024-03-23T15:41:18.652151+0000","last_became_peered":"2024-03-23T15:41:18.652151+0000","last_unstale":"2024-03-23T15:42:16.627449+0000","last_undegraded":"2024-03-23T15:42:16.627449+0000","last_fullsized":"2024-03-23T15:42:16.627449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:39.133585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622943+0000","last_change":"2024-03-23T15:41:18.657265+0000","last_active":"2024-03-23T15:42:16.622943+0000","last_peered":"2024-03-23T15:42:16.622943+0000","last_clean":"2024-03-23T15:42:16.622943+0000","last_became_active":"2024-03-23T15:41:18.656970+0000","last_became_peered":"2024-03-23T15:41:18.656970+0000","last_unstale":"2024-03-23T15:42:16.622943+0000","last_undegraded":"2024-03-23T15:42:16.622943+0000","last_fullsized":"2024-03-23T15:42:16.622943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:22.808034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.625658+0000","last_change":"2024-03-23T15:41:18.655254+0000","last_active":"2024-03-23T15:42:16.625658+0000","last_peered":"2024-03-23T15:42:16.625658+0000","last_clean":"2024-03-23T15:42:16.625658+0000","last_became_active":"2024-03-23T15:41:18.654879+0000","last_became_peered":"2024-03-23T15:41:18.654879+0000","last_unstale":"2024-03-23T15:42:16.625658+0000","last_undegraded":"2024-03-23T15:42:16.625658+0000","last_fullsized":"2024-03-23T15:42:16.625658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:36:18.923723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-23T15:42:16.622784+0000","last_change":"2024-03-23T15:41:18.658652+0000","last_active":"2024-03-23T15:42:16.622784+0000","last_peered":"2024-03-23T15:42:16.622784+0000","last_clean":"2024-03-23T15:42:16.622784+0000","last_became_active":"2024-03-23T15:41:18.658261+0000","last_became_peered":"2024-03-23T15:41:18.658261+0000","last_unstale":"2024-03-23T15:42:16.622784+0000","last_undegraded":"2024-03-23T15:42:16.622784+0000","last_fullsized":"2024-03-23T15:42:16.622784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:45.145101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245950,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":457,"num_read_kb":297,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":195141,"data_compressed":5607,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":456395,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1089536,"data_stored":1108922,"data_compressed":43282,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456631,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456551,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":1048481,"data_compressed":41413,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":946176,"data_stored":456020,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":516859,"data_compressed":7678,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457291,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":518600,"data_compressed":7678,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":456331,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":456457,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":1048165,"data_compressed":41413,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33965,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62787,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62624,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T15:42:39.682 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-23T15:42:39.685 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T15:42:39.685 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-23T15:42:39.685 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-23T15:42:39.926 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:40.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:40 smithi183 ceph-mon[36296]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:42:40.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:40 smithi183 ceph-mon[36296]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:42:41.006 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:40 smithi076 ceph-mon[32176]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:42:41.007 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:40 smithi076 ceph-mon[32176]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:42:41.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:40 smithi192 ceph-mon[36653]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:42:41.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:40 smithi192 ceph-mon[36653]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 -- 172.21.15.76:0/2987706251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4106140 msgr2=0x7f0fd4106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2987706251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4106140 0x7f0fd4106520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009990 tx=0x7f0fc4009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 -- 172.21.15.76:0/2987706251 shutdown_connections 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2987706251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 0x7f0fd410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2987706251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4106140 0x7f0fd4106520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2987706251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fd4101c50 0x7f0fd41020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 -- 172.21.15.76:0/2987706251 >> 172.21.15.76:0/2987706251 conn(0x7f0fd4076460 msgr2=0x7f0fd4076870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 -- 172.21.15.76:0/2987706251 shutdown_connections 2024-03-23T15:42:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.564+0000 7f0fda77f700 1 -- 172.21.15.76:0/2987706251 wait complete. 2024-03-23T15:42:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 Processor -- start 2024-03-23T15:42:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 -- start start 2024-03-23T15:42:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4101c50 0x7f0fd4197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 0x7f0fd4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fd4106140 0x7f0fd419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd419c530 con 0x7f0fd4101c50 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd419c6a0 con 0x7f0fd4106140 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fda77f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd419c860 con 0x7f0fd4102610 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fd3fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4101c50 0x7f0fd4197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fd37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 0x7f0fd4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fd3fff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4101c50 0x7f0fd4197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43934/0 (socket says 172.21.15.76:43934) 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fd37fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 0x7f0fd4197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:35226/0 (socket says 172.21.15.76:35226) 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fd3fff700 1 -- 172.21.15.76:0/2837437202 learned_addr learned my addr 172.21.15.76:0/2837437202 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.565+0000 7f0fd8d1c700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fd4106140 0x7f0fd419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd3fff700 1 -- 172.21.15.76:0/2837437202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 msgr2=0x7f0fd4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd3fff700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 0x7f0fd4197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd3fff700 1 -- 172.21.15.76:0/2837437202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fd4106140 msgr2=0x7f0fd419bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd3fff700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fd4106140 0x7f0fd419bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd3fff700 1 -- 172.21.15.76:0/2837437202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc40095f0 con 0x7f0fd4101c50 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd37fe700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 0x7f0fd4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd8d1c700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fd4106140 0x7f0fd419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd3fff700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4101c50 0x7f0fd4197650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0fc401ed00 tx=0x7f0fc401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd17fa700 1 -- 172.21.15.76:0/2837437202 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400be50 con 0x7f0fd4101c50 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd419cae0 con 0x7f0fd4101c50 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd17fa700 1 -- 172.21.15.76:0/2837437202 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fc400b610 con 0x7f0fd4101c50 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd4100d10 con 0x7f0fd4101c50 2024-03-23T15:42:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.566+0000 7f0fd17fa700 1 -- 172.21.15.76:0/2837437202 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4026770 con 0x7f0fd4101c50 2024-03-23T15:42:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.567+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd406a550 con 0x7f0fd4101c50 2024-03-23T15:42:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.568+0000 7f0fd17fa700 1 -- 172.21.15.76:0/2837437202 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f0fc400fa40 con 0x7f0fd4101c50 2024-03-23T15:42:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.569+0000 7f0fd17fa700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fbc0a21e0 0x7f0fbc0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.569+0000 7f0fd37fe700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fbc0a21e0 0x7f0fbc0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.569+0000 7f0fd17fa700 1 -- 172.21.15.76:0/2837437202 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0fc40d36e0 con 0x7f0fd4101c50 2024-03-23T15:42:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.569+0000 7f0fd37fe700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fbc0a21e0 0x7f0fbc0a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4076a60 tx=0x7f0fc8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.573+0000 7f0fd17fa700 1 -- 172.21.15.76:0/2837437202 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0fc409c5c0 con 0x7f0fd4101c50 2024-03-23T15:42:41.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.814+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0fd41989b0 con 0x7f0fd4101c50 2024-03-23T15:42:41.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.815+0000 7f0fd17fa700 1 -- 172.21.15.76:0/2837437202 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0fc409fc80 con 0x7f0fd4101c50 2024-03-23T15:42:41.818 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:41.818 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T15:42:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fbc0a21e0 msgr2=0x7f0fbc0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fbc0a21e0 0x7f0fbc0a46a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4076a60 tx=0x7f0fc8009260 comp rx=0 tx=0).stop 2024-03-23T15:42:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4101c50 msgr2=0x7f0fd4197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4101c50 0x7f0fd4197650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0fc401ed00 tx=0x7f0fc401ed30 comp rx=0 tx=0).stop 2024-03-23T15:42:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 shutdown_connections 2024-03-23T15:42:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0fbc0a21e0 0x7f0fbc0a46a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fd4106140 0x7f0fd419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4101c50 0x7f0fd4197650 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 --2- 172.21.15.76:0/2837437202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fd4102610 0x7f0fd4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 >> 172.21.15.76:0/2837437202 conn(0x7f0fd4076460 msgr2=0x7f0fd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 shutdown_connections 2024-03-23T15:42:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:41.817+0000 7f0fda77f700 1 -- 172.21.15.76:0/2837437202 wait complete. 2024-03-23T15:42:42.475 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-23T15:42:42.476 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:42:42.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 -- 172.21.15.76:0/1437175818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c1032e0 msgr2=0x7f2a6c109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/1437175818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c1032e0 0x7f2a6c109b70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009ca0 tx=0x7f2a6802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 -- 172.21.15.76:0/1437175818 shutdown_connections 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/1437175818 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c1032e0 0x7f2a6c109b70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/1437175818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a6c102920 0x7f2a6c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/1437175818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c101730 0x7f2a6c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 -- 172.21.15.76:0/1437175818 >> 172.21.15.76:0/1437175818 conn(0x7f2a6c0fccc0 msgr2=0x7f2a6c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 -- 172.21.15.76:0/1437175818 shutdown_connections 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.579+0000 7f2a73fc7640 1 -- 172.21.15.76:0/1437175818 wait complete. 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 Processor -- start 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 -- start start 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a6c101730 0x7f2a6c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c102920 0x7f2a6c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c1032e0 0x7f2a6c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c19f250 con 0x7f2a6c102920 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c19f3c0 con 0x7f2a6c101730 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a73fc7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c19f580 con 0x7f2a6c1032e0 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c102920 0x7f2a6c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c102920 0x7f2a6c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43952/0 (socket says 172.21.15.76:43952) 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 -- 172.21.15.76:0/3225657538 learned_addr learned my addr 172.21.15.76:0/3225657538 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a72564640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c1032e0 0x7f2a6c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71d63640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a6c101730 0x7f2a6c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 -- 172.21.15.76:0/3225657538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c1032e0 msgr2=0x7f2a6c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c1032e0 0x7f2a6c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 -- 172.21.15.76:0/3225657538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a6c101730 msgr2=0x7f2a6c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a6c101730 0x7f2a6c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 -- 172.21.15.76:0/3225657538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68009900 con 0x7f2a6c102920 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71d63640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a6c101730 0x7f2a6c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.580+0000 7f2a71562640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c102920 0x7f2a6c197ca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00e9f0 tx=0x7f2a5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.581+0000 7f2a62ffd640 1 -- 172.21.15.76:0/3225657538 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c009700 con 0x7f2a6c102920 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.581+0000 7f2a62ffd640 1 -- 172.21.15.76:0/3225657538 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c00cd80 con 0x7f2a6c102920 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.581+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c19f860 con 0x7f2a6c102920 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.581+0000 7f2a62ffd640 1 -- 172.21.15.76:0/3225657538 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c018920 con 0x7f2a6c102920 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.581+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c19fdc0 con 0x7f2a6c102920 2024-03-23T15:42:42.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.582+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c108a50 con 0x7f2a6c102920 2024-03-23T15:42:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.583+0000 7f2a62ffd640 1 -- 172.21.15.76:0/3225657538 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f2a5c018a80 con 0x7f2a6c102920 2024-03-23T15:42:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.583+0000 7f2a62ffd640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2a580a21d0 0x7f2a580a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.583+0000 7f2a71d63640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2a580a21d0 0x7f2a580a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.583+0000 7f2a62ffd640 1 -- 172.21.15.76:0/3225657538 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f2a5c014070 con 0x7f2a6c102920 2024-03-23T15:42:42.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.584+0000 7f2a71d63640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2a580a21d0 0x7f2a580a4690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c102780 tx=0x7f2a64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:42.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.588+0000 7f2a62ffd640 1 -- 172.21.15.76:0/3225657538 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2a5c053280 con 0x7f2a6c102920 2024-03-23T15:42:42.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.767+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a6c1084c0 con 0x7f2a6c102920 2024-03-23T15:42:42.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.768+0000 7f2a62ffd640 1 -- 172.21.15.76:0/3225657538 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22729 (secure 0 0 0) 0x7f2a5c097040 con 0x7f2a6c102920 2024-03-23T15:42:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:42.772 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:42:16.614449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2a580a21d0 msgr2=0x7f2a580a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2a580a21d0 0x7f2a580a4690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c102780 tx=0x7f2a64005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c102920 msgr2=0x7f2a6c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c102920 0x7f2a6c197ca0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00e9f0 tx=0x7f2a5c00eeb0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 shutdown_connections 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a6c1032e0 0x7f2a6c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c102920 0x7f2a6c197ca0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f2a580a21d0 0x7f2a580a4690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 --2- 172.21.15.76:0/3225657538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a6c101730 0x7f2a6c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 >> 172.21.15.76:0/3225657538 conn(0x7f2a6c0fccc0 msgr2=0x7f2a6c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 shutdown_connections 2024-03-23T15:42:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.771+0000 7f2a73fc7640 1 -- 172.21.15.76:0/3225657538 wait complete. 2024-03-23T15:42:42.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:42 smithi076 ceph-mon[32176]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:42:42.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:42 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2837437202' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T15:42:42.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:42:42.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:42:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:42 smithi183 ceph-mon[36296]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:42:42.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:42 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2837437202' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 -- 172.21.15.76:0/2512821946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 msgr2=0x7f8748100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 --2- 172.21.15.76:0/2512821946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748100a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8738009900 tx=0x7f873802bc30 comp rx=0 tx=0).stop 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 -- 172.21.15.76:0/2512821946 shutdown_connections 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 --2- 172.21.15.76:0/2512821946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87481021c0 0x7f8748108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 --2- 172.21.15.76:0/2512821946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8748101800 0x7f8748101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 --2- 172.21.15.76:0/2512821946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748100a30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 -- 172.21.15.76:0/2512821946 >> 172.21.15.76:0/2512821946 conn(0x7f87480fbb80 msgr2=0x7f87480fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:42.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 -- 172.21.15.76:0/2512821946 shutdown_connections 2024-03-23T15:42:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 -- 172.21.15.76:0/2512821946 wait complete. 2024-03-23T15:42:42.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.889+0000 7f874e180640 1 Processor -- start 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f874e180640 1 -- start start 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f874e180640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f874e180640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8748101800 0x7f874819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f874e180640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87481021c0 0x7f874819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f874e180640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f874819f300 con 0x7f8748100610 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f874e180640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f874819f470 con 0x7f87481021c0 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f874e180640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f874819f630 con 0x7f8748101800 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748069400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43968/0 (socket says 172.21.15.76:43968) 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 -- 172.21.15.76:0/1320422729 learned_addr learned my addr 172.21.15.76:0/1320422729 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f8746ffd640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8748101800 0x7f874819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f8747fff640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87481021c0 0x7f874819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 -- 172.21.15.76:0/1320422729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8748101800 msgr2=0x7f874819c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8748101800 0x7f874819c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 -- 172.21.15.76:0/1320422729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87481021c0 msgr2=0x7f874819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87481021c0 0x7f874819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 -- 172.21.15.76:0/1320422729 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8740009840 con 0x7f8748100610 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.890+0000 7f87477fe640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748069400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8738005ab0 tx=0x7f8738005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.891+0000 7f8744ff9640 1 -- 172.21.15.76:0/1320422729 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87380043d0 con 0x7f8748100610 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.891+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8738009560 con 0x7f8748100610 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.891+0000 7f8744ff9640 1 -- 172.21.15.76:0/1320422729 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87380052d0 con 0x7f8748100610 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.891+0000 7f8744ff9640 1 -- 172.21.15.76:0/1320422729 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738003cd0 con 0x7f8748100610 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.891+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874819fc30 con 0x7f8748100610 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.892+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87481079a0 con 0x7f8748100610 2024-03-23T15:42:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.893+0000 7f8744ff9640 1 -- 172.21.15.76:0/1320422729 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f8738005440 con 0x7f8748100610 2024-03-23T15:42:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.893+0000 7f8744ff9640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f87300a2150 0x7f87300a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.893+0000 7f8746ffd640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f87300a2150 0x7f87300a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.893+0000 7f8744ff9640 1 -- 172.21.15.76:0/1320422729 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f87380efe70 con 0x7f8748100610 2024-03-23T15:42:42.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.894+0000 7f8746ffd640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f87300a2150 0x7f87300a4610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8748069170 tx=0x7f873c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:42.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:42.898+0000 7f8744ff9640 1 -- 172.21.15.76:0/1320422729 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f87380b8dd0 con 0x7f8748100610 2024-03-23T15:42:43.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:42 smithi192 ceph-mon[36653]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:42:43.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:42 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2837437202' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T15:42:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.070+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f874810c270 con 0x7f8748100610 2024-03-23T15:42:43.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.070+0000 7f8744ff9640 1 -- 172.21.15.76:0/1320422729 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f87380bc490 con 0x7f8748100610 2024-03-23T15:42:43.074 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.072+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f87300a2150 msgr2=0x7f87300a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.072+0000 7f874e180640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f87300a2150 0x7f87300a4610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8748069170 tx=0x7f873c005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.072+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 msgr2=0x7f8748069400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.072+0000 7f874e180640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748069400 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8738005ab0 tx=0x7f8738005e50 comp rx=0 tx=0).stop 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 shutdown_connections 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f87300a2150 0x7f87300a4610 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87481021c0 0x7f874819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8748101800 0x7f874819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 --2- 172.21.15.76:0/1320422729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8748100610 0x7f8748069400 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 >> 172.21.15.76:0/1320422729 conn(0x7f87480fbb80 msgr2=0x7f8748104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:43.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 shutdown_connections 2024-03-23T15:42:43.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.073+0000 7f874e180640 1 -- 172.21.15.76:0/1320422729 wait complete. 2024-03-23T15:42:43.087 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 -- 172.21.15.76:0/2807783125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00101800 msgr2=0x7f9c00101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 --2- 172.21.15.76:0/2807783125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00101800 0x7f9c00101c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf402bc30 comp rx=0 tx=0).stop 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 -- 172.21.15.76:0/2807783125 shutdown_connections 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 --2- 172.21.15.76:0/2807783125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c001021c0 0x7f9c00108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 --2- 172.21.15.76:0/2807783125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00101800 0x7f9c00101c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 --2- 172.21.15.76:0/2807783125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c00100610 0x7f9c00100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 -- 172.21.15.76:0/2807783125 >> 172.21.15.76:0/2807783125 conn(0x7f9c000fbba0 msgr2=0x7f9c000fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 -- 172.21.15.76:0/2807783125 shutdown_connections 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.188+0000 7f9c062eb640 1 -- 172.21.15.76:0/2807783125 wait complete. 2024-03-23T15:42:43.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 Processor -- start 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 -- start start 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00100610 0x7f9c00069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c00101800 0x7f9c0019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c001021c0 0x7f9c0019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0019f2c0 con 0x7f9c00100610 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0019f430 con 0x7f9c00101800 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c062eb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0019f5f0 con 0x7f9c001021c0 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00100610 0x7f9c00069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9bfffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00100610 0x7f9c00069400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43988/0 (socket says 172.21.15.76:43988) 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9bfffff640 1 -- 172.21.15.76:0/1090371316 learned_addr learned my addr 172.21.15.76:0/1090371316 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9bff7fe640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c00101800 0x7f9c0019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.189+0000 7f9c04888640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c001021c0 0x7f9c0019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfffff640 1 -- 172.21.15.76:0/1090371316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c001021c0 msgr2=0x7f9c0019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfffff640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c001021c0 0x7f9c0019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfffff640 1 -- 172.21.15.76:0/1090371316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c00101800 msgr2=0x7f9c0019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfffff640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c00101800 0x7f9c0019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfffff640 1 -- 172.21.15.76:0/1090371316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf0009490 con 0x7f9c00100610 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfffff640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00100610 0x7f9c00069400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000e9c0 tx=0x7f9bf000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfd7fa640 1 -- 172.21.15.76:0/1090371316 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf000cde0 con 0x7f9c00100610 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfd7fa640 1 -- 172.21.15.76:0/1090371316 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf0010910 con 0x7f9c00100610 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf4009560 con 0x7f9c00100610 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9bfd7fa640 1 -- 172.21.15.76:0/1090371316 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0018a10 con 0x7f9c00100610 2024-03-23T15:42:43.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.190+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0019fc20 con 0x7f9c00100610 2024-03-23T15:42:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.191+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c00068500 con 0x7f9c00100610 2024-03-23T15:42:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.192+0000 7f9bfd7fa640 1 -- 172.21.15.76:0/1090371316 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f9bf0010a80 con 0x7f9c00100610 2024-03-23T15:42:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.193+0000 7f9bfd7fa640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9be80a2150 0x7f9be80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.193+0000 7f9bff7fe640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9be80a2150 0x7f9be80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.193+0000 7f9bfd7fa640 1 -- 172.21.15.76:0/1090371316 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9bf0014070 con 0x7f9c00100610 2024-03-23T15:42:43.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.193+0000 7f9bff7fe640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9be80a2150 0x7f9be80a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009230 tx=0x7f9bf4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:43.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.197+0000 7f9bfd7fa640 1 -- 172.21.15.76:0/1090371316 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9bf0097370 con 0x7f9c00100610 2024-03-23T15:42:43.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.371+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c001079b0 con 0x7f9c00100610 2024-03-23T15:42:43.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.371+0000 7f9bfd7fa640 1 -- 172.21.15.76:0/1090371316 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f9bf009aa30 con 0x7f9c00100610 2024-03-23T15:42:43.374 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.373+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9be80a2150 msgr2=0x7f9be80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.373+0000 7f9c062eb640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9be80a2150 0x7f9be80a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009230 tx=0x7f9bf4005fd0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.373+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00100610 msgr2=0x7f9c00069400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.373+0000 7f9c062eb640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00100610 0x7f9c00069400 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000e9c0 tx=0x7f9bf000ee80 comp rx=0 tx=0).stop 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 shutdown_connections 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c001021c0 0x7f9c0019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f9be80a2150 0x7f9be80a4610 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c00101800 0x7f9c0019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 --2- 172.21.15.76:0/1090371316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c00100610 0x7f9c00069400 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 >> 172.21.15.76:0/1090371316 conn(0x7f9c000fbba0 msgr2=0x7f9c00104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 shutdown_connections 2024-03-23T15:42:43.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.374+0000 7f9c062eb640 1 -- 172.21.15.76:0/1090371316 wait complete. 2024-03-23T15:42:43.388 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:42:43.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 -- 172.21.15.76:0/2275713338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 msgr2=0x7ffb781045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 --2- 172.21.15.76:0/2275713338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781045e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009ca0 tx=0x7ffb6802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 -- 172.21.15.76:0/2275713338 shutdown_connections 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 --2- 172.21.15.76:0/2275713338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb78107450 0x7ffb78109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 --2- 172.21.15.76:0/2275713338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb78104b20 0x7ffb78106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 --2- 172.21.15.76:0/2275713338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781045e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 -- 172.21.15.76:0/2275713338 >> 172.21.15.76:0/2275713338 conn(0x7ffb780fbb80 msgr2=0x7ffb780fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.490+0000 7ffb80081640 1 -- 172.21.15.76:0/2275713338 shutdown_connections 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 -- 172.21.15.76:0/2275713338 wait complete. 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 Processor -- start 2024-03-23T15:42:43.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 -- start start 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb78104b20 0x7ffb78197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb78107450 0x7ffb7819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7819f2c0 con 0x7ffb781021f0 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7819f430 con 0x7ffb78107450 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb80081640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7819f5f0 con 0x7ffb78104b20 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb7de1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb7de1d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43998/0 (socket says 172.21.15.76:43998) 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb7de1d640 1 -- 172.21.15.76:0/3016248249 learned_addr learned my addr 172.21.15.76:0/3016248249 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb7e61e640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb78107450 0x7ffb7819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.491+0000 7ffb7d61c640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb78104b20 0x7ffb78197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb7de1d640 1 -- 172.21.15.76:0/3016248249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb78104b20 msgr2=0x7ffb78197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb7de1d640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb78104b20 0x7ffb78197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb7de1d640 1 -- 172.21.15.76:0/3016248249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb78107450 msgr2=0x7ffb7819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb7de1d640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb78107450 0x7ffb7819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb7de1d640 1 -- 172.21.15.76:0/3016248249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb68009900 con 0x7ffb781021f0 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb7de1d640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781977d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009dc0 tx=0x7ffb680093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb76ffd640 1 -- 172.21.15.76:0/3016248249 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680329b0 con 0x7ffb781021f0 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb76ffd640 1 -- 172.21.15.76:0/3016248249 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb68003e50 con 0x7ffb781021f0 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7819f870 con 0x7ffb781021f0 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb76ffd640 1 -- 172.21.15.76:0/3016248249 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68040450 con 0x7ffb781021f0 2024-03-23T15:42:43.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.492+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7819fd20 con 0x7ffb781021f0 2024-03-23T15:42:43.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.493+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb780fd670 con 0x7ffb781021f0 2024-03-23T15:42:43.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.494+0000 7ffb76ffd640 1 -- 172.21.15.76:0/3016248249 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7ffb6800a040 con 0x7ffb781021f0 2024-03-23T15:42:43.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.494+0000 7ffb76ffd640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ffb600a2150 0x7ffb600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.495+0000 7ffb7d61c640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ffb600a2150 0x7ffb600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.495+0000 7ffb76ffd640 1 -- 172.21.15.76:0/3016248249 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ffb680f0ae0 con 0x7ffb781021f0 2024-03-23T15:42:43.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.495+0000 7ffb7d61c640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ffb600a2150 0x7ffb600a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb78198ce0 tx=0x7ffb6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:43.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.499+0000 7ffb76ffd640 1 -- 172.21.15.76:0/3016248249 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ffb680b9a40 con 0x7ffb781021f0 2024-03-23T15:42:43.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.674+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb7802d160 con 0x7ffb781021f0 2024-03-23T15:42:43.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.675+0000 7ffb76ffd640 1 -- 172.21.15.76:0/3016248249 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7ffb680bd100 con 0x7ffb781021f0 2024-03-23T15:42:43.678 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.676+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ffb600a2150 msgr2=0x7ffb600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.676+0000 7ffb80081640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ffb600a2150 0x7ffb600a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb78198ce0 tx=0x7ffb6c005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.676+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 msgr2=0x7ffb781977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.676+0000 7ffb80081640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781977d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009dc0 tx=0x7ffb680093f0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.676+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 shutdown_connections 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.676+0000 7ffb80081640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ffb600a2150 0x7ffb600a4610 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.676+0000 7ffb80081640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb78107450 0x7ffb7819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.677+0000 7ffb80081640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb78104b20 0x7ffb78197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.677+0000 7ffb80081640 1 --2- 172.21.15.76:0/3016248249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb781021f0 0x7ffb781977d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.677+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 >> 172.21.15.76:0/3016248249 conn(0x7ffb780fbb80 msgr2=0x7ffb78102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:43.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.677+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 shutdown_connections 2024-03-23T15:42:43.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.677+0000 7ffb80081640 1 -- 172.21.15.76:0/3016248249 wait complete. 2024-03-23T15:42:43.690 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:43 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3225657538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:43.690 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:43 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1320422729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:42:43.690 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:43 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1090371316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:42:43.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:42:43.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 -- 172.21.15.76:0/2723968574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc102200 msgr2=0x7fb5cc108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/2723968574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc102200 0x7fb5cc108a90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009ca0 tx=0x7fb5c402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 -- 172.21.15.76:0/2723968574 shutdown_connections 2024-03-23T15:42:43.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/2723968574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc102200 0x7fb5cc108a90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/2723968574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5cc101840 0x7fb5cc101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/2723968574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5cc100650 0x7fb5cc100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 -- 172.21.15.76:0/2723968574 >> 172.21.15.76:0/2723968574 conn(0x7fb5cc0fbbc0 msgr2=0x7fb5cc0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 -- 172.21.15.76:0/2723968574 shutdown_connections 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.791+0000 7fb5d1a07640 1 -- 172.21.15.76:0/2723968574 wait complete. 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 Processor -- start 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 -- start start 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc100650 0x7fb5cc1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5cc101840 0x7fb5cc197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5cc102200 0x7fb5cc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc19f2a0 con 0x7fb5cc100650 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc19f410 con 0x7fb5cc101840 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5d1a07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc19f5d0 con 0x7fb5cc102200 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc100650 0x7fb5cc1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc100650 0x7fb5cc1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44018/0 (socket says 172.21.15.76:44018) 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 -- 172.21.15.76:0/4116146451 learned_addr learned my addr 172.21.15.76:0/4116146451 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5cb7fe640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5cc102200 0x7fb5cc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5ca7fc640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5cc101840 0x7fb5cc197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 -- 172.21.15.76:0/4116146451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5cc102200 msgr2=0x7fb5cc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5cc102200 0x7fb5cc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 -- 172.21.15.76:0/4116146451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5cc101840 msgr2=0x7fb5cc197cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5cc101840 0x7fb5cc197cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.792+0000 7fb5caffd640 1 -- 172.21.15.76:0/4116146451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c4009900 con 0x7fb5cc100650 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.793+0000 7fb5caffd640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc100650 0x7fb5cc1977b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e9c0 tx=0x7fb5bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.793+0000 7fb5d0a05640 1 -- 172.21.15.76:0/4116146451 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0096e0 con 0x7fb5cc100650 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.793+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc19f8b0 con 0x7fb5cc100650 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.793+0000 7fb5d0a05640 1 -- 172.21.15.76:0/4116146451 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5bc00cd80 con 0x7fb5cc100650 2024-03-23T15:42:43.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.793+0000 7fb5d0a05640 1 -- 172.21.15.76:0/4116146451 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc010430 con 0x7fb5cc100650 2024-03-23T15:42:43.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.793+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc19fd70 con 0x7fb5cc100650 2024-03-23T15:42:43.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.794+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5cc10c060 con 0x7fb5cc100650 2024-03-23T15:42:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.795+0000 7fb5d0a05640 1 -- 172.21.15.76:0/4116146451 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fb5bc003680 con 0x7fb5cc100650 2024-03-23T15:42:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.795+0000 7fb5d0a05640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb5b40a2150 0x7fb5b40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.795+0000 7fb5ca7fc640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb5b40a2150 0x7fb5b40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.795+0000 7fb5d0a05640 1 -- 172.21.15.76:0/4116146451 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb5bc014070 con 0x7fb5cc100650 2024-03-23T15:42:43.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.796+0000 7fb5ca7fc640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb5b40a2150 0x7fb5b40a4610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc198cc0 tx=0x7fb5c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:43.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:43.800+0000 7fb5d0a05640 1 -- 172.21.15.76:0/4116146451 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fb5bc097030 con 0x7fb5cc100650 2024-03-23T15:42:43.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:43 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3225657538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:43.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:43 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1320422729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:42:43.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:43 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1090371316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:42:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.013+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5cc19fff0 con 0x7fb5cc100650 2024-03-23T15:42:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.013+0000 7fb5d0a05640 1 -- 172.21.15.76:0/4116146451 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7fb5cc19fff0 con 0x7fb5cc100650 2024-03-23T15:42:44.017 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:44.017 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:41:22.694125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:42:44.017 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 21 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb5b40a2150 msgr2=0x7fb5b40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb5b40a2150 0x7fb5b40a4610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc198cc0 tx=0x7fb5c0005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc100650 msgr2=0x7fb5cc1977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc100650 0x7fb5cc1977b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e9c0 tx=0x7fb5bc00ee80 comp rx=0 tx=0).stop 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 shutdown_connections 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5cc102200 0x7fb5cc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fb5b40a2150 0x7fb5b40a4610 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5cc101840 0x7fb5cc197cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 --2- 172.21.15.76:0/4116146451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc100650 0x7fb5cc1977b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.015+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 >> 172.21.15.76:0/4116146451 conn(0x7fb5cc0fbbc0 msgr2=0x7fb5cc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.016+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 shutdown_connections 2024-03-23T15:42:44.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.016+0000 7fb5d1a07640 1 -- 172.21.15.76:0/4116146451 wait complete. 2024-03-23T15:42:44.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:42:44.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:43 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3225657538' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:44.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:43 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1320422729' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:42:44.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:43 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1090371316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 -- 172.21.15.76:0/2826181189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 msgr2=0x7f96c8100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 --2- 172.21.15.76:0/2826181189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8100a70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009900 tx=0x7f96bc02bc30 comp rx=0 tx=0).stop 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 -- 172.21.15.76:0/2826181189 shutdown_connections 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 --2- 172.21.15.76:0/2826181189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8102200 0x7f96c8108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 --2- 172.21.15.76:0/2826181189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c8101840 0x7f96c8101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 --2- 172.21.15.76:0/2826181189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8100a70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 -- 172.21.15.76:0/2826181189 >> 172.21.15.76:0/2826181189 conn(0x7f96c80fbbc0 msgr2=0x7f96c80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 -- 172.21.15.76:0/2826181189 shutdown_connections 2024-03-23T15:42:44.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.131+0000 7f96ced93640 1 -- 172.21.15.76:0/2826181189 wait complete. 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 Processor -- start 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 -- start start 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8101840 0x7f96c8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c8102200 0x7f96c819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f96c819f300 con 0x7f96c8100650 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96c819f470 con 0x7f96c8102200 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f96c819f630 con 0x7f96c8101840 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44044/0 (socket says 172.21.15.76:44044) 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 -- 172.21.15.76:0/2973570046 learned_addr learned my addr 172.21.15.76:0/2973570046 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96cd330640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c8102200 0x7f96c819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96c7fff640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8101840 0x7f96c8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 -- 172.21.15.76:0/2973570046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8101840 msgr2=0x7f96c8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8101840 0x7f96c8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 -- 172.21.15.76:0/2973570046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c8102200 msgr2=0x7f96c819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c8102200 0x7f96c819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 -- 172.21.15.76:0/2973570046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b4009490 con 0x7f96c8100650 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96c7fff640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8101840 0x7f96c8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ccb2f640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8197810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96bc005ab0 tx=0x7f96bc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96c5ffb640 1 -- 172.21.15.76:0/2973570046 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0043d0 con 0x7f96c8100650 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96c5ffb640 1 -- 172.21.15.76:0/2973570046 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96bc0052d0 con 0x7f96c8100650 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc009560 con 0x7f96c8100650 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.132+0000 7f96c5ffb640 1 -- 172.21.15.76:0/2973570046 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc003d40 con 0x7f96c8100650 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.133+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c819fc30 con 0x7f96c8100650 2024-03-23T15:42:44.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.134+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c810c060 con 0x7f96c8100650 2024-03-23T15:42:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.135+0000 7f96c5ffb640 1 -- 172.21.15.76:0/2973570046 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f96bc005440 con 0x7f96c8100650 2024-03-23T15:42:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.135+0000 7f96c5ffb640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f96b80a2150 0x7f96b80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.135+0000 7f96c7fff640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f96b80a2150 0x7f96b80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.135+0000 7f96c5ffb640 1 -- 172.21.15.76:0/2973570046 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f96bc0eff10 con 0x7f96c8100650 2024-03-23T15:42:44.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.136+0000 7f96c7fff640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f96b80a2150 0x7f96b80a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96c8198d20 tx=0x7f96b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:44.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.139+0000 7f96c5ffb640 1 -- 172.21.15.76:0/2973570046 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f96bc0b8e70 con 0x7f96c8100650 2024-03-23T15:42:44.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.316+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96c810c1f0 con 0x7f96c8100650 2024-03-23T15:42:44.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.317+0000 7f96c5ffb640 1 -- 172.21.15.76:0/2973570046 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22729 (secure 0 0 0) 0x7f96bc0bc530 con 0x7f96c8100650 2024-03-23T15:42:44.320 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:44.321 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:42:16.614449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f96b80a2150 msgr2=0x7f96b80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f96b80a2150 0x7f96b80a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96c8198d20 tx=0x7f96b4005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 msgr2=0x7f96c8197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8197810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96bc005ab0 tx=0x7f96bc005e50 comp rx=0 tx=0).stop 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 shutdown_connections 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f96b80a2150 0x7f96b80a4610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96c8102200 0x7f96c819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c8101840 0x7f96c8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 --2- 172.21.15.76:0/2973570046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96c8100650 0x7f96c8197810 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 >> 172.21.15.76:0/2973570046 conn(0x7f96c80fbbc0 msgr2=0x7f96c80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 shutdown_connections 2024-03-23T15:42:44.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.319+0000 7f96ced93640 1 -- 172.21.15.76:0/2973570046 wait complete. 2024-03-23T15:42:44.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:42:44.334 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:42:44.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 -- 172.21.15.76:0/2692257412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101250 msgr2=0x7f38ac108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 --2- 172.21.15.76:0/2692257412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101250 0x7f38ac108af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009ca0 tx=0x7f38a402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 -- 172.21.15.76:0/2692257412 shutdown_connections 2024-03-23T15:42:44.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 --2- 172.21.15.76:0/2692257412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101250 0x7f38ac108af0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 --2- 172.21.15.76:0/2692257412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac100890 0x7f38ac100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 --2- 172.21.15.76:0/2692257412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ac101c40 0x7f38ac102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 -- 172.21.15.76:0/2692257412 >> 172.21.15.76:0/2692257412 conn(0x7f38ac0fb320 msgr2=0x7f38ac0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:44.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 -- 172.21.15.76:0/2692257412 shutdown_connections 2024-03-23T15:42:44.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 -- 172.21.15.76:0/2692257412 wait complete. 2024-03-23T15:42:44.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.439+0000 7f38b222a640 1 Processor -- start 2024-03-23T15:42:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 -- start start 2024-03-23T15:42:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac100890 0x7f38ac1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ac101250 0x7f38ac19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101c40 0x7f38ac19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac19f290 con 0x7f38ac101c40 2024-03-23T15:42:44.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac19f400 con 0x7f38ac100890 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac19f5c0 con 0x7f38ac101250 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101c40 0x7f38ac19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38ab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac100890 0x7f38ac1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101c40 0x7f38ac19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44054/0 (socket says 172.21.15.76:44054) 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 -- 172.21.15.76:0/3634818765 learned_addr learned my addr 172.21.15.76:0/3634818765 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38aaffd640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ac101250 0x7f38ac19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 -- 172.21.15.76:0/3634818765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ac101250 msgr2=0x7f38ac19c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ac101250 0x7f38ac19c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 -- 172.21.15.76:0/3634818765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac100890 msgr2=0x7f38ac1002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac100890 0x7f38ac1002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 -- 172.21.15.76:0/3634818765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a4009900 con 0x7f38ac101c40 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38ab7fe640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac100890 0x7f38ac1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:44.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38abfff640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101c40 0x7f38ac19eab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f38a4037040 tx=0x7f38a4032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38a8ff9640 1 -- 172.21.15.76:0/3634818765 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4034c90 con 0x7f38ac101c40 2024-03-23T15:42:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac19f7e0 con 0x7f38ac101c40 2024-03-23T15:42:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.440+0000 7f38a8ff9640 1 -- 172.21.15.76:0/3634818765 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a4034df0 con 0x7f38ac101c40 2024-03-23T15:42:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.441+0000 7f38a8ff9640 1 -- 172.21.15.76:0/3634818765 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4040450 con 0x7f38ac101c40 2024-03-23T15:42:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.441+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac19fc90 con 0x7f38ac101c40 2024-03-23T15:42:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.441+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac107970 con 0x7f38ac101c40 2024-03-23T15:42:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.442+0000 7f38a8ff9640 1 -- 172.21.15.76:0/3634818765 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f38a400a040 con 0x7f38ac101c40 2024-03-23T15:42:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.443+0000 7f38a8ff9640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f38940a2150 0x7f38940a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.443+0000 7f38ab7fe640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f38940a2150 0x7f38940a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.443+0000 7f38a8ff9640 1 -- 172.21.15.76:0/3634818765 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f38a40f0b50 con 0x7f38ac101c40 2024-03-23T15:42:44.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.443+0000 7f38ab7fe640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f38940a2150 0x7f38940a4610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38ac069ba0 tx=0x7f389c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:44.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.447+0000 7f38a8ff9640 1 -- 172.21.15.76:0/3634818765 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f38a40b9ab0 con 0x7f38ac101c40 2024-03-23T15:42:44.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.623+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38ac02d190 con 0x7f38ac101c40 2024-03-23T15:42:44.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.623+0000 7f38a8ff9640 1 -- 172.21.15.76:0/3634818765 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f38a40bd170 con 0x7f38ac101c40 2024-03-23T15:42:44.626 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f38940a2150 msgr2=0x7f38940a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f38940a2150 0x7f38940a4610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38ac069ba0 tx=0x7f389c005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101c40 msgr2=0x7f38ac19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101c40 0x7f38ac19eab0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f38a4037040 tx=0x7f38a4032fc0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 shutdown_connections 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38ac101c40 0x7f38ac19eab0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ac101250 0x7f38ac19c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f38940a2150 0x7f38940a4610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 --2- 172.21.15.76:0/3634818765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac100890 0x7f38ac1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 >> 172.21.15.76:0/3634818765 conn(0x7f38ac0fb320 msgr2=0x7f38ac104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:44.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 shutdown_connections 2024-03-23T15:42:44.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.625+0000 7f38b222a640 1 -- 172.21.15.76:0/3634818765 wait complete. 2024-03-23T15:42:44.639 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:42:44.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 -- 172.21.15.76:0/968391385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 msgr2=0x7fd8ec106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 --2- 172.21.15.76:0/968391385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec106eb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009900 tx=0x7fd8e002bc30 comp rx=0 tx=0).stop 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 -- 172.21.15.76:0/968391385 shutdown_connections 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 --2- 172.21.15.76:0/968391385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ec1073f0 0x7fd8ec109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 --2- 172.21.15.76:0/968391385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec106eb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 --2- 172.21.15.76:0/968391385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec102190 0x7fd8ec104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 -- 172.21.15.76:0/968391385 >> 172.21.15.76:0/968391385 conn(0x7fd8ec0fbba0 msgr2=0x7fd8ec0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 -- 172.21.15.76:0/968391385 shutdown_connections 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.740+0000 7fd8f3207640 1 -- 172.21.15.76:0/968391385 wait complete. 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 Processor -- start 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 -- start start 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec102190 0x7fd8ec1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ec1073f0 0x7fd8ec19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec19f340 con 0x7fd8ec104ac0 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec19f4b0 con 0x7fd8ec102190 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f3207640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec19f670 con 0x7fd8ec1073f0 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51060/0 (socket says 172.21.15.76:51060) 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 -- 172.21.15.76:0/1542353146 learned_addr learned my addr 172.21.15.76:0/1542353146 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f0fa3640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec102190 0x7fd8ec1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8f17a4640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ec1073f0 0x7fd8ec19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 -- 172.21.15.76:0/1542353146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ec1073f0 msgr2=0x7fd8ec19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ec1073f0 0x7fd8ec19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 -- 172.21.15.76:0/1542353146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec102190 msgr2=0x7fd8ec1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec102190 0x7fd8ec1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.741+0000 7fd8ebfff640 1 -- 172.21.15.76:0/1542353146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e0009560 con 0x7fd8ec104ac0 2024-03-23T15:42:44.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.742+0000 7fd8ebfff640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec197de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0005ab0 tx=0x7fd8e0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:44.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.742+0000 7fd8e9ffb640 1 -- 172.21.15.76:0/1542353146 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00043d0 con 0x7fd8ec104ac0 2024-03-23T15:42:44.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.742+0000 7fd8e9ffb640 1 -- 172.21.15.76:0/1542353146 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e00052d0 con 0x7fd8ec104ac0 2024-03-23T15:42:44.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.742+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec19f8f0 con 0x7fd8ec104ac0 2024-03-23T15:42:44.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.742+0000 7fd8e9ffb640 1 -- 172.21.15.76:0/1542353146 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0003d40 con 0x7fd8ec104ac0 2024-03-23T15:42:44.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.742+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec19fe50 con 0x7fd8ec104ac0 2024-03-23T15:42:44.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.743+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec0fd690 con 0x7fd8ec104ac0 2024-03-23T15:42:44.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.744+0000 7fd8e9ffb640 1 -- 172.21.15.76:0/1542353146 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fd8e0005440 con 0x7fd8ec104ac0 2024-03-23T15:42:44.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.745+0000 7fd8e9ffb640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd8d40a21d0 0x7fd8d40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:44.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.745+0000 7fd8f0fa3640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd8d40a21d0 0x7fd8d40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:44.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.745+0000 7fd8e9ffb640 1 -- 172.21.15.76:0/1542353146 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd8e00efde0 con 0x7fd8ec104ac0 2024-03-23T15:42:44.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.745+0000 7fd8f0fa3640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd8d40a21d0 0x7fd8d40a4690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009bd0 tx=0x7fd8dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:44.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.749+0000 7fd8e9ffb640 1 -- 172.21.15.76:0/1542353146 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd8e00b8d40 con 0x7fd8ec104ac0 2024-03-23T15:42:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:44 smithi183 ceph-mon[36296]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:44 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3016248249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:42:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:44 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/4116146451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:44.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:44 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2973570046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.923+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd8ec10d2b0 con 0x7fd8ec104ac0 2024-03-23T15:42:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.923+0000 7fd8e9ffb640 1 -- 172.21.15.76:0/1542353146 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd8e00bc400 con 0x7fd8ec104ac0 2024-03-23T15:42:44.927 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:42:44.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd8d40a21d0 msgr2=0x7fd8d40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd8d40a21d0 0x7fd8d40a4690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009bd0 tx=0x7fd8dc005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 msgr2=0x7fd8ec197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec197de0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0005ab0 tx=0x7fd8e0005e50 comp rx=0 tx=0).stop 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 shutdown_connections 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ec1073f0 0x7fd8ec19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec104ac0 0x7fd8ec197de0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd8d40a21d0 0x7fd8d40a4690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 --2- 172.21.15.76:0/1542353146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec102190 0x7fd8ec1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 >> 172.21.15.76:0/1542353146 conn(0x7fd8ec0fbba0 msgr2=0x7fd8ec102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 shutdown_connections 2024-03-23T15:42:44.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:44.925+0000 7fd8f3207640 1 -- 172.21.15.76:0/1542353146 wait complete. 2024-03-23T15:42:44.938 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:44 smithi076 ceph-mon[32176]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:44.938 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3016248249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:42:44.938 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/4116146451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:44.939 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:44 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2973570046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:44.939 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.042+0000 7f78c3e10640 1 -- 172.21.15.76:0/3774367703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc100610 msgr2=0x7f78bc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.042+0000 7f78c3e10640 1 --2- 172.21.15.76:0/3774367703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc100610 0x7f78bc100a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009be0 tx=0x7f78b802bef0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.042+0000 7f78c3e10640 1 -- 172.21.15.76:0/3774367703 shutdown_connections 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.042+0000 7f78c3e10640 1 --2- 172.21.15.76:0/3774367703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc1021c0 0x7f78bc108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.042+0000 7f78c3e10640 1 --2- 172.21.15.76:0/3774367703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78bc101800 0x7f78bc101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.042+0000 7f78c3e10640 1 --2- 172.21.15.76:0/3774367703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc100610 0x7f78bc100a30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.042+0000 7f78c3e10640 1 -- 172.21.15.76:0/3774367703 >> 172.21.15.76:0/3774367703 conn(0x7f78bc0fbb80 msgr2=0x7f78bc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:45.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 -- 172.21.15.76:0/3774367703 shutdown_connections 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 -- 172.21.15.76:0/3774367703 wait complete. 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 Processor -- start 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 -- start start 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc100610 0x7f78bc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc101800 0x7f78bc197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78bc1021c0 0x7f78bc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc19f2f0 con 0x7f78bc100610 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc19f460 con 0x7f78bc101800 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c3e10640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc19f620 con 0x7f78bc1021c0 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c1bac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc100610 0x7f78bc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c1bac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc100610 0x7f78bc197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51082/0 (socket says 172.21.15.76:51082) 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c1bac640 1 -- 172.21.15.76:0/2764397068 learned_addr learned my addr 172.21.15.76:0/2764397068 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:45.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c13ab640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc101800 0x7f78bc197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.043+0000 7f78c23ad640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78bc1021c0 0x7f78bc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c1bac640 1 -- 172.21.15.76:0/2764397068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78bc1021c0 msgr2=0x7f78bc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c1bac640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78bc1021c0 0x7f78bc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c1bac640 1 -- 172.21.15.76:0/2764397068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc101800 msgr2=0x7f78bc197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c1bac640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc101800 0x7f78bc197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c1bac640 1 -- 172.21.15.76:0/2764397068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b4009560 con 0x7f78bc100610 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c13ab640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc101800 0x7f78bc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c1bac640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc100610 0x7f78bc197800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f78b8000f80 tx=0x7f78b8033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78b2ffd640 1 -- 172.21.15.76:0/2764397068 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8003b70 con 0x7f78bc100610 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b8009840 con 0x7f78bc100610 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78b2ffd640 1 -- 172.21.15.76:0/2764397068 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b8003cd0 con 0x7f78bc100610 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78b2ffd640 1 -- 172.21.15.76:0/2764397068 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8034850 con 0x7f78bc100610 2024-03-23T15:42:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.044+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc19fc10 con 0x7f78bc100610 2024-03-23T15:42:45.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.045+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc10bff0 con 0x7f78bc100610 2024-03-23T15:42:45.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.046+0000 7f78b2ffd640 1 -- 172.21.15.76:0/2764397068 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f78b80349d0 con 0x7f78bc100610 2024-03-23T15:42:45.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.046+0000 7f78b2ffd640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a80a2150 0x7f78a80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.046+0000 7f78c13ab640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a80a2150 0x7f78a80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.047+0000 7f78b2ffd640 1 -- 172.21.15.76:0/2764397068 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f78b80f0420 con 0x7f78bc100610 2024-03-23T15:42:45.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.047+0000 7f78c13ab640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a80a2150 0x7f78a80a4610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78bc198d10 tx=0x7f78ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:45.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.051+0000 7f78b2ffd640 1 -- 172.21.15.76:0/2764397068 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f78b80b9430 con 0x7f78bc100610 2024-03-23T15:42:45.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:44 smithi192 ceph-mon[36653]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:42:45.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:44 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3016248249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:42:45.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:44 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/4116146451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:45.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:44 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2973570046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:45.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.223+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78bc19ffb0 con 0x7f78bc100610 2024-03-23T15:42:45.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.224+0000 7f78b2ffd640 1 -- 172.21.15.76:0/2764397068 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f78b80bcaf0 con 0x7f78bc100610 2024-03-23T15:42:45.227 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:42:45.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a80a2150 msgr2=0x7f78a80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a80a2150 0x7f78a80a4610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78bc198d10 tx=0x7f78ac005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc100610 msgr2=0x7f78bc197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc100610 0x7f78bc197800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f78b8000f80 tx=0x7f78b8033ad0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 shutdown_connections 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78bc1021c0 0x7f78bc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f78a80a2150 0x7f78a80a4610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78bc101800 0x7f78bc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 --2- 172.21.15.76:0/2764397068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78bc100610 0x7f78bc197800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 >> 172.21.15.76:0/2764397068 conn(0x7f78bc0fbb80 msgr2=0x7f78bc068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 shutdown_connections 2024-03-23T15:42:45.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.226+0000 7f78c3e10640 1 -- 172.21.15.76:0/2764397068 wait complete. 2024-03-23T15:42:45.240 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.339+0000 7f0a78a0b640 1 -- 172.21.15.76:0/1129313381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74103300 msgr2=0x7f0a74109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.339+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/1129313381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74103300 0x7f0a74109b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009860 tx=0x7f0a6c02bb70 comp rx=0 tx=0).stop 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.339+0000 7f0a78a0b640 1 -- 172.21.15.76:0/1129313381 shutdown_connections 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.339+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/1129313381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74103300 0x7f0a74109b90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.339+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/1129313381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a74102940 0x7f0a74102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/1129313381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a74101750 0x7f0a74101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 -- 172.21.15.76:0/1129313381 >> 172.21.15.76:0/1129313381 conn(0x7f0a740fccc0 msgr2=0x7f0a740ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 -- 172.21.15.76:0/1129313381 shutdown_connections 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 -- 172.21.15.76:0/1129313381 wait complete. 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 Processor -- start 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 -- start start 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74101750 0x7f0a741977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a74102940 0x7f0a74197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a74103300 0x7f0a7419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7419f290 con 0x7f0a74101750 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7419f400 con 0x7f0a74102940 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.340+0000 7f0a78a0b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7419f5c0 con 0x7f0a74103300 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74101750 0x7f0a741977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74101750 0x7f0a741977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51114/0 (socket says 172.21.15.76:51114) 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 -- 172.21.15.76:0/3336360871 learned_addr learned my addr 172.21.15.76:0/3336360871 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a71d9b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a74102940 0x7f0a74197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a72d9d640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a74103300 0x7f0a7419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 -- 172.21.15.76:0/3336360871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a74103300 msgr2=0x7f0a7419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a74103300 0x7f0a7419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 -- 172.21.15.76:0/3336360871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a74102940 msgr2=0x7f0a74197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a74102940 0x7f0a74197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 -- 172.21.15.76:0/3336360871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a64009900 con 0x7f0a74101750 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a7259c640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74101750 0x7f0a741977a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c3f0 tx=0x7f0a64009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a637fe640 1 -- 172.21.15.76:0/3336360871 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6400b630 con 0x7f0a74101750 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6c009490 con 0x7f0a74101750 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a637fe640 1 -- 172.21.15.76:0/3336360871 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a6400bc70 con 0x7f0a74101750 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a637fe640 1 -- 172.21.15.76:0/3336360871 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64016e40 con 0x7f0a74101750 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.341+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7419fc70 con 0x7f0a74101750 2024-03-23T15:42:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.342+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a74068500 con 0x7f0a74101750 2024-03-23T15:42:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.344+0000 7f0a637fe640 1 -- 172.21.15.76:0/3336360871 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f0a6400a040 con 0x7f0a74101750 2024-03-23T15:42:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.344+0000 7f0a637fe640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a5c0a2150 0x7f0a5c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.345+0000 7f0a71d9b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a5c0a2150 0x7f0a5c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.345+0000 7f0a637fe640 1 -- 172.21.15.76:0/3336360871 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0a640ce2e0 con 0x7f0a74101750 2024-03-23T15:42:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.345+0000 7f0a71d9b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a5c0a2150 0x7f0a5c0a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a74198cb0 tx=0x7f0a68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.348+0000 7f0a637fe640 1 -- 172.21.15.76:0/3336360871 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0a640971e0 con 0x7f0a74101750 2024-03-23T15:42:45.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.589+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a74108a10 con 0x7f0a74101750 2024-03-23T15:42:45.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.589+0000 7f0a637fe640 1 -- 172.21.15.76:0/3336360871 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f0a6409a8a0 con 0x7f0a74101750 2024-03-23T15:42:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:45.593 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:41:22.694125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:42:45.593 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 21 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a5c0a2150 msgr2=0x7f0a5c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a5c0a2150 0x7f0a5c0a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0a74198cb0 tx=0x7f0a68005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74101750 msgr2=0x7f0a741977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74101750 0x7f0a741977a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c3f0 tx=0x7f0a64009f90 comp rx=0 tx=0).stop 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 shutdown_connections 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a74103300 0x7f0a7419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f0a5c0a2150 0x7f0a5c0a4610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a74102940 0x7f0a74197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 --2- 172.21.15.76:0/3336360871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a74101750 0x7f0a741977a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 >> 172.21.15.76:0/3336360871 conn(0x7f0a740fccc0 msgr2=0x7f0a74105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 shutdown_connections 2024-03-23T15:42:45.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.591+0000 7f0a78a0b640 1 -- 172.21.15.76:0/3336360871 wait complete. 2024-03-23T15:42:45.606 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- 172.21.15.76:0/2178783751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8100620 msgr2=0x7f71c8100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 --2- 172.21.15.76:0/2178783751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8100620 0x7f71c8100a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02bc30 comp rx=0 tx=0).stop 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- 172.21.15.76:0/2178783751 shutdown_connections 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 --2- 172.21.15.76:0/2178783751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71c81021d0 0x7f71c8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 --2- 172.21.15.76:0/2178783751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c8101810 0x7f71c8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 --2- 172.21.15.76:0/2178783751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8100620 0x7f71c8100a40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- 172.21.15.76:0/2178783751 >> 172.21.15.76:0/2178783751 conn(0x7f71c80fbbb0 msgr2=0x7f71c80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- 172.21.15.76:0/2178783751 shutdown_connections 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- 172.21.15.76:0/2178783751 wait complete. 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 Processor -- start 2024-03-23T15:42:45.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- start start 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71c8100620 0x7f71c806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8101810 0x7f71c819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c81021d0 0x7f71c819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71c819f370 con 0x7f71c8101810 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71c819f4e0 con 0x7f71c81021d0 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71cef0c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71c819f6a0 con 0x7f71c8100620 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.707+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8101810 0x7f71c819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8101810 0x7f71c819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51136/0 (socket says 172.21.15.76:51136) 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 -- 172.21.15.76:0/3509044373 learned_addr learned my addr 172.21.15.76:0/3509044373 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71ccca8640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71c8100620 0x7f71c806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71cd4a9640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c81021d0 0x7f71c819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 -- 172.21.15.76:0/3509044373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71c8100620 msgr2=0x7f71c806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71c8100620 0x7f71c806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 -- 172.21.15.76:0/3509044373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c81021d0 msgr2=0x7f71c819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c81021d0 0x7f71c819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 -- 172.21.15.76:0/3509044373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc009560 con 0x7f71c8101810 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c7fff640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8101810 0x7f71c819c570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71b400e9f0 tx=0x7f71b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c5ffb640 1 -- 172.21.15.76:0/3509044373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4009700 con 0x7f71c8101810 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c5ffb640 1 -- 172.21.15.76:0/3509044373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b400cd80 con 0x7f71c8101810 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71c5ffb640 1 -- 172.21.15.76:0/3509044373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4018920 con 0x7f71c8101810 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c819f920 con 0x7f71c8101810 2024-03-23T15:42:45.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.708+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c819fe80 con 0x7f71c8101810 2024-03-23T15:42:45.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.709+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c8107940 con 0x7f71c8101810 2024-03-23T15:42:45.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.710+0000 7f71c5ffb640 1 -- 172.21.15.76:0/3509044373 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f71b4018a80 con 0x7f71c8101810 2024-03-23T15:42:45.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.711+0000 7f71c5ffb640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f71b80a21d0 0x7f71b80a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:45.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.711+0000 7f71ccca8640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f71b80a21d0 0x7f71b80a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:45.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.711+0000 7f71c5ffb640 1 -- 172.21.15.76:0/3509044373 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f71b4014070 con 0x7f71c8101810 2024-03-23T15:42:45.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.711+0000 7f71ccca8640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f71b80a21d0 0x7f71b80a4690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f71bc02c240 tx=0x7f71bc02c7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:45.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.715+0000 7f71c5ffb640 1 -- 172.21.15.76:0/3509044373 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f71b4053280 con 0x7f71c8101810 2024-03-23T15:42:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:45 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3634818765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:42:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:45 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1542353146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:42:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:45 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2764397068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:42:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:45 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3336360871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:45.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.893+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71c810c2a0 con 0x7f71c8101810 2024-03-23T15:42:45.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.895+0000 7f71c5ffb640 1 -- 172.21.15.76:0/3509044373 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22729 (secure 0 0 0) 0x7f71b4097020 con 0x7f71c8101810 2024-03-23T15:42:45.898 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:45.898 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:42:16.614449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f71b80a21d0 msgr2=0x7f71b80a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f71b80a21d0 0x7f71b80a4690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f71bc02c240 tx=0x7f71bc02c7e0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8101810 msgr2=0x7f71c819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8101810 0x7f71c819c570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71b400e9f0 tx=0x7f71b400eeb0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 shutdown_connections 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f71b80a21d0 0x7f71b80a4690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c81021d0 0x7f71c819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71c8101810 0x7f71c819c570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 --2- 172.21.15.76:0/3509044373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71c8100620 0x7f71c806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 >> 172.21.15.76:0/3509044373 conn(0x7f71c80fbbb0 msgr2=0x7f71c8104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:45.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 shutdown_connections 2024-03-23T15:42:45.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:45.897+0000 7f71cef0c640 1 -- 172.21.15.76:0/3509044373 wait complete. 2024-03-23T15:42:45.910 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:45 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3634818765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:42:45.910 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:45 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1542353146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:42:45.910 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:45 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2764397068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:42:45.910 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:45 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3336360871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:45.911 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:42:46.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.010+0000 7f76ac688640 1 -- 172.21.15.76:0/1875439519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 msgr2=0x7f76a4102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.010+0000 7f76ac688640 1 --2- 172.21.15.76:0/1875439519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4102dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7694009900 tx=0x7f769402bc30 comp rx=0 tx=0).stop 2024-03-23T15:42:46.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.010+0000 7f76ac688640 1 -- 172.21.15.76:0/1875439519 shutdown_connections 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.010+0000 7f76ac688640 1 --2- 172.21.15.76:0/1875439519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4103300 0x7f76a4109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.010+0000 7f76ac688640 1 --2- 172.21.15.76:0/1875439519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4102dc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.010+0000 7f76ac688640 1 --2- 172.21.15.76:0/1875439519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a4101750 0x7f76a4101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.010+0000 7f76ac688640 1 -- 172.21.15.76:0/1875439519 >> 172.21.15.76:0/1875439519 conn(0x7f76a40fccc0 msgr2=0x7f76a40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 -- 172.21.15.76:0/1875439519 shutdown_connections 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 -- 172.21.15.76:0/1875439519 wait complete. 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 Processor -- start 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 -- start start 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a4101750 0x7f76a4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4103300 0x7f76a419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f76a419f310 con 0x7f76a4102940 2024-03-23T15:42:46.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76a419f480 con 0x7f76a4101750 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76ac688640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76a419f640 con 0x7f76a4103300 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76a9c23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76aa424640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a4101750 0x7f76a4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76a9c23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51166/0 (socket says 172.21.15.76:51166) 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76a9c23640 1 -- 172.21.15.76:0/929470942 learned_addr learned my addr 172.21.15.76:0/929470942 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.011+0000 7f76aac25640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4103300 0x7f76a419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76a9c23640 1 -- 172.21.15.76:0/929470942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4103300 msgr2=0x7f76a419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76a9c23640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4103300 0x7f76a419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76a9c23640 1 -- 172.21.15.76:0/929470942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a4101750 msgr2=0x7f76a4197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76a9c23640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a4101750 0x7f76a4197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76a9c23640 1 -- 172.21.15.76:0/929470942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0009840 con 0x7f76a4102940 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76aa424640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a4101750 0x7f76a4197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76a9c23640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4197d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7694006010 tx=0x7f7694005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f769b7fe640 1 -- 172.21.15.76:0/929470942 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76940043d0 con 0x7f76a4102940 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f769b7fe640 1 -- 172.21.15.76:0/929470942 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76940052d0 con 0x7f76a4102940 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c009490 con 0x7f76a4102940 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f769b7fe640 1 -- 172.21.15.76:0/929470942 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694003bc0 con 0x7f76a4102940 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.012+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7694009560 con 0x7f76a4102940 2024-03-23T15:42:46.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.013+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a4108a70 con 0x7f76a4102940 2024-03-23T15:42:46.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.014+0000 7f769b7fe640 1 -- 172.21.15.76:0/929470942 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f7694003d40 con 0x7f76a4102940 2024-03-23T15:42:46.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.015+0000 7f769b7fe640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76900a2150 0x7f76900a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.015+0000 7f76aa424640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76900a2150 0x7f76900a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.015+0000 7f769b7fe640 1 -- 172.21.15.76:0/929470942 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f76940eff60 con 0x7f76a4102940 2024-03-23T15:42:46.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.015+0000 7f76aa424640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76900a2150 0x7f76900a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009fd0 tx=0x7f76a0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.019+0000 7f769b7fe640 1 -- 172.21.15.76:0/929470942 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f76940b8ec0 con 0x7f76a4102940 2024-03-23T15:42:46.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:45 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3634818765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:42:46.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:45 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1542353146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:42:46.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:45 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2764397068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:42:46.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:45 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3336360871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:46.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.229+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76a410d390 con 0x7f76a4102940 2024-03-23T15:42:46.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.230+0000 7f769b7fe640 1 -- 172.21.15.76:0/929470942 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f76940bc580 con 0x7f76a4102940 2024-03-23T15:42:46.233 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:42:46.233 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:41:22.694125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:42:46.234 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 21 2024-03-23T15:42:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76900a2150 msgr2=0x7f76900a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76900a2150 0x7f76900a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009fd0 tx=0x7f76a0009500 comp rx=0 tx=0).stop 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 msgr2=0x7f76a4197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4197d60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7694006010 tx=0x7f7694005e80 comp rx=0 tx=0).stop 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 shutdown_connections 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a4103300 0x7f76a419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76a4102940 0x7f76a4197d60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76900a2150 0x7f76900a4610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 --2- 172.21.15.76:0/929470942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76a4101750 0x7f76a4197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 >> 172.21.15.76:0/929470942 conn(0x7f76a40fccc0 msgr2=0x7f76a4105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 shutdown_connections 2024-03-23T15:42:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.232+0000 7f76ac688640 1 -- 172.21.15.76:0/929470942 wait complete. 2024-03-23T15:42:46.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T15:41:17.990893+0000', 'modified': '2024-03-23T15:41:22.694125+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24479, 'mds_1': 24463, 'mds_2': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1781188743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1781188743}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1781188743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/1421961871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 1421961871}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 1421961871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6837/26718199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 26718199}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 26718199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.183:6837/3732793280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3732793280}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3732793280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.192:6839/2120704815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 2120704815}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 2120704815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6839/1020785546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1020785546}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1020785546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24479, 24449, 24463]} 2024-03-23T15:42:46.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-23T15:42:46.247 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-23T15:42:46.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 -- 172.21.15.76:0/3172470549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 msgr2=0x7f8f54100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 --2- 172.21.15.76:0/3172470549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f54100a40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009900 tx=0x7f8f4402bc30 comp rx=0 tx=0).stop 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 -- 172.21.15.76:0/3172470549 shutdown_connections 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 --2- 172.21.15.76:0/3172470549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f541021d0 0x7f8f54108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 --2- 172.21.15.76:0/3172470549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f54101810 0x7f8f54101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 --2- 172.21.15.76:0/3172470549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f54100a40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 -- 172.21.15.76:0/3172470549 >> 172.21.15.76:0/3172470549 conn(0x7f8f540fbbb0 msgr2=0x7f8f540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 -- 172.21.15.76:0/3172470549 shutdown_connections 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.347+0000 7f8f59381640 1 -- 172.21.15.76:0/3172470549 wait complete. 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 Processor -- start 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 -- start start 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f5406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f54101810 0x7f8f5419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f541021d0 0x7f8f5419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5419f2f0 con 0x7f8f54100620 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5419f460 con 0x7f8f54101810 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5419f620 con 0x7f8f541021d0 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f5406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f5406a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51184/0 (socket says 172.21.15.76:51184) 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 -- 172.21.15.76:0/2762858769 learned_addr learned my addr 172.21.15.76:0/2762858769 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f54101810 0x7f8f5419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f537fe640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f541021d0 0x7f8f5419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 -- 172.21.15.76:0/2762858769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f541021d0 msgr2=0x7f8f5419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f541021d0 0x7f8f5419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 -- 172.21.15.76:0/2762858769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f54101810 msgr2=0x7f8f5419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f54101810 0x7f8f5419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 -- 172.21.15.76:0/2762858769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f44009560 con 0x7f8f54100620 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f5406a870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005ab0 tx=0x7f8f44005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f50ff9640 1 -- 172.21.15.76:0/2762858769 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f440043d0 con 0x7f8f54100620 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5419f8a0 con 0x7f8f54100620 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f50ff9640 1 -- 172.21.15.76:0/2762858769 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f440052d0 con 0x7f8f54100620 2024-03-23T15:42:46.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f50ff9640 1 -- 172.21.15.76:0/2762858769 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44003d40 con 0x7f8f54100620 2024-03-23T15:42:46.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.348+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5419fdb0 con 0x7f8f54100620 2024-03-23T15:42:46.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.350+0000 7f8f50ff9640 1 -- 172.21.15.76:0/2762858769 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7f8f44005440 con 0x7f8f54100620 2024-03-23T15:42:46.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 Processor -- start 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 -- start start 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f50ff9640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f400a2150 0x7f8f400a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f537fe640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f400a2150 0x7f8f400a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f50ff9640 1 -- 172.21.15.76:0/2762858769 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8f440efe60 con 0x7f8f54100620 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38b9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38ba2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558c38b0c180 con 0x558c38ba30c0 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558c38b096b0 con 0x558c38b9df50 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558c38b04e90 con 0x558c38b9d560 2024-03-23T15:42:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38ba2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f52ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38b9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38ba2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:34022/0 (socket says 172.21.15.76:34022) 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.351+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3753204213 learned_addr learned my addr 172.21.15.76:0/3753204213 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f537fe640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f400a2150 0x7f8f400a4610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c005fd0 tx=0x7f8f4c0118f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3753204213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 msgr2=0x558c38b9d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38b9d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3753204213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 msgr2=0x558c38ba5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3753204213 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c38ac1d80 con 0x558c38b9df50 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f537fe640 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38ba2a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009c90 tx=0x7f8f3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99cd2a687206324 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f497fa640 1 -- 172.21.15.76:0/3753204213 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c019630 con 0x558c38b9df50 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f497fa640 1 -- 172.21.15.76:0/3753204213 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c019c70 con 0x558c38b9df50 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.352+0000 7f8f497fa640 1 -- 172.21.15.76:0/3753204213 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00eb90 con 0x558c38b9df50 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3753204213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 msgr2=0x558c38ba2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38ba2a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009c90 tx=0x7f8f3c011810 comp rx=0 tx=0).stop 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3753204213 shutdown_connections 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38ba2a50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3753204213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38b9d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3753204213 >> 172.21.15.76:0/3753204213 conn(0x558c38b98ad0 msgr2=0x558c38b98ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3753204213 shutdown_connections 2024-03-23T15:42:46.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3753204213 wait complete. 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- start start 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 Processor -- start 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38ba7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38ba30c0 0x558c38baeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558c38aa9d50 con 0x558c38b9df50 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558c387d8310 con 0x558c38ba30c0 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558c38a8a8f0 con 0x558c38b9d560 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f5bbfd740 10 client.0 _ll_register_callbacks cb 0x558c38815ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.353+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38ba7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38ba7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51200/0 (socket says 172.21.15.76:51200) 2024-03-23T15:42:46.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38ba30c0 0x558c38baeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3302692332 learned_addr learned my addr 172.21.15.76:0/3302692332 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3302692332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 msgr2=0x558c38ba7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3302692332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38ba30c0 msgr2=0x558c38baeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38ba30c0 0x558c38baeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 -- 172.21.15.76:0/3302692332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c0098f0 con 0x558c38b9df50 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38ba7aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c01b000 tx=0x7f8f3c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3eb9a4e3213de00 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/3302692332 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c005450 con 0x558c38b9df50 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/3302692332 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8f3c00e460 con 0x558c38b9df50 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c388046c0 con 0x558c38b9df50 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/3302692332 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f3c0055b0 con 0x558c38b9df50 2024-03-23T15:42:46.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f5bbfd740 10 client.15078 fetch_fsmap learned FSMap version 21 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558c38b03510 con 0x558c38b9df50 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.354+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/3302692332 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558c388046c0 con 0x558c38b9df50 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/3302692332 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8f3c004ba0 con 0x558c38b9df50 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f5bbfd740 10 client.15078 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f5bbfd740 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f5bbfd740 20 client.15078 populate_metadata read hostname 'smithi076' 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38bb3cf0 0x558c38bb5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f5bbfd740 4 client.15078 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c38855270 con 0x558c38bb3cf0 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38bb3cf0 0x558c38bb5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f3a7fc640 10 client.15078 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.355+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38bb3cf0 0x558c38bb5160 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=137f6264522c3434 server_cookie=a43e99e982ddafa9 in_seq=0 out_seq=0 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/3302692332 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558c38855270 con 0x558c38bb3cf0 2024-03-23T15:42:46.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f3a7fc640 10 client.15078 handle_command_reply: tid=0 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 client.15078 shutdown 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 msgr2=0x558c38ba7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38ba7aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c01b000 tx=0x7f8f3c005850 comp rx=0 tx=0).stop 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 shutdown_connections 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38ba30c0 0x558c38baeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38bb3cf0 0x558c38bb5160 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38ba7aa0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/3302692332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f3a7fc640 0 client.15078 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 >> 172.21.15.76:0/3302692332 conn(0x558c38b98ad0 msgr2=0x558c38b97ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 shutdown_connections 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/3302692332 wait complete. 2024-03-23T15:42:46.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.356+0000 7f8f5bbfd740 20 client.15078 trim_cache size 0 max 16384 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 Processor -- start 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 -- start start 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38baeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558c38b98990 con 0x558c38ba30c0 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558c38bb3850 con 0x558c38b9df50 2024-03-23T15:42:46.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558c38bb45f0 con 0x558c38b9d560 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51216/0 (socket says 172.21.15.76:51216) 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f537fe640 1 -- 172.21.15.76:0/2358567798 learned_addr learned my addr 172.21.15.76:0/2358567798 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38baeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.374+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f537fe640 1 -- 172.21.15.76:0/2358567798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 msgr2=0x558c38ba54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f537fe640 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f537fe640 1 -- 172.21.15.76:0/2358567798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 msgr2=0x558c38baeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f537fe640 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38baeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f537fe640 1 -- 172.21.15.76:0/2358567798 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c38a9fa80 con 0x558c38ba30c0 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38baeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f537fe640 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba2890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00fb60 tx=0x7f8f4c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bee3ae69b01aa7d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f48ff9640 1 -- 172.21.15.76:0/2358567798 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c01fbb0 con 0x558c38ba30c0 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f48ff9640 1 -- 172.21.15.76:0/2358567798 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f4c01fd10 con 0x558c38ba30c0 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f48ff9640 1 -- 172.21.15.76:0/2358567798 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0058b0 con 0x558c38ba30c0 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2358567798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 msgr2=0x558c38ba2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.375+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba2890 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00fb60 tx=0x7f8f4c01c000 comp rx=0 tx=0).stop 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2358567798 shutdown_connections 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38ba30c0 0x558c38ba2890 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9df50 0x558c38baeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2358567798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38b9d560 0x558c38ba54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2358567798 >> 172.21.15.76:0/2358567798 conn(0x558c38b98ad0 msgr2=0x558c38b97ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2358567798 shutdown_connections 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2358567798 wait complete. 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- start start 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 Processor -- start 2024-03-23T15:42:46.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9d560 0x558c38baa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38baa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38ba30c0 0x558c38bad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558c38badad0 con 0x558c38b9df50 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558c38badc40 con 0x558c38b9d560 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558c38bade00 con 0x558c38ba30c0 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f5bbfd740 10 client.0 _ll_register_callbacks cb 0x7f8f3c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38baa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f52ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9d560 0x558c38baa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f4bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38baa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51228/0 (socket says 172.21.15.76:51228) 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f4bfff640 1 -- 172.21.15.76:0/2991419202 learned_addr learned my addr 172.21.15.76:0/2991419202 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.376+0000 7f8f537fe640 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38ba30c0 0x558c38bad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f4bfff640 1 -- 172.21.15.76:0/2991419202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38ba30c0 msgr2=0x558c38bad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38ba30c0 0x558c38bad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f4bfff640 1 -- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9d560 msgr2=0x558c38baa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9d560 0x558c38baa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f4bfff640 1 -- 172.21.15.76:0/2991419202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c38b03510 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9d560 0x558c38baa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f4bfff640 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38baa570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0058c0 tx=0x7f8f3c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde26025bad01b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/2991419202 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00e420 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/2991419202 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8f3c000e40 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3c019c70 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/2991419202 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f3c00e580 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 10 client.15084 fetch_fsmap learned FSMap version 21 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x7f8f4c00f860 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/2991419202 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c019c70 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/2991419202 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8f3c041020 con 0x558c38b9df50 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 10 client.15084 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:42:46.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 20 client.15084 populate_metadata read hostname 'smithi076' 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38c1dd30 0x558c38c1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 4 client.15084 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.377+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "status"}) -- 0x558c38841610 con 0x558c38c1dd30 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38c1dd30 0x558c38c1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f3a7fc640 10 client.15084 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f52ffd640 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38c1dd30 0x558c38c1ca40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e84ec8fd77b792d server_cookie=dd5d2f082c046583 in_seq=0 out_seq=0 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f3a7fc640 1 -- 172.21.15.76:0/2991419202 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558c38841610 con 0x558c38c1dd30 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f3a7fc640 10 client.15084 handle_command_reply: tid=0 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 client.15084 shutdown 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 msgr2=0x558c38baa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38baa570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0058c0 tx=0x7f8f3c009e40 comp rx=0 tx=0).stop 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 shutdown_connections 2024-03-23T15:42:46.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558c38ba30c0 0x558c38bad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x558c38c1dd30 0x558c38c1ca40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558c38b9df50 0x558c38baa570 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 --2- 172.21.15.76:0/2991419202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c38b9d560 0x558c38baa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f3a7fc640 0 client.15084 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 >> 172.21.15.76:0/2991419202 conn(0x558c38b98ad0 msgr2=0x558c38b98eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 shutdown_connections 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 1 -- 172.21.15.76:0/2991419202 wait complete. 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.378+0000 7f8f5bbfd740 20 client.15084 trim_cache size 0 max 16384 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:42:46.389 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 24449, 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 84.688047781999998, 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 21, 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 87, 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 105.449551274 2024-03-23T15:42:46.390 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:42:46.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f400a2150 msgr2=0x7f8f400a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f400a2150 0x7f8f400a4610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c005fd0 tx=0x7f8f4c0118f0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 msgr2=0x7f8f5406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f5406a870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005ab0 tx=0x7f8f44005e50 comp rx=0 tx=0).stop 2024-03-23T15:42:46.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 shutdown_connections 2024-03-23T15:42:46.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f541021d0 0x7f8f5419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f400a2150 0x7f8f400a4610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f54101810 0x7f8f5419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 --2- 172.21.15.76:0/2762858769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f54100620 0x7f8f5406a870 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 >> 172.21.15.76:0/2762858769 conn(0x7f8f540fbbb0 msgr2=0x7f8f54104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 shutdown_connections 2024-03-23T15:42:46.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.379+0000 7f8f59381640 1 -- 172.21.15.76:0/2762858769 wait complete. 2024-03-23T15:42:46.393 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.493+0000 7fc732125640 1 -- 172.21.15.76:0/2458627813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c103380 msgr2=0x7fc72c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.493+0000 7fc732125640 1 --2- 172.21.15.76:0/2458627813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c103380 0x7fc72c109c10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc724009ca0 tx=0x7fc72402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- 172.21.15.76:0/2458627813 shutdown_connections 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 --2- 172.21.15.76:0/2458627813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c103380 0x7fc72c109c10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 --2- 172.21.15.76:0/2458627813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72c1029c0 0x7fc72c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 --2- 172.21.15.76:0/2458627813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c0ff9f0 0x7fc72c0ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- 172.21.15.76:0/2458627813 >> 172.21.15.76:0/2458627813 conn(0x7fc72c0fb320 msgr2=0x7fc72c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- 172.21.15.76:0/2458627813 shutdown_connections 2024-03-23T15:42:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- 172.21.15.76:0/2458627813 wait complete. 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 Processor -- start 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- start start 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c0ff9f0 0x7fc72c197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c1029c0 0x7fc72c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72c103380 0x7fc72c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c19f220 con 0x7fc72c1029c0 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c19f390 con 0x7fc72c103380 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.494+0000 7fc732125640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c19f550 con 0x7fc72c0ff9f0 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c1029c0 0x7fc72c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c1029c0 0x7fc72c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51244/0 (socket says 172.21.15.76:51244) 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 -- 172.21.15.76:0/2206244665 learned_addr learned my addr 172.21.15.76:0/2206244665 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c0ff9f0 0x7fc72c197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc72bfff640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72c103380 0x7fc72c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 -- 172.21.15.76:0/2206244665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c0ff9f0 msgr2=0x7fc72c197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c0ff9f0 0x7fc72c197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 -- 172.21.15.76:0/2206244665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72c103380 msgr2=0x7fc72c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72c103380 0x7fc72c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 -- 172.21.15.76:0/2206244665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc724009900 con 0x7fc72c1029c0 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc722ffd640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c1029c0 0x7fc72c197c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc71400e9f0 tx=0x7fc71400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc7297fa640 1 -- 172.21.15.76:0/2206244665 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714009700 con 0x7fc72c1029c0 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c19f830 con 0x7fc72c1029c0 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc7297fa640 1 -- 172.21.15.76:0/2206244665 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71400cd80 con 0x7fc72c1029c0 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc7297fa640 1 -- 172.21.15.76:0/2206244665 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714018920 con 0x7fc72c1029c0 2024-03-23T15:42:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.495+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c19fd40 con 0x7fc72c1029c0 2024-03-23T15:42:46.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc7297fa640 1 -- 172.21.15.76:0/2206244665 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 17) ==== 136807+0+0 (secure 0 0 0) 0x7fc714018a80 con 0x7fc72c1029c0 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 Processor -- start 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 -- start start 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b2182980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2182f50 0x5555b2187a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b21880c0 0x5555b218a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555b20f1180 con 0x5555b21880c0 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555b20ee6b0 con 0x5555b2182f50 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc734de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5555b20e9e90 con 0x5555b2182560 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b21880c0 0x5555b218a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b21880c0 0x5555b218a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51246/0 (socket says 172.21.15.76:51246) 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc722ffd640 1 -- 172.21.15.76:0/1898902136 learned_addr learned my addr 172.21.15.76:0/1898902136 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc72b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2182f50 0x5555b2187a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.498+0000 7fc72bfff640 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b2182980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc7297fa640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7180a2150 0x7fc7180a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc722ffd640 1 -- 172.21.15.76:0/1898902136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 msgr2=0x5555b2182980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc722ffd640 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b2182980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc722ffd640 1 -- 172.21.15.76:0/1898902136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2182f50 msgr2=0x5555b2187a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc722ffd640 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2182f50 0x5555b2187a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc722ffd640 1 -- 172.21.15.76:0/1898902136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b20a6d80 con 0x5555b21880c0 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2182f50 0x5555b2187a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc72bfff640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7180a2150 0x7fc7180a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc7297fa640 1 -- 172.21.15.76:0/2206244665 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc714014070 con 0x7fc72c1029c0 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc722ffd640 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b21880c0 0x5555b218a550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc7140cba90 tx=0x7fc714024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d96a17e023a858 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc721ffb640 1 -- 172.21.15.76:0/1898902136 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71409abf0 con 0x5555b21880c0 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc721ffb640 1 -- 172.21.15.76:0/1898902136 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71409a960 con 0x5555b21880c0 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc72bfff640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7180a2150 0x7fc7180a4610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc724005e30 tx=0x7fc7240095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.499+0000 7fc721ffb640 1 -- 172.21.15.76:0/1898902136 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71409eec0 con 0x5555b21880c0 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 -- 172.21.15.76:0/1898902136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b21880c0 msgr2=0x5555b218a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b21880c0 0x5555b218a550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc7140cba90 tx=0x7fc714024000 comp rx=0 tx=0).stop 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 -- 172.21.15.76:0/1898902136 shutdown_connections 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b21880c0 0x5555b218a550 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2182f50 0x5555b2187a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 --2- 172.21.15.76:0/1898902136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b2182980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 -- 172.21.15.76:0/1898902136 >> 172.21.15.76:0/1898902136 conn(0x5555b217dad0 msgr2=0x5555b217ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 -- 172.21.15.76:0/1898902136 shutdown_connections 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 -- 172.21.15.76:0/1898902136 wait complete. 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 -- start start 2024-03-23T15:42:46.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.500+0000 7fc734de8740 1 Processor -- start 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b218c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182f50 0x5555b218ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b21880c0 0x5555b2193a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555b208ed50 con 0x5555b2182f50 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555b1dbd310 con 0x5555b21880c0 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5555b206f8f0 con 0x5555b2182560 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 10 client.0 _ll_register_callbacks cb 0x5555b1dfaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182f50 0x5555b218ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182f50 0x5555b218ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51254/0 (socket says 172.21.15.76:51254) 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 -- 172.21.15.76:0/1622391407 learned_addr learned my addr 172.21.15.76:0/1622391407 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc72bfff640 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b21880c0 0x5555b2193a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b218c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 -- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 msgr2=0x5555b218c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b218c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 -- 172.21.15.76:0/1622391407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b21880c0 msgr2=0x5555b2193a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b21880c0 0x5555b2193a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 -- 172.21.15.76:0/1622391407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c19fd40 con 0x5555b2182f50 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc722ffd640 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182f50 0x5555b218ca40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc71400e9c0 tx=0x7fc71401f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117c5a6cb037fc73 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc70affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc70affd640 1 -- 172.21.15.76:0/1622391407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71409a780 con 0x5555b2182f50 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc70affd640 1 -- 172.21.15.76:0/1622391407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc7140a3510 con 0x5555b2182f50 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555b1de96c0 con 0x5555b2182f50 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc70affd640 1 -- 172.21.15.76:0/1622391407 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc71409eec0 con 0x5555b2182f50 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 10 client.15096 fetch_fsmap learned FSMap version 21 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.501+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5555b20e8510 con 0x5555b2182f50 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc70affd640 1 -- 172.21.15.76:0/1622391407 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7140b2080 con 0x5555b2182f50 2024-03-23T15:42:46.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc70affd640 1 -- 172.21.15.76:0/1622391407 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc7140d1680 con 0x5555b2182f50 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc734de8740 10 client.15096 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc734de8740 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc734de8740 20 client.15096 populate_metadata read hostname 'smithi076' 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc734de8740 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2198a00 0x5555b2199e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc734de8740 4 client.15096 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555b1e3a270 con 0x5555b2198a00 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2198a00 0x5555b2199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc70affd640 10 client.15096 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.502+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2198a00 0x5555b2199e70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a1d842b272ce0b4 server_cookie=ec76babb1be36fbe in_seq=0 out_seq=0 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc70affd640 1 -- 172.21.15.76:0/1622391407 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5555b1e3a270 con 0x5555b2198a00 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc70affd640 10 client.15096 handle_command_reply: tid=0 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 client.15096 shutdown 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182f50 msgr2=0x5555b218ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182f50 0x5555b218ca40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc71400e9c0 tx=0x7fc71401f330 comp rx=0 tx=0).stop 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 shutdown_connections 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2198a00 0x5555b2199e70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b21880c0 0x5555b2193a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182f50 0x5555b218ca40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc70affd640 0 client.15096 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.503+0000 7fc734de8740 1 --2- 172.21.15.76:0/1622391407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b218c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.504+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 >> 172.21.15.76:0/1622391407 conn(0x5555b217dad0 msgr2=0x5555b217cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.504+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 shutdown_connections 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.504+0000 7fc734de8740 1 -- 172.21.15.76:0/1622391407 wait complete. 2024-03-23T15:42:46.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.504+0000 7fc734de8740 20 client.15096 trim_cache size 0 max 16384 2024-03-23T15:42:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.521+0000 7fc734de8740 1 Processor -- start 2024-03-23T15:42:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.521+0000 7fc734de8740 1 -- start start 2024-03-23T15:42:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.521+0000 7fc734de8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2187bc0 0x5555b217c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.521+0000 7fc734de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b217dad0 0x5555b2193a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.521+0000 7fc734de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b217c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.521+0000 7fc734de8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555b2194340 con 0x5555b2187bc0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.521+0000 7fc734de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555b217e520 con 0x5555b217dad0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc734de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5555b2187a80 con 0x5555b2182560 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2187bc0 0x5555b217c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2187bc0 0x5555b217c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51262/0 (socket says 172.21.15.76:51262) 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 -- 172.21.15.76:0/647418318 learned_addr learned my addr 172.21.15.76:0/647418318 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc722ffd640 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b217dad0 0x5555b2193a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72bfff640 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b217c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 -- 172.21.15.76:0/647418318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 msgr2=0x5555b217c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b217c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 -- 172.21.15.76:0/647418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b217dad0 msgr2=0x5555b2193a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b217dad0 0x5555b2193a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 -- 172.21.15.76:0/647418318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b2084a80 con 0x5555b2187bc0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc722ffd640 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b217dad0 0x5555b2193a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2187bc0 0x5555b217c450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009d10 tx=0x7fc71c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3996743ca321903 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc7217fa640 1 -- 172.21.15.76:0/647418318 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c017030 con 0x5555b2187bc0 2024-03-23T15:42:46.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc7217fa640 1 -- 172.21.15.76:0/647418318 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c018040 con 0x5555b2187bc0 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.522+0000 7fc7217fa640 1 -- 172.21.15.76:0/647418318 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c010a60 con 0x5555b2187bc0 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- 172.21.15.76:0/647418318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2187bc0 msgr2=0x5555b217c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2187bc0 0x5555b217c450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009d10 tx=0x7fc71c005f90 comp rx=0 tx=0).stop 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- 172.21.15.76:0/647418318 shutdown_connections 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b2182560 0x5555b217c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b217dad0 0x5555b2193a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 --2- 172.21.15.76:0/647418318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2187bc0 0x5555b217c450 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- 172.21.15.76:0/647418318 >> 172.21.15.76:0/647418318 conn(0x5555b2178b80 msgr2=0x5555b2184230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- 172.21.15.76:0/647418318 shutdown_connections 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- 172.21.15.76:0/647418318 wait complete. 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- start start 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 Processor -- start 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b217dad0 0x5555b218f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182560 0x5555b218fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2187bc0 0x5555b2192da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5555b2197e10 con 0x5555b2182560 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555b21932e0 con 0x5555b2187bc0 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5555b21934a0 con 0x5555b217dad0 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.523+0000 7fc734de8740 10 client.0 _ll_register_callbacks cb 0x7fc71c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182560 0x5555b218fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182560 0x5555b218fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51270/0 (socket says 172.21.15.76:51270) 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 -- 172.21.15.76:0/2028271027 learned_addr learned my addr 172.21.15.76:0/2028271027 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc72bfff640 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2187bc0 0x5555b2192da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b217dad0 0x5555b218f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 -- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b217dad0 msgr2=0x5555b218f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b217dad0 0x5555b218f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 -- 172.21.15.76:0/2028271027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2187bc0 msgr2=0x5555b2192da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2187bc0 0x5555b2192da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 -- 172.21.15.76:0/2028271027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c004040 con 0x5555b2182560 2024-03-23T15:42:46.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc722ffd640 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182560 0x5555b218fd80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc71409adb0 tx=0x7fc714096de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c899aa429ea4692 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc70affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc70affd640 1 -- 172.21.15.76:0/2028271027 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71409a780 con 0x5555b2182560 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc70affd640 1 -- 172.21.15.76:0/2028271027 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc7140a3510 con 0x5555b2182560 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc71409a960 con 0x5555b2182560 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc70affd640 1 -- 172.21.15.76:0/2028271027 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc714096c60 con 0x5555b2182560 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc734de8740 10 client.15102 fetch_fsmap learned FSMap version 21 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5555b20aa7e0 con 0x5555b2182560 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc70affd640 1 -- 172.21.15.76:0/2028271027 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7140a7020 con 0x5555b2182560 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc70affd640 1 -- 172.21.15.76:0/2028271027 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc71409a270 con 0x5555b2182560 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.524+0000 7fc734de8740 10 client.15102 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 20 client.15102 populate_metadata read hostname 'smithi076' 2024-03-23T15:42:46.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2182f50 0x5555b2201b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 4 client.15102 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "status"}) -- 0x5555b1e26610 con 0x5555b2182f50 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2182f50 0x5555b2201b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc72b7fe640 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2182f50 0x5555b2201b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=534ae442e675a067 server_cookie=1865804cfdfed7ef in_seq=0 out_seq=0 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc70affd640 10 client.15102 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc70affd640 1 -- 172.21.15.76:0/2028271027 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5555b1e26610 con 0x5555b2182f50 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc70affd640 10 client.15102 handle_command_reply: tid=0 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 client.15102 shutdown 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182560 msgr2=0x5555b218fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182560 0x5555b218fd80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc71409adb0 tx=0x7fc714096de0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 shutdown_connections 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5555b2182f50 0x5555b2201b20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555b2187bc0 0x5555b2192da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5555b2182560 0x5555b218fd80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc70affd640 0 client.15102 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:42:46.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 --2- 172.21.15.76:0/2028271027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5555b217dad0 0x5555b218f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 >> 172.21.15.76:0/2028271027 conn(0x5555b2178b80 msgr2=0x5555b2180d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.525+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 shutdown_connections 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc734de8740 1 -- 172.21.15.76:0/2028271027 wait complete. 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc734de8740 20 client.15102 trim_cache size 0 max 16384 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 24463, 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-23T15:42:46.536 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 85.848418886999994, 2024-03-23T15:42:46.537 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 21, 2024-03-23T15:42:46.537 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 87, 2024-03-23T15:42:46.537 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T15:42:46.537 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 102.685360464 2024-03-23T15:42:46.537 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7180a2150 msgr2=0x7fc7180a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7180a2150 0x7fc7180a4610 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc724005e30 tx=0x7fc7240095c0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c1029c0 msgr2=0x7fc72c197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c1029c0 0x7fc72c197c70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc71400e9f0 tx=0x7fc71400eeb0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 shutdown_connections 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc7180a2150 0x7fc7180a4610 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc72c103380 0x7fc72c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc72c1029c0 0x7fc72c197c70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 --2- 172.21.15.76:0/2206244665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc72c0ff9f0 0x7fc72c197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 >> 172.21.15.76:0/2206244665 conn(0x7fc72c0fb320 msgr2=0x7fc72c105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 shutdown_connections 2024-03-23T15:42:46.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.526+0000 7fc732125640 1 -- 172.21.15.76:0/2206244665 wait complete. 2024-03-23T15:42:46.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T15:42:46.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 -- 172.21.15.76:0/460765761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3841032d0 msgr2=0x7fa384109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 --2- 172.21.15.76:0/460765761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3841032d0 0x7fa384109b60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009ca0 tx=0x7fa37c02c0f0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 -- 172.21.15.76:0/460765761 shutdown_connections 2024-03-23T15:42:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 --2- 172.21.15.76:0/460765761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3841032d0 0x7fa384109b60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 --2- 172.21.15.76:0/460765761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384102910 0x7fa384102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 --2- 172.21.15.76:0/460765761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa384101720 0x7fa384101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 -- 172.21.15.76:0/460765761 >> 172.21.15.76:0/460765761 conn(0x7fa3840fccb0 msgr2=0x7fa3840ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 -- 172.21.15.76:0/460765761 shutdown_connections 2024-03-23T15:42:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 -- 172.21.15.76:0/460765761 wait complete. 2024-03-23T15:42:46.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.645+0000 7fa38953e640 1 Processor -- start 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa38953e640 1 -- start start 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa38953e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa384101720 0x7fa3840754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa38953e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa384102910 0x7fa384073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa38953e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3841032d0 0x7fa384074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa38953e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa38419b5d0 con 0x7fa384102910 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa38953e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa384074580 con 0x7fa3841032d0 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa38953e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa384074740 con 0x7fa384101720 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa37a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa384102910 0x7fa384073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa37a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa384102910 0x7fa384073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51286/0 (socket says 172.21.15.76:51286) 2024-03-23T15:42:46.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1638986890 learned_addr learned my addr 172.21.15.76:0/1638986890 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa382ffd640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa384101720 0x7fa3840754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa3837fe640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3841032d0 0x7fa384074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1638986890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa384101720 msgr2=0x7fa3840754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa384101720 0x7fa3840754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.646+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1638986890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3841032d0 msgr2=0x7fa384074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3841032d0 0x7fa384074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1638986890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c009900 con 0x7fa384102910 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa3837fe640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3841032d0 0x7fa384074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa384102910 0x7fa384073b00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00d950 tx=0x7fa36c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa380ff9640 1 -- 172.21.15.76:0/1638986890 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c009a30 con 0x7fa384102910 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa380ff9640 1 -- 172.21.15.76:0/1638986890 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36c009b90 con 0x7fa384102910 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa380ff9640 1 -- 172.21.15.76:0/1638986890 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c004a00 con 0x7fa384102910 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3841a5fc0 con 0x7fa384102910 2024-03-23T15:42:46.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.647+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3841a6330 con 0x7fa384102910 2024-03-23T15:42:46.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.649+0000 7fa380ff9640 1 -- 172.21.15.76:0/1638986890 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fa36c009d00 con 0x7fa384102910 2024-03-23T15:42:46.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa380ff9640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3700a20c0 0x7fa3700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 Processor -- start 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa382ffd640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3700a20c0 0x7fa3700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 -- start start 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa380ff9640 1 -- 172.21.15.76:0/1638986890 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa36c0cd5f0 con 0x7fa384102910 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67a560 0x55618d67a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67af50 0x55618d67fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d682550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55618d5e9180 con 0x55618d6800c0 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55618d5e66b0 con 0x55618d67af50 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.650+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55618d5e1e90 con 0x55618d67a560 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67a560 0x55618d67a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d682550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d682550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51298/0 (socket says 172.21.15.76:51298) 2024-03-23T15:42:46.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa3837fe640 1 -- 172.21.15.76:0/2863795899 learned_addr learned my addr 172.21.15.76:0/2863795899 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67a560 0x55618d67a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:45962/0 (socket says 172.21.15.76:45962) 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67af50 0x55618d67fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3700a20c0 0x7fa3700a4580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa374005fd0 tx=0x7fa37400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 -- 172.21.15.76:0/2863795899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67af50 msgr2=0x55618d67fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67af50 0x55618d67fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 -- 172.21.15.76:0/2863795899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 msgr2=0x55618d682550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d682550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 -- 172.21.15.76:0/2863795899 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618d59ed80 con 0x55618d67a560 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa3837fe640 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d682550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa382ffd640 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67a560 0x55618d67a980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa37400ba00 tx=0x7fa374013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c655f0cecf14a31f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa3797fa640 1 -- 172.21.15.76:0/2863795899 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37401e620 con 0x55618d67a560 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa3797fa640 1 -- 172.21.15.76:0/2863795899 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa374015070 con 0x55618d67a560 2024-03-23T15:42:46.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.651+0000 7fa3797fa640 1 -- 172.21.15.76:0/2863795899 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37401db60 con 0x55618d67a560 2024-03-23T15:42:46.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2863795899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67a560 msgr2=0x55618d67a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67a560 0x55618d67a980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa37400ba00 tx=0x7fa374013000 comp rx=0 tx=0).stop 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2863795899 shutdown_connections 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d682550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67af50 0x55618d67fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2863795899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67a560 0x55618d67a980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2863795899 >> 172.21.15.76:0/2863795899 conn(0x55618d675ad0 msgr2=0x55618d675ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2863795899 shutdown_connections 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.652+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2863795899 wait complete. 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 -- start start 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 Processor -- start 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d684540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67af50 0x55618d684a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d6800c0 0x55618d68bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55618d586d50 con 0x55618d67af50 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55618d2b5310 con 0x55618d67a560 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55618d5678f0 con 0x55618d6800c0 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa38c1fc740 10 client.0 _ll_register_callbacks cb 0x55618d2f2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67af50 0x55618d684a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67af50 0x55618d684a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51310/0 (socket says 172.21.15.76:51310) 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1450566320 learned_addr learned my addr 172.21.15.76:0/1450566320 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa3837fe640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d6800c0 0x55618d68bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa382ffd640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d684540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1450566320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d6800c0 msgr2=0x55618d68bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d6800c0 0x55618d68bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 msgr2=0x55618d684540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d684540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.653+0000 7fa37a7fc640 1 -- 172.21.15.76:0/1450566320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa374010c00 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa382ffd640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d684540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67af50 0x55618d684a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa36c09a290 tx=0x7fa36c09a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a888507e78be5f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa36a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa36a7fc640 1 -- 172.21.15.76:0/1450566320 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c005680 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa36a7fc640 1 -- 172.21.15.76:0/1450566320 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa36c096970 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55618d2e16c0 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa36a7fc640 1 -- 172.21.15.76:0/1450566320 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa36c096b30 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa38c1fc740 10 client.15114 fetch_fsmap learned FSMap version 21 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55618d5e0510 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa36a7fc640 1 -- 172.21.15.76:0/1450566320 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55618d2e16c0 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa36a7fc640 1 -- 172.21.15.76:0/1450566320 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa36c0961d0 con 0x55618d67af50 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.654+0000 7fa38c1fc740 10 client.15114 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa38c1fc740 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T15:42:46.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa38c1fc740 20 client.15114 populate_metadata read hostname 'smithi076' 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d690de0 0x55618d692250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa38c1fc740 4 client.15114 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618d332270 con 0x55618d690de0 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa382ffd640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d690de0 0x55618d692250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa36a7fc640 10 client.15114 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.655+0000 7fa382ffd640 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d690de0 0x55618d692250 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b4cc9612f59a08 server_cookie=361259bd853aa40a in_seq=0 out_seq=0 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa36a7fc640 1 -- 172.21.15.76:0/1450566320 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55618d332270 con 0x55618d690de0 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa36a7fc640 10 client.15114 handle_command_reply: tid=0 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 client.15114 shutdown 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67af50 msgr2=0x55618d684a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67af50 0x55618d684a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa36c09a290 tx=0x7fa36c09a3c0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 shutdown_connections 2024-03-23T15:42:46.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d6800c0 0x55618d68bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67af50 0x55618d684a80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d690de0 0x55618d692250 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/1450566320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d684540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa36a7fc640 0 client.15114 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 >> 172.21.15.76:0/1450566320 conn(0x55618d675ad0 msgr2=0x55618d674ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 shutdown_connections 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 1 -- 172.21.15.76:0/1450566320 wait complete. 2024-03-23T15:42:46.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.656+0000 7fa38c1fc740 20 client.15114 trim_cache size 0 max 16384 2024-03-23T15:42:46.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.674+0000 7fa38c1fc740 1 Processor -- start 2024-03-23T15:42:46.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa38c1fc740 1 -- start start 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67a560 0x55618d670810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d68fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d6800c0 0x55618d675eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55618d690940 con 0x55618d67a560 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55618d67ba10 con 0x55618d6800c0 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55618d68c380 con 0x55618d67af50 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67a560 0x55618d670810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67a560 0x55618d670810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51312/0 (socket says 172.21.15.76:51312) 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 -- 172.21.15.76:0/2987654248 learned_addr learned my addr 172.21.15.76:0/2987654248 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d68fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa3837fe640 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d6800c0 0x55618d675eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 -- 172.21.15.76:0/2987654248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 msgr2=0x55618d68fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d68fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 -- 172.21.15.76:0/2987654248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d6800c0 msgr2=0x55618d675eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d6800c0 0x55618d675eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 -- 172.21.15.76:0/2987654248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618d57ca80 con 0x55618d67a560 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa3837fe640 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d6800c0 0x55618d675eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.675+0000 7fa382ffd640 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67a560 0x55618d670810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa374010ef0 tx=0x7fa374016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c31ea093131e86a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa378ff9640 1 -- 172.21.15.76:0/2987654248 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37400b870 con 0x55618d67a560 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa378ff9640 1 -- 172.21.15.76:0/2987654248 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37402d040 con 0x55618d67a560 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa378ff9640 1 -- 172.21.15.76:0/2987654248 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37401e680 con 0x55618d67a560 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2987654248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67a560 msgr2=0x55618d670810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67a560 0x55618d670810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa374010ef0 tx=0x7fa374016000 comp rx=0 tx=0).stop 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2987654248 shutdown_connections 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d6800c0 0x55618d675eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d68fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/2987654248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d67a560 0x55618d670810 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2987654248 >> 172.21.15.76:0/2987654248 conn(0x55618d675ad0 msgr2=0x55618d6795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2987654248 shutdown_connections 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.676+0000 7fa38c1fc740 1 -- 172.21.15.76:0/2987654248 wait complete. 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 -- start start 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 Processor -- start 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d687110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d687650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d68a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55618d68abb0 con 0x55618d6800c0 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55618d68ad20 con 0x55618d67a560 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55618d68aee0 con 0x55618d67af50 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa38c1fc740 10 client.0 _ll_register_callbacks cb 0x7fa36c099ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d68a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa37a7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d687650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d68a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51320/0 (socket says 172.21.15.76:51320) 2024-03-23T15:42:46.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 -- 172.21.15.76:0/3416540347 learned_addr learned my addr 172.21.15.76:0/3416540347 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa382ffd640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d687110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 -- 172.21.15.76:0/3416540347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 msgr2=0x55618d687650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d687650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 -- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 msgr2=0x55618d687110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d687110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa3837fe640 1 -- 172.21.15.76:0/3416540347 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618d59ed80 con 0x55618d6800c0 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa37a7fc640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d687650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.677+0000 7fa382ffd640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d687110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa3837fe640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d68a670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0098d0 tx=0x7fa37c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfeeb05b3f722da server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa36a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa36a7fc640 1 -- 172.21.15.76:0/3416540347 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c045040 con 0x55618d6800c0 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa36a7fc640 1 -- 172.21.15.76:0/3416540347 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa37c003b70 con 0x55618d6800c0 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa374015070 con 0x55618d6800c0 2024-03-23T15:42:46.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa36a7fc640 1 -- 172.21.15.76:0/3416540347 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa37c003d30 con 0x55618d6800c0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 10 client.15120 fetch_fsmap learned FSMap version 21 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55618d5a27e0 con 0x55618d6800c0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa36a7fc640 1 -- 172.21.15.76:0/3416540347 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c05a080 con 0x55618d6800c0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa36a7fc640 1 -- 172.21.15.76:0/3416540347 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa37c054030 con 0x55618d6800c0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 10 client.15120 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 20 client.15120 populate_metadata read hostname 'smithi076' 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d6fb5a0 0x55618d6fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 4 client.15120 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "status"}) -- 0x55618d31e610 con 0x55618d6fb5a0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.678+0000 7fa382ffd640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d6fb5a0 0x55618d6fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa36a7fc640 10 client.15120 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa382ffd640 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d6fb5a0 0x55618d6fa2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac17f19bff9689d server_cookie=29e2eb0939de39dd in_seq=0 out_seq=0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa36a7fc640 1 -- 172.21.15.76:0/3416540347 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55618d31e610 con 0x55618d6fb5a0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa36a7fc640 10 client.15120 handle_command_reply: tid=0 2024-03-23T15:42:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 client.15120 shutdown 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 msgr2=0x55618d68a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d68a670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa37c0098d0 tx=0x7fa37c009540 comp rx=0 tx=0).stop 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 shutdown_connections 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55618d6800c0 0x55618d68a670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55618d67af50 0x55618d687650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55618d6fb5a0 0x55618d6fa2b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 --2- 172.21.15.76:0/3416540347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618d67a560 0x55618d687110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa36a7fc640 0 client.15120 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 >> 172.21.15.76:0/3416540347 conn(0x55618d675ad0 msgr2=0x55618d6795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 shutdown_connections 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 1 -- 172.21.15.76:0/3416540347 wait complete. 2024-03-23T15:42:46.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.679+0000 7fa38c1fc740 20 client.15120 trim_cache size 0 max 16384 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 24479, 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 88.028374377000006, 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 21, 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 87, 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 94.212448550999994 2024-03-23T15:42:46.690 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:42:46.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3700a20c0 msgr2=0x7fa3700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3700a20c0 0x7fa3700a4580 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa374005fd0 tx=0x7fa37400b390 comp rx=0 tx=0).stop 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa384102910 msgr2=0x7fa384073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa384102910 0x7fa384073b00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00d950 tx=0x7fa36c00de10 comp rx=0 tx=0).stop 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 shutdown_connections 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fa3700a20c0 0x7fa3700a4580 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3841032d0 0x7fa384074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa384102910 0x7fa384073b00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 --2- 172.21.15.76:0/1638986890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa384101720 0x7fa3840754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 >> 172.21.15.76:0/1638986890 conn(0x7fa3840fccb0 msgr2=0x7fa384105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 shutdown_connections 2024-03-23T15:42:46.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:46.680+0000 7fa38953e640 1 -- 172.21.15.76:0/1638986890 wait complete. 2024-03-23T15:42:46.694 INFO:tasks.cephadm:Setup complete, yielding 2024-03-23T15:42:46.694 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T15:42:46.703 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:42:46.704 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-23T15:42:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:46 smithi076 ceph-mon[32176]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:46 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3509044373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:46.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:46 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/929470942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:46 smithi183 ceph-mon[36296]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:46 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3509044373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:46.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:46 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/929470942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:47.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:46 smithi192 ceph-mon[36653]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:42:47.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:46 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3509044373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:42:47.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:46 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/929470942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:42:48.038 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:47 smithi076 ceph-mon[32176]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-23T15:42:48.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:47 smithi192 ceph-mon[36653]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-23T15:42:48.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:47 smithi183 ceph-mon[36296]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-23T15:42:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 -- 172.21.15.76:0/3874409120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4102320 msgr2=0x7fd4f4102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3874409120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4102320 0x7fd4f4102780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00099c0 tx=0x7fd4e0009cd0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 -- 172.21.15.76:0/3874409120 shutdown_connections 2024-03-23T15:42:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3874409120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4f4102cc0 0x7fd4f4109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3874409120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f4101120 0x7fd4f4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3874409120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4102320 0x7fd4f4102780 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 -- 172.21.15.76:0/3874409120 >> 172.21.15.76:0/3874409120 conn(0x7fd4f40fc6c0 msgr2=0x7fd4f40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 -- 172.21.15.76:0/3874409120 shutdown_connections 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.556+0000 7fd4fb215700 1 -- 172.21.15.76:0/3874409120 wait complete. 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 Processor -- start 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 -- start start 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4101120 0x7fd4f406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4f4102320 0x7fd4f419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f4102cc0 0x7fd4f419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f419fd00 con 0x7fd4f4101120 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f419fe70 con 0x7fd4f4102320 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4fb215700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41a0030 con 0x7fd4f4102cc0 2024-03-23T15:42:48.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4f8fb1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4101120 0x7fd4f406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4f8fb1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4101120 0x7fd4f406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51342/0 (socket says 172.21.15.76:51342) 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4f8fb1700 1 -- 172.21.15.76:0/3029141362 learned_addr learned my addr 172.21.15.76:0/3029141362 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4f3fff700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4f4102320 0x7fd4f419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.557+0000 7fd4f97b2700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f4102cc0 0x7fd4f419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f8fb1700 1 -- 172.21.15.76:0/3029141362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f4102cc0 msgr2=0x7fd4f419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f8fb1700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f4102cc0 0x7fd4f419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f8fb1700 1 -- 172.21.15.76:0/3029141362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4f4102320 msgr2=0x7fd4f419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f8fb1700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4f4102320 0x7fd4f419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f8fb1700 1 -- 172.21.15.76:0/3029141362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e00095f0 con 0x7fd4f4101120 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f8fb1700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4101120 0x7fd4f406b4e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800e970 tx=0x7fd4e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f1ffb700 1 -- 172.21.15.76:0/3029141362 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8009790 con 0x7fd4f4101120 2024-03-23T15:42:48.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f41a0310 con 0x7fd4f4101120 2024-03-23T15:42:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f1ffb700 1 -- 172.21.15.76:0/3029141362 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e800ccc0 con 0x7fd4f4101120 2024-03-23T15:42:48.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.558+0000 7fd4f1ffb700 1 -- 172.21.15.76:0/3029141362 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8003d80 con 0x7fd4f4101120 2024-03-23T15:42:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.559+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f41a07e0 con 0x7fd4f4101120 2024-03-23T15:42:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.560+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f41084e0 con 0x7fd4f4101120 2024-03-23T15:42:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.560+0000 7fd4f1ffb700 1 -- 172.21.15.76:0/3029141362 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fd4e800c7e0 con 0x7fd4f4101120 2024-03-23T15:42:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.561+0000 7fd4f1ffb700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4e40a2150 0x7fd4e40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.561+0000 7fd4f3fff700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4e40a2150 0x7fd4e40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.561+0000 7fd4f1ffb700 1 -- 172.21.15.76:0/3029141362 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd4e8014070 con 0x7fd4f4101120 2024-03-23T15:42:48.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.561+0000 7fd4f3fff700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4e40a2150 0x7fd4e40a4610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0003890 tx=0x7fd4e0005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:48.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.565+0000 7fd4f1ffb700 1 -- 172.21.15.76:0/3029141362 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd4e8096de0 con 0x7fd4f4101120 2024-03-23T15:42:48.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.780+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd4f402dc00 con 0x7fd4e40a2150 2024-03-23T15:42:48.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.781+0000 7fd4f1ffb700 1 -- 172.21.15.76:0/3029141362 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd4f402dc00 con 0x7fd4e40a2150 2024-03-23T15:42:48.785 INFO:teuthology.orchestra.run.smithi076.stdout:Backend: cephadm 2024-03-23T15:42:48.785 INFO:teuthology.orchestra.run.smithi076.stdout:Available: Yes 2024-03-23T15:42:48.785 INFO:teuthology.orchestra.run.smithi076.stdout:Paused: No 2024-03-23T15:42:48.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4e40a2150 msgr2=0x7fd4e40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:48.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4e40a2150 0x7fd4e40a4610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0003890 tx=0x7fd4e0005ad0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4101120 msgr2=0x7fd4f406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:48.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4101120 0x7fd4f406b4e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800e970 tx=0x7fd4e800ec80 comp rx=0 tx=0).stop 2024-03-23T15:42:48.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 shutdown_connections 2024-03-23T15:42:48.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd4e40a2150 0x7fd4e40a4610 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f4102cc0 0x7fd4f419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f4101120 0x7fd4f406b4e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 --2- 172.21.15.76:0/3029141362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4f4102320 0x7fd4f419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:48.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 >> 172.21.15.76:0/3029141362 conn(0x7fd4f40fc6c0 msgr2=0x7fd4f4105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:48.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 shutdown_connections 2024-03-23T15:42:48.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:48.784+0000 7fd4fb215700 1 -- 172.21.15.76:0/3029141362 wait complete. 2024-03-23T15:42:49.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:48 smithi192 ceph-mon[36653]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-23T15:42:49.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:48 smithi076 ceph-mon[32176]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-23T15:42:49.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:48 smithi183 ceph-mon[36296]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-23T15:42:49.530 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-23T15:42:49.772 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:50.868 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:50 smithi076 ceph-mon[32176]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:50.869 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:50 smithi076 ceph-mon[32176]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:42:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:50 smithi192 ceph-mon[36653]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:51.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:50 smithi192 ceph-mon[36653]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:42:51.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:50 smithi183 ceph-mon[36296]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:51.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:50 smithi183 ceph-mon[36296]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:42:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 -- 172.21.15.76:0/1247679329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c102d40 msgr2=0x7fe09c1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:51.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 --2- 172.21.15.76:0/1247679329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c102d40 0x7fe09c1095f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe09400b0d0 tx=0x7fe09400b3e0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 -- 172.21.15.76:0/1247679329 shutdown_connections 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 --2- 172.21.15.76:0/1247679329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c102d40 0x7fe09c1095f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 --2- 172.21.15.76:0/1247679329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe09c1011a0 0x7fe09c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 --2- 172.21.15.76:0/1247679329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09c1023a0 0x7fe09c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 -- 172.21.15.76:0/1247679329 >> 172.21.15.76:0/1247679329 conn(0x7fe09c0fc760 msgr2=0x7fe09c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 -- 172.21.15.76:0/1247679329 shutdown_connections 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.376+0000 7fe0a1404700 1 -- 172.21.15.76:0/1247679329 wait complete. 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 Processor -- start 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 -- start start 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09c1011a0 0x7fe09c1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c1023a0 0x7fe09c197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:51.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe09c102d40 0x7fe09c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c19ed80 con 0x7fe09c1023a0 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c19eec0 con 0x7fe09c1011a0 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe0a1404700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c19f050 con 0x7fe09c102d40 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe09a7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c1023a0 0x7fe09c197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe09a7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c1023a0 0x7fe09c197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51360/0 (socket says 172.21.15.76:51360) 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe09a7fc700 1 -- 172.21.15.76:0/2401556526 learned_addr learned my addr 172.21.15.76:0/2401556526 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe09affd700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09c1011a0 0x7fe09c1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.377+0000 7fe09b7fe700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe09c102d40 0x7fe09c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe09a7fc700 1 -- 172.21.15.76:0/2401556526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe09c102d40 msgr2=0x7fe09c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe09a7fc700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe09c102d40 0x7fe09c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe09a7fc700 1 -- 172.21.15.76:0/2401556526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09c1011a0 msgr2=0x7fe09c1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe09a7fc700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09c1011a0 0x7fe09c1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe09a7fc700 1 -- 172.21.15.76:0/2401556526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe094009d00 con 0x7fe09c1023a0 2024-03-23T15:42:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe09affd700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09c1011a0 0x7fe09c1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe09a7fc700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c1023a0 0x7fe09c197730 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00eae0 tx=0x7fe08c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe083fff700 1 -- 172.21.15.76:0/2401556526 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c013070 con 0x7fe09c1023a0 2024-03-23T15:42:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c06a960 con 0x7fe09c1023a0 2024-03-23T15:42:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe083fff700 1 -- 172.21.15.76:0/2401556526 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08c017410 con 0x7fe09c1023a0 2024-03-23T15:42:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe083fff700 1 -- 172.21.15.76:0/2401556526 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c009790 con 0x7fe09c1023a0 2024-03-23T15:42:51.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.378+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c06ae60 con 0x7fe09c1023a0 2024-03-23T15:42:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.379+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c068ec0 con 0x7fe09c1023a0 2024-03-23T15:42:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.380+0000 7fe083fff700 1 -- 172.21.15.76:0/2401556526 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fe08c017910 con 0x7fe09c1023a0 2024-03-23T15:42:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.381+0000 7fe083fff700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe0840a2150 0x7fe0840a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.381+0000 7fe09affd700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe0840a2150 0x7fe0840a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.381+0000 7fe083fff700 1 -- 172.21.15.76:0/2401556526 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe08c0cf3a0 con 0x7fe09c1023a0 2024-03-23T15:42:51.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.381+0000 7fe09affd700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe0840a2150 0x7fe0840a4610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe09c102200 tx=0x7fe090009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:51.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.385+0000 7fe083fff700 1 -- 172.21.15.76:0/2401556526 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe08c098350 con 0x7fe09c1023a0 2024-03-23T15:42:51.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.592+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe09c108020 con 0x7fe0840a2150 2024-03-23T15:42:51.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.605+0000 7fe083fff700 1 -- 172.21.15.76:0/2401556526 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fe09c108020 con 0x7fe0840a2150 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.a smithi076 running (102s) 17s ago 101s 17.3M - 19.0.0-1855-g21b3bd67 553463a90fd5 c4dd412540ed 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.b smithi183 running (99s) 33s ago 99s 26.0M - 19.0.0-1855-g21b3bd67 553463a90fd5 cf5580cbf4f2 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.c smithi192 running (114s) 33s ago 113s 19.2M - 19.0.0-1855-g21b3bd67 553463a90fd5 982978143faa 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.d smithi076 running (2m) 17s ago 2m 15.3M - 19.0.0-1855-g21b3bd67 553463a90fd5 8383b805fa59 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.e smithi183 running (119s) 33s ago 119s 19.2M - 19.0.0-1855-g21b3bd67 553463a90fd5 94452d8f2e84 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.f smithi192 running (116s) 33s ago 116s 16.5M - 19.0.0-1855-g21b3bd67 553463a90fd5 f049503cec02 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.g smithi076 running (105s) 17s ago 104s 16.3M - 19.0.0-1855-g21b3bd67 553463a90fd5 4776c633b345 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.h smithi183 running (111s) 33s ago 110s 18.9M - 19.0.0-1855-g21b3bd67 553463a90fd5 be9a99f2bdc1 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.i smithi192 running (108s) 33s ago 107s 17.6M - 19.0.0-1855-g21b3bd67 553463a90fd5 0bc7bf21b48a 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.j smithi076 running (2m) 17s ago 2m 14.1M - 19.0.0-1855-g21b3bd67 553463a90fd5 71b8f665b914 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.k smithi183 running (2m) 33s ago 2m 16.4M - 19.0.0-1855-g21b3bd67 553463a90fd5 db71d6822185 2024-03-23T15:42:51.609 INFO:teuthology.orchestra.run.smithi076.stdout:mds.l smithi192 running (2m) 33s ago 2m 16.3M - 19.0.0-1855-g21b3bd67 553463a90fd5 ce914a2b3593 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:mgr.x smithi076 *:9283,8765 running (9m) 17s ago 9m 513M - 19.0.0-1855-g21b3bd67 553463a90fd5 333621a8ab1b 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:mgr.y smithi183 *:8443,8765 running (7m) 33s ago 7m 452M - 19.0.0-1855-g21b3bd67 553463a90fd5 31d95d411636 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:mgr.z smithi192 *:8443,8765 running (7m) 33s ago 7m 451M - 19.0.0-1855-g21b3bd67 553463a90fd5 5d993f699ea1 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:mon.a smithi076 running (9m) 17s ago 9m 51.3M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c68c8b1c86ce 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:mon.b smithi183 running (7m) 33s ago 7m 40.8M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 c1f8e62a41f7 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:mon.c smithi192 running (7m) 33s ago 7m 39.0M 2048M 19.0.0-1855-g21b3bd67 553463a90fd5 a2d3f72e1291 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0 smithi076 running (7m) 17s ago 7m 46.2M 4272M 19.0.0-1855-g21b3bd67 553463a90fd5 649067e388d8 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1 smithi183 running (6m) 33s ago 6m 57.5M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 3a58df9a51ad 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2 smithi192 running (6m) 33s ago 6m 48.4M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 429ad4021fce 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3 smithi076 running (5m) 17s ago 5m 55.6M 4272M 19.0.0-1855-g21b3bd67 553463a90fd5 0b560c074823 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4 smithi183 running (5m) 33s ago 5m 45.5M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 93f534254e16 2024-03-23T15:42:51.610 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5 smithi192 running (5m) 33s ago 5m 55.3M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 522fadbeed6c 2024-03-23T15:42:51.611 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6 smithi076 running (4m) 17s ago 4m 50.7M 4272M 19.0.0-1855-g21b3bd67 553463a90fd5 f8ca7c6d09b3 2024-03-23T15:42:51.611 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7 smithi183 running (4m) 33s ago 4m 59.0M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 277ef7ab467f 2024-03-23T15:42:51.611 INFO:teuthology.orchestra.run.smithi076.stdout:osd.8 smithi192 running (3m) 33s ago 3m 54.2M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 6bbdaa138d90 2024-03-23T15:42:51.611 INFO:teuthology.orchestra.run.smithi076.stdout:osd.9 smithi076 running (3m) 17s ago 3m 40.4M 4272M 19.0.0-1855-g21b3bd67 553463a90fd5 497764e16f6c 2024-03-23T15:42:51.611 INFO:teuthology.orchestra.run.smithi076.stdout:osd.10 smithi183 running (2m) 33s ago 2m 55.4M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 89ab8420319b 2024-03-23T15:42:51.611 INFO:teuthology.orchestra.run.smithi076.stdout:osd.11 smithi192 running (2m) 33s ago 2m 51.5M 4270M 19.0.0-1855-g21b3bd67 553463a90fd5 5d337b0fe268 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.607+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe0840a2150 msgr2=0x7fe0840a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.607+0000 7fe0a1404700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe0840a2150 0x7fe0840a4610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe09c102200 tx=0x7fe090009260 comp rx=0 tx=0).stop 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.607+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c1023a0 msgr2=0x7fe09c197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.607+0000 7fe0a1404700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c1023a0 0x7fe09c197730 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00eae0 tx=0x7fe08c00edf0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.607+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 shutdown_connections 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.607+0000 7fe0a1404700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fe0840a2150 0x7fe0840a4610 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.607+0000 7fe0a1404700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe09c102d40 0x7fe09c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.608+0000 7fe0a1404700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe09c1011a0 0x7fe09c1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.608+0000 7fe0a1404700 1 --2- 172.21.15.76:0/2401556526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe09c1023a0 0x7fe09c197730 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:51.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.608+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 >> 172.21.15.76:0/2401556526 conn(0x7fe09c0fc760 msgr2=0x7fe09c105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:51.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.608+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 shutdown_connections 2024-03-23T15:42:51.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:51.608+0000 7fe0a1404700 1 -- 172.21.15.76:0/2401556526 wait complete. 2024-03-23T15:42:52.466 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-23T15:42:52.712 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:52.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:52 smithi076 ceph-mon[32176]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:52.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:52 smithi076 ceph-mon[32176]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:53.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:52 smithi192 ceph-mon[36653]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:53.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:52 smithi192 ceph-mon[36653]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:53.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:52 smithi183 ceph-mon[36296]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:53.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:52 smithi183 ceph-mon[36296]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 -- 172.21.15.76:0/2975686113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514100550 msgr2=0x7ff514100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 --2- 172.21.15.76:0/2975686113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514100550 0x7ff514100970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff508009990 tx=0x7ff508009ca0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 -- 172.21.15.76:0/2975686113 shutdown_connections 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 --2- 172.21.15.76:0/2975686113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff514103f70 0x7ff51410a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 --2- 172.21.15.76:0/2975686113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514100550 0x7ff514100970 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 --2- 172.21.15.76:0/2975686113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5141035d0 0x7ff514103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 -- 172.21.15.76:0/2975686113 >> 172.21.15.76:0/2975686113 conn(0x7ff5140fbe90 msgr2=0x7ff5140fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.519+0000 7ff51ab01700 1 -- 172.21.15.76:0/2975686113 shutdown_connections 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 -- 172.21.15.76:0/2975686113 wait complete. 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 Processor -- start 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 -- start start 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff514100550 0x7ff514102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:54.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5141035d0 0x7ff514101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514103f70 0x7ff514101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff514102240 con 0x7ff514103f70 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141023b0 con 0x7ff5141035d0 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51ab01700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141a2250 con 0x7ff514100550 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51909e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514103f70 0x7ff514101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51909e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514103f70 0x7ff514101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38500/0 (socket says 172.21.15.76:38500) 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51909e700 1 -- 172.21.15.76:0/3550736251 learned_addr learned my addr 172.21.15.76:0/3550736251 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff513fff700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5141035d0 0x7ff514101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.520+0000 7ff51889d700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff514100550 0x7ff514102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51909e700 1 -- 172.21.15.76:0/3550736251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff514100550 msgr2=0x7ff514102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51909e700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff514100550 0x7ff514102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51909e700 1 -- 172.21.15.76:0/3550736251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5141035d0 msgr2=0x7ff514101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51909e700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5141035d0 0x7ff514101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51909e700 1 -- 172.21.15.76:0/3550736251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5080095f0 con 0x7ff514103f70 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff513fff700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5141035d0 0x7ff514101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:54.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51909e700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514103f70 0x7ff514101ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff50c007ee0 tx=0x7ff50c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff511ffb700 1 -- 172.21.15.76:0/3550736251 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c00dd20 con 0x7ff514103f70 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5141a2530 con 0x7ff514103f70 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff511ffb700 1 -- 172.21.15.76:0/3550736251 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c00f040 con 0x7ff514103f70 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff511ffb700 1 -- 172.21.15.76:0/3550736251 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0048e0 con 0x7ff514103f70 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.521+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a2a80 con 0x7ff514103f70 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.522+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff514109790 con 0x7ff514103f70 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.523+0000 7ff511ffb700 1 -- 172.21.15.76:0/3550736251 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7ff50c003a00 con 0x7ff514103f70 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.523+0000 7ff511ffb700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff5040a2150 0x7ff5040a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.524+0000 7ff51889d700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff5040a2150 0x7ff5040a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:54.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.524+0000 7ff511ffb700 1 -- 172.21.15.76:0/3550736251 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff50c0cd4d0 con 0x7ff514103f70 2024-03-23T15:42:54.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.524+0000 7ff51889d700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff5040a2150 0x7ff5040a4610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff50800b650 tx=0x7ff508005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:54.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.528+0000 7ff511ffb700 1 -- 172.21.15.76:0/3550736251 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7ff50c096480 con 0x7ff514103f70 2024-03-23T15:42:54.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.738+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff514062640 con 0x7ff5040a2150 2024-03-23T15:42:54.742 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:54 smithi076 ceph-mon[32176]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:54.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.745+0000 7ff511ffb700 1 -- 172.21.15.76:0/3550736251 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff514062640 con 0x7ff5040a2150 2024-03-23T15:42:54.748 INFO:teuthology.orchestra.run.smithi076.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-23T15:42:54.748 INFO:teuthology.orchestra.run.smithi076.stdout:mds.all 12/12 36s ago 2m smithi076=a;smithi076=d;smithi076=g;smithi076=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-03-23T15:42:54.748 INFO:teuthology.orchestra.run.smithi076.stdout:mgr 3/3 36s ago 7m smithi076=x;smithi183=y;smithi192=z;count:3 2024-03-23T15:42:54.749 INFO:teuthology.orchestra.run.smithi076.stdout:mon 3/3 36s ago 8m smithi076:172.21.15.76=a;smithi183:172.21.15.183=b;smithi192:172.21.15.192=c;count:3 2024-03-23T15:42:54.749 INFO:teuthology.orchestra.run.smithi076.stdout:osd 12 36s ago - 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff5040a2150 msgr2=0x7ff5040a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff5040a2150 0x7ff5040a4610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff50800b650 tx=0x7ff508005fb0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514103f70 msgr2=0x7ff514101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514103f70 0x7ff514101ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff50c007ee0 tx=0x7ff50c00d3b0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 shutdown_connections 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7ff5040a2150 0x7ff5040a4610 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff514103f70 0x7ff514101ac0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff514100550 0x7ff514102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 --2- 172.21.15.76:0/3550736251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5141035d0 0x7ff514101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.747+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 >> 172.21.15.76:0/3550736251 conn(0x7ff5140fbe90 msgr2=0x7ff5141068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.748+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 shutdown_connections 2024-03-23T15:42:54.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:54.748+0000 7ff51ab01700 1 -- 172.21.15.76:0/3550736251 wait complete. 2024-03-23T15:42:55.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:54 smithi192 ceph-mon[36653]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:55.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:54 smithi183 ceph-mon[36296]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:55.512 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-23T15:42:55.754 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:56.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:56 smithi076 ceph-mon[32176]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:56.913 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:56 smithi076 ceph-mon[32176]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:56.935 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:56 smithi183 ceph-mon[36296]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:56.935 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:56 smithi183 ceph-mon[36296]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:57.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:56 smithi192 ceph-mon[36653]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:57.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:56 smithi192 ceph-mon[36653]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.495+0000 7f8f7b731700 1 -- 172.21.15.76:0/4267665237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 msgr2=0x7f8f74102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.495+0000 7f8f7b731700 1 --2- 172.21.15.76:0/4267665237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 0x7f8f74102790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f640099c0 tx=0x7f8f64009cd0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.495+0000 7f8f7b731700 1 -- 172.21.15.76:0/4267665237 shutdown_connections 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.495+0000 7f8f7b731700 1 --2- 172.21.15.76:0/4267665237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f74102cd0 0x7f8f74109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.495+0000 7f8f7b731700 1 --2- 172.21.15.76:0/4267665237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 0x7f8f74101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.495+0000 7f8f7b731700 1 --2- 172.21.15.76:0/4267665237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 0x7f8f74102790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.495+0000 7f8f7b731700 1 -- 172.21.15.76:0/4267665237 >> 172.21.15.76:0/4267665237 conn(0x7f8f740fc6f0 msgr2=0x7f8f740feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 -- 172.21.15.76:0/4267665237 shutdown_connections 2024-03-23T15:42:57.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 -- 172.21.15.76:0/4267665237 wait complete. 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 Processor -- start 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 -- start start 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 0x7f8f7419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 0x7f8f7419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f74102cd0 0x7f8f741a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741a1f60 con 0x7f8f74102330 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741a20a0 con 0x7f8f74101130 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.496+0000 7f8f7b731700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741a2230 con 0x7f8f74102cd0 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f78ccc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 0x7f8f7419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 0x7f8f7419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 0x7f8f7419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:45824/0 (socket says 172.21.15.76:45824) 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 -- 172.21.15.76:0/862228750 learned_addr learned my addr 172.21.15.76:0/862228750 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:42:57.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f79cce700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f74102cd0 0x7f8f741a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 -- 172.21.15.76:0/862228750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f74102cd0 msgr2=0x7f8f741a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f74102cd0 0x7f8f741a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 -- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 msgr2=0x7f8f7419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 0x7f8f7419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 -- 172.21.15.76:0/862228750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f640095f0 con 0x7f8f74101130 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f78ccc700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 0x7f8f7419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f794cd700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 0x7f8f7419a3d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000e970 tx=0x7f8f7000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f6a7fc700 1 -- 172.21.15.76:0/862228750 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70009790 con 0x7f8f74101130 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f6a7fc700 1 -- 172.21.15.76:0/862228750 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7000ccc0 con 0x7f8f74101130 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f741a2510 con 0x7f8f74101130 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f6a7fc700 1 -- 172.21.15.76:0/862228750 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70003d60 con 0x7f8f74101130 2024-03-23T15:42:57.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.497+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f741a2a10 con 0x7f8f74101130 2024-03-23T15:42:57.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.498+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f74068ec0 con 0x7f8f74101130 2024-03-23T15:42:57.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.500+0000 7f8f6a7fc700 1 -- 172.21.15.76:0/862228750 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f8f7000c7e0 con 0x7f8f74101130 2024-03-23T15:42:57.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.500+0000 7f8f6a7fc700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f600a2150 0x7f8f600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:42:57.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.500+0000 7f8f78ccc700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f600a2150 0x7f8f600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:42:57.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.500+0000 7f8f6a7fc700 1 -- 172.21.15.76:0/862228750 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8f70014070 con 0x7f8f74101130 2024-03-23T15:42:57.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.501+0000 7f8f78ccc700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f600a2150 0x7f8f600a4610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f64003890 tx=0x7f8f64005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:42:57.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.504+0000 7f8f6a7fc700 1 -- 172.21.15.76:0/862228750 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8f70096dd0 con 0x7f8f74101130 2024-03-23T15:42:57.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.708+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8f7406b030 con 0x7f8f600a2150 2024-03-23T15:42:57.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.709+0000 7f8f6a7fc700 1 -- 172.21.15.76:0/862228750 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8f7406b030 con 0x7f8f600a2150 2024-03-23T15:42:57.711 INFO:teuthology.orchestra.run.smithi076.stdout:HOST ADDR LABELS STATUS 2024-03-23T15:42:57.711 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 172.21.15.76 2024-03-23T15:42:57.711 INFO:teuthology.orchestra.run.smithi076.stdout:smithi183 172.21.15.183 2024-03-23T15:42:57.711 INFO:teuthology.orchestra.run.smithi076.stdout:smithi192 172.21.15.192 2024-03-23T15:42:57.712 INFO:teuthology.orchestra.run.smithi076.stdout:3 hosts in cluster 2024-03-23T15:42:57.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f600a2150 msgr2=0x7f8f600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:57.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f600a2150 0x7f8f600a4610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f64003890 tx=0x7f8f64005ad0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 msgr2=0x7f8f7419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 0x7f8f7419a3d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000e970 tx=0x7f8f7000ec80 comp rx=0 tx=0).stop 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 shutdown_connections 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8f600a2150 0x7f8f600a4610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f74102cd0 0x7f8f741a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f74101130 0x7f8f7419a3d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 --2- 172.21.15.76:0/862228750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f74102330 0x7f8f7419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 >> 172.21.15.76:0/862228750 conn(0x7f8f740fc6f0 msgr2=0x7f8f7406a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 shutdown_connections 2024-03-23T15:42:57.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:42:57.711+0000 7f8f7b731700 1 -- 172.21.15.76:0/862228750 wait complete. 2024-03-23T15:42:57.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:57 smithi076 ceph-mon[32176]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-23T15:42:58.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:57 smithi192 ceph-mon[36653]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-23T15:42:58.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:57 smithi183 ceph-mon[36296]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-23T15:42:58.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-23T15:42:58.769 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:42:58.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:58 smithi076 ceph-mon[32176]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:59.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:58 smithi192 ceph-mon[36653]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:59.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:58 smithi183 ceph-mon[36296]: from='client.24935 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:42:59.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:42:59 smithi076 ceph-mon[32176]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:43:00.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:42:59 smithi192 ceph-mon[36653]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:43:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:42:59 smithi183 ceph-mon[36296]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-23T15:43:00.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 -- 172.21.15.76:0/260802690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288103dd0 msgr2=0x7f828810a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:00.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 --2- 172.21.15.76:0/260802690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288103dd0 0x7f828810a680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f828000b0d0 tx=0x7f828000b3e0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 -- 172.21.15.76:0/260802690 shutdown_connections 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 --2- 172.21.15.76:0/260802690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288103dd0 0x7f828810a680 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 --2- 172.21.15.76:0/260802690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8288102230 0x7f8288102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 --2- 172.21.15.76:0/260802690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8288103430 0x7f8288103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 -- 172.21.15.76:0/260802690 >> 172.21.15.76:0/260802690 conn(0x7f82880fd810 msgr2=0x7f82880ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 -- 172.21.15.76:0/260802690 shutdown_connections 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.423+0000 7f828eb0f700 1 -- 172.21.15.76:0/260802690 wait complete. 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 Processor -- start 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 -- start start 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288102230 0x7f8288075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8288103430 0x7f82880745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8288103dd0 0x7f8288074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:00.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f82880752b0 con 0x7f8288102230 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8288075420 con 0x7f8288103dd0 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828eb0f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82881a2340 con 0x7f8288103430 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288102230 0x7f8288075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288102230 0x7f8288075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38540/0 (socket says 172.21.15.76:38540) 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 -- 172.21.15.76:0/2829404448 learned_addr learned my addr 172.21.15.76:0/2829404448 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f8287fff700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8288103430 0x7f82880745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828d0ac700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8288103dd0 0x7f8288074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 -- 172.21.15.76:0/2829404448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8288103430 msgr2=0x7f82880745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8288103430 0x7f82880745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 -- 172.21.15.76:0/2829404448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8288103dd0 msgr2=0x7f8288074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8288103dd0 0x7f8288074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.424+0000 7f828c8ab700 1 -- 172.21.15.76:0/2829404448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8280009d00 con 0x7f8288102230 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.425+0000 7f8287fff700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8288103430 0x7f82880745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:00.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.425+0000 7f828c8ab700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288102230 0x7f8288075fa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f827c00e970 tx=0x7f827c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:00.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.425+0000 7f8285ffb700 1 -- 172.21.15.76:0/2829404448 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c009790 con 0x7f8288102230 2024-03-23T15:43:00.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.425+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82881a2620 con 0x7f8288102230 2024-03-23T15:43:00.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.425+0000 7f8285ffb700 1 -- 172.21.15.76:0/2829404448 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827c00ccc0 con 0x7f8288102230 2024-03-23T15:43:00.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.425+0000 7f8285ffb700 1 -- 172.21.15.76:0/2829404448 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c003d80 con 0x7f8288102230 2024-03-23T15:43:00.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.425+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82881a2b70 con 0x7f8288102230 2024-03-23T15:43:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.426+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82881095f0 con 0x7f8288102230 2024-03-23T15:43:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.427+0000 7f8285ffb700 1 -- 172.21.15.76:0/2829404448 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f827c00c7e0 con 0x7f8288102230 2024-03-23T15:43:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.427+0000 7f8285ffb700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82780a2150 0x7f82780a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.428+0000 7f8287fff700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82780a2150 0x7f82780a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.428+0000 7f8285ffb700 1 -- 172.21.15.76:0/2829404448 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f827c014070 con 0x7f8288102230 2024-03-23T15:43:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.428+0000 7f8287fff700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82780a2150 0x7f82780a4610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8274011490 tx=0x7f8274009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.432+0000 7f8285ffb700 1 -- 172.21.15.76:0/2829404448 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f827c096e10 con 0x7f8288102230 2024-03-23T15:43:00.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.637+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f828802dc00 con 0x7f82780a2150 2024-03-23T15:43:00.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.639+0000 7f8285ffb700 1 -- 172.21.15.76:0/2829404448 <== mgr.14150 v2:172.21.15.76:6800/2479514030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f828802dc00 con 0x7f82780a2150 2024-03-23T15:43:00.641 INFO:teuthology.orchestra.run.smithi076.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-23T15:43:00.641 INFO:teuthology.orchestra.run.smithi076.stdout:smithi076 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400U4400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-23T15:43:00.641 INFO:teuthology.orchestra.run.smithi076.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-23T15:43:00.641 INFO:teuthology.orchestra.run.smithi076.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-23T15:43:00.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82780a2150 msgr2=0x7f82780a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:00.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82780a2150 0x7f82780a4610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8274011490 tx=0x7f8274009260 comp rx=0 tx=0).stop 2024-03-23T15:43:00.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288102230 msgr2=0x7f8288075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288102230 0x7f8288075fa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f827c00e970 tx=0x7f827c00ec80 comp rx=0 tx=0).stop 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 shutdown_connections 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f82780a2150 0x7f82780a4610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8288103dd0 0x7f8288074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8288102230 0x7f8288075fa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 --2- 172.21.15.76:0/2829404448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8288103430 0x7f82880745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 >> 172.21.15.76:0/2829404448 conn(0x7f82880fd810 msgr2=0x7f8288106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 shutdown_connections 2024-03-23T15:43:00.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:00.641+0000 7f828eb0f700 1 -- 172.21.15.76:0/2829404448 wait complete. 2024-03-23T15:43:01.065 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:00 smithi076 ceph-mon[32176]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:01.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:00 smithi192 ceph-mon[36653]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:01.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:00 smithi183 ceph-mon[36296]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:01.262 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T15:43:01.271 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:43:01.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-23T15:43:01.514 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:43:01.905 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:01 smithi076 ceph-mon[32176]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:43:02.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:01 smithi192 ceph-mon[36653]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:43:02.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:01 smithi183 ceph-mon[36296]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 -- 172.21.15.76:0/807945735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 msgr2=0x7feccc102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 --2- 172.21.15.76:0/807945735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc102650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0099c0 tx=0x7fecbc009cd0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 -- 172.21.15.76:0/807945735 shutdown_connections 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 --2- 172.21.15.76:0/807945735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feccc103dd0 0x7feccc10a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 --2- 172.21.15.76:0/807945735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc102650 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 --2- 172.21.15.76:0/807945735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc103430 0x7feccc103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 -- 172.21.15.76:0/807945735 >> 172.21.15.76:0/807945735 conn(0x7feccc0fd810 msgr2=0x7feccc0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 -- 172.21.15.76:0/807945735 shutdown_connections 2024-03-23T15:43:03.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.336+0000 7fecd3a47700 1 -- 172.21.15.76:0/807945735 wait complete. 2024-03-23T15:43:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 Processor -- start 2024-03-23T15:43:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 -- start start 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc103430 0x7feccc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feccc103dd0 0x7feccc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feccc19fd80 con 0x7feccc102230 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feccc19fec0 con 0x7feccc103430 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd3a47700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1a0050 con 0x7feccc103dd0 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd17e3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd17e3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38554/0 (socket says 172.21.15.76:38554) 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd17e3700 1 -- 172.21.15.76:0/3410008067 learned_addr learned my addr 172.21.15.76:0/3410008067 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.337+0000 7fecd0fe2700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc103430 0x7feccc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd1fe4700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feccc103dd0 0x7feccc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd17e3700 1 -- 172.21.15.76:0/3410008067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feccc103dd0 msgr2=0x7feccc19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd17e3700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feccc103dd0 0x7feccc19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd17e3700 1 -- 172.21.15.76:0/3410008067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc103430 msgr2=0x7feccc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd17e3700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc103430 0x7feccc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd17e3700 1 -- 172.21.15.76:0/3410008067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc0095f0 con 0x7feccc102230 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd0fe2700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc103430 0x7feccc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd17e3700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc1981f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009fd0 tx=0x7fecbc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecc27fc700 1 -- 172.21.15.76:0/3410008067 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc01d070 con 0x7feccc102230 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1a02d0 con 0x7feccc102230 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecc27fc700 1 -- 172.21.15.76:0/3410008067 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecbc005940 con 0x7feccc102230 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a07c0 con 0x7feccc102230 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.338+0000 7fecc27fc700 1 -- 172.21.15.76:0/3410008067 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc022e90 con 0x7feccc102230 2024-03-23T15:43:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.339+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc109670 con 0x7feccc102230 2024-03-23T15:43:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.340+0000 7fecc27fc700 1 -- 172.21.15.76:0/3410008067 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fecbc022470 con 0x7feccc102230 2024-03-23T15:43:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.341+0000 7fecc27fc700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecb80a6560 0x7fecb80a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.341+0000 7fecd0fe2700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecb80a6560 0x7fecb80a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.341+0000 7fecc27fc700 1 -- 172.21.15.76:0/3410008067 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fecbc0ceef0 con 0x7feccc102230 2024-03-23T15:43:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.341+0000 7fecd0fe2700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecb80a6560 0x7fecb80a8a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fecc8011490 tx=0x7fecc8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:03.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.344+0000 7fecc27fc700 1 -- 172.21.15.76:0/3410008067 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fecbc097f30 con 0x7feccc102230 2024-03-23T15:43:03.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.587+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7feccc068ec0 con 0x7feccc102230 2024-03-23T15:43:03.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.588+0000 7fecc27fc700 1 -- 172.21.15.76:0/3410008067 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3050 (secure 0 0 0) 0x7fecbc09b5f0 con 0x7feccc102230 2024-03-23T15:43:03.590 INFO:teuthology.orchestra.run.smithi076.stdout:e21 2024-03-23T15:43:03.590 INFO:teuthology.orchestra.run.smithi076.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T15:43:03.590 INFO:teuthology.orchestra.run.smithi076.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T15:43:03.590 INFO:teuthology.orchestra.run.smithi076.stdout:legacy client fscid: 1 2024-03-23T15:43:03.590 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:Filesystem 'cephfs' (1) 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:fs_name cephfs 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 21 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:created 2024-03-23T15:41:17.990893+0000 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:modified 2024-03-23T15:41:22.694125+0000 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:tableserver 0 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:root 0 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:session_timeout 300 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:session_autoclose 300 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:max_file_size 1099511627776 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:max_xattr_size 65536 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:required_client_features {} 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:last_failure 0 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:last_failure_osd_epoch 0 2024-03-23T15:43:03.591 INFO:teuthology.orchestra.run.smithi076.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:max_mds 3 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:in 0,1,2 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:up {0=24479,1=24463,2=24449} 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:failed 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:damaged 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:stopped 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:data_pools [3] 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:metadata_pool 2 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:inline_data disabled 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:balancer 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:bal_rank_mask -1 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:standby_count_wanted 0 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:qdb_cluster leader: 24479 members: 24479,24449,24463 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b{0:24479} state up:active seq 3 addr [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c{0:24436} state up:standby-replay seq 1 addr [v2:172.21.15.192:6836/26718199,v1:172.21.15.192:6837/26718199] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.592 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.i{1:24463} state up:active seq 6 addr [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.f{1:24424} state up:standby-replay seq 1 addr [v2:172.21.15.192:6834/1421961871,v1:172.21.15.192:6835/1421961871] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.h{2:24449} state up:active seq 7 addr [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.e{2:24416} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/1781188743,v1:172.21.15.183:6835/1781188743] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:Standby daemons: 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.76:6834/1996137428,v1:172.21.15.76:6835/1996137428] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.76:6836/3380265538,v1:172.21.15.76:6837/3380265538] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.g{-1:14640} state up:standby seq 1 addr [v2:172.21.15.76:6838/2542851051,v1:172.21.15.76:6839/2542851051] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a{-1:14652} state up:standby seq 1 addr [v2:172.21.15.76:6840/925396691,v1:172.21.15.76:6841/925396691] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.k{-1:24386} state up:standby seq 1 addr [v2:172.21.15.183:6832/2543151700,v1:172.21.15.183:6833/2543151700] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.l{-1:24391} state up:standby seq 1 addr [v2:172.21.15.192:6832/996338359,v1:172.21.15.192:6833/996338359] compat {c=[1],r=[1],i=[1fff]}] 2024-03-23T15:43:03.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecb80a6560 msgr2=0x7fecb80a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:03.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecb80a6560 0x7fecb80a8a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fecc8011490 tx=0x7fecc8009260 comp rx=0 tx=0).stop 2024-03-23T15:43:03.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 msgr2=0x7feccc1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc1981f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009fd0 tx=0x7fecbc00f690 comp rx=0 tx=0).stop 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 shutdown_connections 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fecb80a6560 0x7fecb80a8a20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feccc103dd0 0x7feccc19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feccc102230 0x7feccc1981f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 --2- 172.21.15.76:0/3410008067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc103430 0x7feccc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 >> 172.21.15.76:0/3410008067 conn(0x7feccc0fd810 msgr2=0x7feccc1067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 shutdown_connections 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:03.590+0000 7fecd3a47700 1 -- 172.21.15.76:0/3410008067 wait complete. 2024-03-23T15:43:03.595 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 21 2024-03-23T15:43:04.376 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-23T15:43:04.620 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:43:04.743 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:04 smithi076 ceph-mon[32176]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:43:04.743 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:04 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3410008067' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T15:43:04.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:04 smithi192 ceph-mon[36653]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:43:04.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:04 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3410008067' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T15:43:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:04 smithi183 ceph-mon[36296]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-23T15:43:04.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:04 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3410008067' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T15:43:06.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 -- 172.21.15.76:0/2027892739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 msgr2=0x7f00ec102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:06.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 --2- 172.21.15.76:0/2027892739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec102790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f00e00099c0 tx=0x7f00e0009cd0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 -- 172.21.15.76:0/2027892739 shutdown_connections 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 --2- 172.21.15.76:0/2027892739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ec102cd0 0x7f00ec109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 --2- 172.21.15.76:0/2027892739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec101130 0x7f00ec101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 --2- 172.21.15.76:0/2027892739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec102790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 -- 172.21.15.76:0/2027892739 >> 172.21.15.76:0/2027892739 conn(0x7f00ec0fc6d0 msgr2=0x7f00ec0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 -- 172.21.15.76:0/2027892739 shutdown_connections 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 -- 172.21.15.76:0/2027892739 wait complete. 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.441+0000 7f00f1770700 1 Processor -- start 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00f1770700 1 -- start start 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00f1770700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ec101130 0x7f00ec1050c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00f1770700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00f1770700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec102cd0 0x7f00ec103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:06.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00f1770700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1043d0 con 0x7f00ec102330 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00f1770700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec104540 con 0x7f00ec101130 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00f1770700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec1a22a0 con 0x7f00ec102cd0 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00eb7fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec102cd0 0x7f00ec103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec103710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35270/0 (socket says 172.21.15.76:35270) 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 -- 172.21.15.76:0/1874775141 learned_addr learned my addr 172.21.15.76:0/1874775141 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00eaffd700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ec101130 0x7f00ec1050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 -- 172.21.15.76:0/1874775141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec102cd0 msgr2=0x7f00ec103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec102cd0 0x7f00ec103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 -- 172.21.15.76:0/1874775141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ec101130 msgr2=0x7f00ec1050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ec101130 0x7f00ec1050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 -- 172.21.15.76:0/1874775141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00095f0 con 0x7f00ec102330 2024-03-23T15:43:06.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.442+0000 7f00ea7fc700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec103710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.443+0000 7f00d3fff700 1 -- 172.21.15.76:0/1874775141 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e001d070 con 0x7f00ec102330 2024-03-23T15:43:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.443+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec1a2520 con 0x7f00ec102330 2024-03-23T15:43:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.443+0000 7f00d3fff700 1 -- 172.21.15.76:0/1874775141 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0003680 con 0x7f00ec102330 2024-03-23T15:43:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.443+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec1a2990 con 0x7f00ec102330 2024-03-23T15:43:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.443+0000 7f00d3fff700 1 -- 172.21.15.76:0/1874775141 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0005290 con 0x7f00ec102330 2024-03-23T15:43:06.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.443+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec068ec0 con 0x7f00ec102330 2024-03-23T15:43:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.445+0000 7f00d3fff700 1 -- 172.21.15.76:0/1874775141 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f00e0004810 con 0x7f00ec102330 2024-03-23T15:43:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.445+0000 7f00d3fff700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f00d40a2150 0x7f00d40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.445+0000 7f00eaffd700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f00d40a2150 0x7f00d40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.445+0000 7f00d3fff700 1 -- 172.21.15.76:0/1874775141 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f00e00cee60 con 0x7f00ec102330 2024-03-23T15:43:06.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.446+0000 7f00eaffd700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f00d40a2150 0x7f00d40a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009550 tx=0x7f00e40091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:06.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.449+0000 7f00d3fff700 1 -- 172.21.15.76:0/1874775141 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f00e0097ea0 con 0x7f00ec102330 2024-03-23T15:43:06.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.644+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f00ec1a2c70 con 0x7f00ec102330 2024-03-23T15:43:06.647 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:06 smithi076 ceph-mon[32176]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:43:06.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.646+0000 7f00d3fff700 1 -- 172.21.15.76:0/1874775141 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5326 (secure 0 0 0) 0x7f00e0004e90 con 0x7f00ec102330 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 87 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:fsid 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:created 2024-03-23T15:33:18.792286+0000 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:modified 2024-03-23T15:42:16.614449+0000 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:crush_version 26 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:full_ratio 0.95 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:backfillfull_ratio 0.9 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:nearfull_ratio 0.85 2024-03-23T15:43:06.648 INFO:teuthology.orchestra.run.smithi076.stdout:require_min_compat_client luminous 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:min_compat_client luminous 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:require_osd_release squid 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:stretch_mode_enabled false 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:max_osd 12 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6802/4179784307,v1:172.21.15.76:6803/4179784307] [v2:172.21.15.76:6804/4179784307,v1:172.21.15.76:6805/4179784307] exists,up ec188648-1f9e-47f7-a11c-31c3080d91dd 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/103132661,v1:172.21.15.183:6801/103132661] [v2:172.21.15.183:6802/103132661,v1:172.21.15.183:6803/103132661] exists,up 99b709e5-af13-424e-9dc5-3b4c7e078d65 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/4182864338,v1:172.21.15.192:6801/4182864338] [v2:172.21.15.192:6802/4182864338,v1:172.21.15.192:6803/4182864338] exists,up e53f5387-30ec-46d0-bd1e-0950c58b3101 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6810/2500918591,v1:172.21.15.76:6811/2500918591] [v2:172.21.15.76:6812/2500918591,v1:172.21.15.76:6813/2500918591] exists,up 70f911b5-5902-413d-9e35-98630a33251d 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/3795301035,v1:172.21.15.183:6809/3795301035] [v2:172.21.15.183:6810/3795301035,v1:172.21.15.183:6811/3795301035] exists,up 9c4ca544-320c-4f9f-ba0c-54737df4a1a5 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/2902288003,v1:172.21.15.192:6809/2902288003] [v2:172.21.15.192:6810/2902288003,v1:172.21.15.192:6811/2902288003] exists,up ce29342e-6466-4e41-ba1e-2c8f10f4a559 2024-03-23T15:43:06.649 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6818/2821822561,v1:172.21.15.76:6819/2821822561] [v2:172.21.15.76:6820/2821822561,v1:172.21.15.76:6821/2821822561] exists,up 3260ee11-29d2-493e-9bd1-156fd317542d 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/336323954,v1:172.21.15.183:6817/336323954] [v2:172.21.15.183:6818/336323954,v1:172.21.15.183:6819/336323954] exists,up d430ca1a-9ec0-43c2-9f12-01ca89b117fb 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/2740228504,v1:172.21.15.192:6817/2740228504] [v2:172.21.15.192:6818/2740228504,v1:172.21.15.192:6819/2740228504] exists,up 5b4a7d4f-99a1-4109-a142-427ef07c1333 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:osd.9 up in weight 1 up_from 67 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6826/1143697448,v1:172.21.15.76:6827/1143697448] [v2:172.21.15.76:6828/1143697448,v1:172.21.15.76:6829/1143697448] exists,up c93eb38c-a6a9-453f-8b9a-da479e25a337 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/596098792,v1:172.21.15.183:6825/596098792] [v2:172.21.15.183:6826/596098792,v1:172.21.15.183:6827/596098792] exists,up ef61a179-f812-4a25-8605-cf03e2d8a31c 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/603123871,v1:172.21.15.192:6825/603123871] [v2:172.21.15.192:6826/603123871,v1:172.21.15.192:6827/603123871] exists,up ab72b35e-6449-42d0-9a9b-7014e9586173 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 2.0 [8,4] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 2.8 [2,11] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 2.e [2,4] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 2.1c [8,11] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 3.3 [4,9] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 3.10 [5,9] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 3.11 [4,7] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 3.14 [4,6] 2024-03-23T15:43:06.650 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 3.28 [5,8] 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:pg_upmap_items 3.2d [3,9] 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:0/871302578 expires 2024-03-24T15:34:15.395471+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:0/3336791879 expires 2024-03-24T15:34:15.395471+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:6801/36977200 expires 2024-03-24T15:34:15.395471+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:0/2354876004 expires 2024-03-24T15:33:50.341648+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:0/2171503933 expires 2024-03-24T15:33:50.341648+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:6800/36977200 expires 2024-03-24T15:34:15.395471+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:0/4014456486 expires 2024-03-24T15:33:50.341648+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:0/542720275 expires 2024-03-24T15:34:15.395471+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:6800/3268079315 expires 2024-03-24T15:33:50.341648+0000 2024-03-23T15:43:06.651 INFO:teuthology.orchestra.run.smithi076.stdout:blocklist 172.21.15.76:6801/3268079315 expires 2024-03-24T15:33:50.341648+0000 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f00d40a2150 msgr2=0x7f00d40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f00d40a2150 0x7f00d40a4610 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009550 tx=0x7f00e40091b0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 msgr2=0x7f00ec103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec103710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f00e0000c00 tx=0x7f00e000bfd0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 shutdown_connections 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f00d40a2150 0x7f00d40a4610 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec102cd0 0x7f00ec103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00ec101130 0x7f00ec1050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 --2- 172.21.15.76:0/1874775141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00ec102330 0x7f00ec103710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:06.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 >> 172.21.15.76:0/1874775141 conn(0x7f00ec0fc6d0 msgr2=0x7f00ec0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:06.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 shutdown_connections 2024-03-23T15:43:06.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:06.648+0000 7f00f1770700 1 -- 172.21.15.76:0/1874775141 wait complete. 2024-03-23T15:43:06.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:06 smithi192 ceph-mon[36653]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:43:06.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:06 smithi183 ceph-mon[36296]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-23T15:43:07.476 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-23T15:43:07.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:43:07.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/372122291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc100630 msgr2=0x7f8fcc100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/372122291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc100630 0x7f8fcc100a50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009900 tx=0x7f8fc002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:07.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/372122291 shutdown_connections 2024-03-23T15:43:07.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/372122291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fcc1021e0 0x7f8fcc108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/372122291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fcc101820 0x7f8fcc101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/372122291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc100630 0x7f8fcc100a50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/372122291 >> 172.21.15.76:0/372122291 conn(0x7f8fcc0fbba0 msgr2=0x7f8fcc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:07.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/372122291 shutdown_connections 2024-03-23T15:43:07.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/372122291 wait complete. 2024-03-23T15:43:07.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.593+0000 7f8fd1a8f640 1 Processor -- start 2024-03-23T15:43:07.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fd1a8f640 1 -- start start 2024-03-23T15:43:07.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fd1a8f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fcc100630 0x7f8fcc1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fd1a8f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fcc101820 0x7f8fcc196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fd1a8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc1021e0 0x7f8fcc19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fd1a8f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc19e1c0 con 0x7f8fcc1021e0 2024-03-23T15:43:07.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fd1a8f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc19e330 con 0x7f8fcc100630 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fd1a8f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc19e4f0 con 0x7f8fcc101820 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc1021e0 0x7f8fcc19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc1021e0 0x7f8fcc19dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35278/0 (socket says 172.21.15.76:35278) 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 -- 172.21.15.76:0/1066904234 learned_addr learned my addr 172.21.15.76:0/1066904234 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fcc100630 0x7f8fcc1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcaffd640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fcc101820 0x7f8fcc196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 -- 172.21.15.76:0/1066904234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fcc101820 msgr2=0x7f8fcc196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fcc101820 0x7f8fcc196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 -- 172.21.15.76:0/1066904234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fcc100630 msgr2=0x7f8fcc1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fcc100630 0x7f8fcc1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 -- 172.21.15.76:0/1066904234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc0009560 con 0x7f8fcc1021e0 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fcc100630 0x7f8fcc1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.594+0000 7f8fcbfff640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc1021e0 0x7f8fcc19dc80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400eb10 tx=0x7f8fc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.595+0000 7f8fc8ff9640 1 -- 172.21.15.76:0/1066904234 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004b20 con 0x7f8fcc1021e0 2024-03-23T15:43:07.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.595+0000 7f8fc8ff9640 1 -- 172.21.15.76:0/1066904234 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc4004c80 con 0x7f8fcc1021e0 2024-03-23T15:43:07.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.595+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc19e7d0 con 0x7f8fcc1021e0 2024-03-23T15:43:07.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.595+0000 7f8fc8ff9640 1 -- 172.21.15.76:0/1066904234 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4010650 con 0x7f8fcc1021e0 2024-03-23T15:43:07.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.595+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc19ece0 con 0x7f8fcc1021e0 2024-03-23T15:43:07.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.596+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fcc107950 con 0x7f8fcc1021e0 2024-03-23T15:43:07.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.597+0000 7f8fc8ff9640 1 -- 172.21.15.76:0/1066904234 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f8fc4005020 con 0x7f8fcc1021e0 2024-03-23T15:43:07.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.598+0000 7f8fc8ff9640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8fb40a20c0 0x7f8fb40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.598+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8fb40a20c0 0x7f8fb40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.598+0000 7f8fc8ff9640 1 -- 172.21.15.76:0/1066904234 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8fc4014070 con 0x7f8fcc1021e0 2024-03-23T15:43:07.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.599+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8fb40a20c0 0x7f8fb40a4580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005ab0 tx=0x7f8fc0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:07.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.603+0000 7f8fc8ff9640 1 -- 172.21.15.76:0/1066904234 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8fc40971c0 con 0x7f8fcc1021e0 2024-03-23T15:43:07.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.785+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fcc02d1d0 con 0x7f8fcc1021e0 2024-03-23T15:43:07.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.787+0000 7f8fc8ff9640 1 -- 172.21.15.76:0/1066904234 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22729 (secure 0 0 0) 0x7f8fc409a880 con 0x7f8fcc1021e0 2024-03-23T15:43:07.789 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:07.789 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:42:16.614449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:43:07.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8fb40a20c0 msgr2=0x7f8fb40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8fb40a20c0 0x7f8fb40a4580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005ab0 tx=0x7f8fc0005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc1021e0 msgr2=0x7f8fcc19dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc1021e0 0x7f8fcc19dc80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400eb10 tx=0x7f8fc400c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 shutdown_connections 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc1021e0 0x7f8fcc19dc80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fcc101820 0x7f8fcc196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8fb40a20c0 0x7f8fb40a4580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 --2- 172.21.15.76:0/1066904234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fcc100630 0x7f8fcc1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 >> 172.21.15.76:0/1066904234 conn(0x7f8fcc0fbba0 msgr2=0x7f8fcc104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 shutdown_connections 2024-03-23T15:43:07.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.789+0000 7f8fd1a8f640 1 -- 172.21.15.76:0/1066904234 wait complete. 2024-03-23T15:43:07.801 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:07 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1874775141' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T15:43:07.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:43:07.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:43:07.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:07 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1874775141' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T15:43:07.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:07 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1874775141' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-23T15:43:07.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 -- 172.21.15.76:0/1447102587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc028104ac0 msgr2=0x7fc028106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 --2- 172.21.15.76:0/1447102587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc028104ac0 0x7fc028106eb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc010009900 tx=0x7fc01002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:07.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 -- 172.21.15.76:0/1447102587 shutdown_connections 2024-03-23T15:43:07.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 --2- 172.21.15.76:0/1447102587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0281073f0 0x7fc028109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 --2- 172.21.15.76:0/1447102587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc028104ac0 0x7fc028106eb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 --2- 172.21.15.76:0/1447102587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102190 0x7fc028104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 -- 172.21.15.76:0/1447102587 >> 172.21.15.76:0/1447102587 conn(0x7fc0280fbb80 msgr2=0x7fc0280fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.908+0000 7fc02d50c640 1 -- 172.21.15.76:0/1447102587 shutdown_connections 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 -- 172.21.15.76:0/1447102587 wait complete. 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 Processor -- start 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 -- start start 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102190 0x7fc028197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc028104ac0 0x7fc028197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0281073f0 0x7fc02819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc02819f270 con 0x7fc0281073f0 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc02819f3e0 con 0x7fc028102190 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc02d50c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc02819f5a0 con 0x7fc028104ac0 2024-03-23T15:43:07.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0281073f0 0x7fc02819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0281073f0 0x7fc02819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35304/0 (socket says 172.21.15.76:35304) 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc0277fe640 1 -- 172.21.15.76:0/2598807084 learned_addr learned my addr 172.21.15.76:0/2598807084 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc026ffd640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102190 0x7fc028197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.909+0000 7fc01ffff640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc028104ac0 0x7fc028197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc0277fe640 1 -- 172.21.15.76:0/2598807084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc028104ac0 msgr2=0x7fc028197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc0277fe640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc028104ac0 0x7fc028197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc0277fe640 1 -- 172.21.15.76:0/2598807084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102190 msgr2=0x7fc028197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc0277fe640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102190 0x7fc028197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc0277fe640 1 -- 172.21.15.76:0/2598807084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc018009490 con 0x7fc0281073f0 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc026ffd640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102190 0x7fc028197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc0277fe640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0281073f0 0x7fc02819ed30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc02000eb10 tx=0x7fc02000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc024ff9640 1 -- 172.21.15.76:0/2598807084 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020004b20 con 0x7fc0281073f0 2024-03-23T15:43:07.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc024ff9640 1 -- 172.21.15.76:0/2598807084 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc020004c80 con 0x7fc0281073f0 2024-03-23T15:43:07.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc010009560 con 0x7fc0281073f0 2024-03-23T15:43:07.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc024ff9640 1 -- 172.21.15.76:0/2598807084 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020010650 con 0x7fc0281073f0 2024-03-23T15:43:07.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.910+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02819fb80 con 0x7fc0281073f0 2024-03-23T15:43:07.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.911+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0280fd670 con 0x7fc0281073f0 2024-03-23T15:43:07.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.913+0000 7fc024ff9640 1 -- 172.21.15.76:0/2598807084 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fc0200107f0 con 0x7fc0281073f0 2024-03-23T15:43:07.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.914+0000 7fc024ff9640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc0140a20c0 0x7fc0140a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:07.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.914+0000 7fc026ffd640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc0140a20c0 0x7fc0140a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.914+0000 7fc024ff9640 1 -- 172.21.15.76:0/2598807084 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc020014070 con 0x7fc0281073f0 2024-03-23T15:43:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.914+0000 7fc026ffd640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc0140a20c0 0x7fc0140a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc018011630 tx=0x7fc018005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:07.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:07.916+0000 7fc024ff9640 1 -- 172.21.15.76:0/2598807084 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc0200973d0 con 0x7fc0281073f0 2024-03-23T15:43:08.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.091+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc02802d1d0 con 0x7fc0281073f0 2024-03-23T15:43:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.092+0000 7fc024ff9640 1 -- 172.21.15.76:0/2598807084 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fc02009aa90 con 0x7fc0281073f0 2024-03-23T15:43:08.094 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc0140a20c0 msgr2=0x7fc0140a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc0140a20c0 0x7fc0140a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc018011630 tx=0x7fc018005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0281073f0 msgr2=0x7fc02819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0281073f0 0x7fc02819ed30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc02000eb10 tx=0x7fc02000c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 shutdown_connections 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0281073f0 0x7fc02819ed30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc028104ac0 0x7fc028197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc0140a20c0 0x7fc0140a4580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 --2- 172.21.15.76:0/2598807084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc028102190 0x7fc028197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 >> 172.21.15.76:0/2598807084 conn(0x7fc0280fbb80 msgr2=0x7fc028102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:08.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 shutdown_connections 2024-03-23T15:43:08.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.094+0000 7fc02d50c640 1 -- 172.21.15.76:0/2598807084 wait complete. 2024-03-23T15:43:08.107 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:43:08.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 -- 172.21.15.76:0/1733148940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf600ffff0 msgr2=0x7faf60100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 --2- 172.21.15.76:0/1733148940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf600ffff0 0x7faf60100410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faf50009900 tx=0x7faf5002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:08.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 -- 172.21.15.76:0/1733148940 shutdown_connections 2024-03-23T15:43:08.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 --2- 172.21.15.76:0/1733148940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf601012f0 0x7faf60101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 --2- 172.21.15.76:0/1733148940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf60100950 0x7faf60100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 --2- 172.21.15.76:0/1733148940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf600ffff0 0x7faf60100410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 -- 172.21.15.76:0/1733148940 >> 172.21.15.76:0/1733148940 conn(0x7faf600fbbb0 msgr2=0x7faf600fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:08.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 -- 172.21.15.76:0/1733148940 shutdown_connections 2024-03-23T15:43:08.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.210+0000 7faf64d84640 1 -- 172.21.15.76:0/1733148940 wait complete. 2024-03-23T15:43:08.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 Processor -- start 2024-03-23T15:43:08.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 -- start start 2024-03-23T15:43:08.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf600ffff0 0x7faf60199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60100950 0x7faf60199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf601012f0 0x7faf601a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faf601a1500 con 0x7faf601012f0 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf601a1670 con 0x7faf600ffff0 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf64d84640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faf601a1830 con 0x7faf60100950 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf601012f0 0x7faf601a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf601012f0 0x7faf601a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35314/0 (socket says 172.21.15.76:35314) 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 -- 172.21.15.76:0/1991124982 learned_addr learned my addr 172.21.15.76:0/1991124982 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5e59c640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf600ffff0 0x7faf60199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5dd9b640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60100950 0x7faf60199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 -- 172.21.15.76:0/1991124982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60100950 msgr2=0x7faf60199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60100950 0x7faf60199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 -- 172.21.15.76:0/1991124982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf600ffff0 msgr2=0x7faf60199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf600ffff0 0x7faf60199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 -- 172.21.15.76:0/1991124982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf50009560 con 0x7faf601012f0 2024-03-23T15:43:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5e59c640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf600ffff0 0x7faf60199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.211+0000 7faf5ed9d640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf601012f0 0x7faf601a0fc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf5800eb10 tx=0x7faf5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.212+0000 7faf4f7fe640 1 -- 172.21.15.76:0/1991124982 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58004b20 con 0x7faf601012f0 2024-03-23T15:43:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.212+0000 7faf4f7fe640 1 -- 172.21.15.76:0/1991124982 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf58004c80 con 0x7faf601012f0 2024-03-23T15:43:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.212+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf601a1b10 con 0x7faf601012f0 2024-03-23T15:43:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.212+0000 7faf4f7fe640 1 -- 172.21.15.76:0/1991124982 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58010650 con 0x7faf601012f0 2024-03-23T15:43:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.212+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf601a2020 con 0x7faf601012f0 2024-03-23T15:43:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.213+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6010af10 con 0x7faf601012f0 2024-03-23T15:43:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.214+0000 7faf4f7fe640 1 -- 172.21.15.76:0/1991124982 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7faf580107d0 con 0x7faf601012f0 2024-03-23T15:43:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.214+0000 7faf4f7fe640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf480a20c0 0x7faf480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.214+0000 7faf5e59c640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf480a20c0 0x7faf480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.214+0000 7faf4f7fe640 1 -- 172.21.15.76:0/1991124982 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7faf58014070 con 0x7faf601012f0 2024-03-23T15:43:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.215+0000 7faf5e59c640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf480a20c0 0x7faf480a4580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faf50009230 tx=0x7faf50005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:08.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.218+0000 7faf4f7fe640 1 -- 172.21.15.76:0/1991124982 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7faf58097390 con 0x7faf601012f0 2024-03-23T15:43:08.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.391+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faf6010f550 con 0x7faf601012f0 2024-03-23T15:43:08.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.391+0000 7faf4f7fe640 1 -- 172.21.15.76:0/1991124982 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7faf5809aa50 con 0x7faf601012f0 2024-03-23T15:43:08.394 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:43:08.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf480a20c0 msgr2=0x7faf480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf480a20c0 0x7faf480a4580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faf50009230 tx=0x7faf50005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf601012f0 msgr2=0x7faf601a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf601012f0 0x7faf601a0fc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf5800eb10 tx=0x7faf5800c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 shutdown_connections 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf601012f0 0x7faf601a0fc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60100950 0x7faf60199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7faf480a20c0 0x7faf480a4580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 --2- 172.21.15.76:0/1991124982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf600ffff0 0x7faf60199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 >> 172.21.15.76:0/1991124982 conn(0x7faf600fbbb0 msgr2=0x7faf600fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:08.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 shutdown_connections 2024-03-23T15:43:08.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.393+0000 7faf64d84640 1 -- 172.21.15.76:0/1991124982 wait complete. 2024-03-23T15:43:08.407 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:43:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 -- 172.21.15.76:0/1205323050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc1021c0 msgr2=0x7fd3dc108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 --2- 172.21.15.76:0/1205323050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc1021c0 0x7fd3dc108a50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009900 tx=0x7fd3d402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 -- 172.21.15.76:0/1205323050 shutdown_connections 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 --2- 172.21.15.76:0/1205323050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc1021c0 0x7fd3dc108a50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 --2- 172.21.15.76:0/1205323050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3dc101800 0x7fd3dc101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 --2- 172.21.15.76:0/1205323050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc100610 0x7fd3dc100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 -- 172.21.15.76:0/1205323050 >> 172.21.15.76:0/1205323050 conn(0x7fd3dc0fbb80 msgr2=0x7fd3dc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 -- 172.21.15.76:0/1205323050 shutdown_connections 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.510+0000 7fd3e1864640 1 -- 172.21.15.76:0/1205323050 wait complete. 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 Processor -- start 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 -- start start 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3dc100610 0x7fd3dc196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc101800 0x7fd3dc196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc1021c0 0x7fd3dc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc19e120 con 0x7fd3dc101800 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc19e290 con 0x7fd3dc1021c0 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3e1864640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc19e450 con 0x7fd3dc100610 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc101800 0x7fd3dc196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc101800 0x7fd3dc196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35338/0 (socket says 172.21.15.76:35338) 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 -- 172.21.15.76:0/3793150907 learned_addr learned my addr 172.21.15.76:0/3793150907 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3daffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3dc100610 0x7fd3dc196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3db7fe640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc1021c0 0x7fd3dc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 -- 172.21.15.76:0/3793150907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3dc100610 msgr2=0x7fd3dc196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3dc100610 0x7fd3dc196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 -- 172.21.15.76:0/3793150907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc1021c0 msgr2=0x7fd3dc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc1021c0 0x7fd3dc19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.511+0000 7fd3da7fc640 1 -- 172.21.15.76:0/3793150907 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d4009560 con 0x7fd3dc101800 2024-03-23T15:43:08.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.512+0000 7fd3da7fc640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc101800 0x7fd3dc196b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e9f0 tx=0x7fd3d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:08.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.512+0000 7fd3e0862640 1 -- 172.21.15.76:0/3793150907 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0009700 con 0x7fd3dc101800 2024-03-23T15:43:08.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.512+0000 7fd3e0862640 1 -- 172.21.15.76:0/3793150907 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d000cd80 con 0x7fd3dc101800 2024-03-23T15:43:08.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.512+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19e730 con 0x7fd3dc101800 2024-03-23T15:43:08.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.512+0000 7fd3e0862640 1 -- 172.21.15.76:0/3793150907 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0018920 con 0x7fd3dc101800 2024-03-23T15:43:08.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.512+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc19ec40 con 0x7fd3dc101800 2024-03-23T15:43:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.513+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc107930 con 0x7fd3dc101800 2024-03-23T15:43:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.514+0000 7fd3e0862640 1 -- 172.21.15.76:0/3793150907 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fd3d0018a80 con 0x7fd3dc101800 2024-03-23T15:43:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.514+0000 7fd3e0862640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd3c40a20c0 0x7fd3c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.514+0000 7fd3daffd640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd3c40a20c0 0x7fd3c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.514+0000 7fd3e0862640 1 -- 172.21.15.76:0/3793150907 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd3d0014070 con 0x7fd3dc101800 2024-03-23T15:43:08.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.515+0000 7fd3daffd640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd3c40a20c0 0x7fd3c40a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc101660 tx=0x7fd3cc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:08.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.518+0000 7fd3e0862640 1 -- 172.21.15.76:0/3793150907 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd3d0053280 con 0x7fd3dc101800 2024-03-23T15:43:08.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.693+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd3dc02d1d0 con 0x7fd3dc101800 2024-03-23T15:43:08.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.694+0000 7fd3e0862640 1 -- 172.21.15.76:0/3793150907 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fd3d0097090 con 0x7fd3dc101800 2024-03-23T15:43:08.696 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd3c40a20c0 msgr2=0x7fd3c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd3c40a20c0 0x7fd3c40a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc101660 tx=0x7fd3cc009500 comp rx=0 tx=0).stop 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc101800 msgr2=0x7fd3dc196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc101800 0x7fd3dc196b70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000e9f0 tx=0x7fd3d000eeb0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 shutdown_connections 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd3c40a20c0 0x7fd3c40a4580 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc1021c0 0x7fd3dc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3dc101800 0x7fd3dc196b70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 --2- 172.21.15.76:0/3793150907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3dc100610 0x7fd3dc196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 >> 172.21.15.76:0/3793150907 conn(0x7fd3dc0fbb80 msgr2=0x7fd3dc104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:08.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 shutdown_connections 2024-03-23T15:43:08.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.696+0000 7fd3e1864640 1 -- 172.21.15.76:0/3793150907 wait complete. 2024-03-23T15:43:08.708 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:08 smithi076 ceph-mon[32176]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:08.708 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:08 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1066904234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:08.708 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:08 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2598807084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:08.708 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:08 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1991124982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:08.710 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:43:08.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 -- 172.21.15.76:0/1872995369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 msgr2=0x7f69f4101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/1872995369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4101b70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009900 tx=0x7f69e402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:08.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 -- 172.21.15.76:0/1872995369 shutdown_connections 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/1872995369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f4103300 0x7f69f4109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/1872995369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f4102940 0x7f69f4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/1872995369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4101b70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 -- 172.21.15.76:0/1872995369 >> 172.21.15.76:0/1872995369 conn(0x7f69f40fccc0 msgr2=0x7f69f40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 -- 172.21.15.76:0/1872995369 shutdown_connections 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 -- 172.21.15.76:0/1872995369 wait complete. 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 Processor -- start 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.810+0000 7f69f9fa0640 1 -- start start 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f4102940 0x7f69f4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f4103300 0x7f69f419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f69f419f310 con 0x7f69f4101750 2024-03-23T15:43:08.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69f419f480 con 0x7f69f4102940 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f69f419f640 con 0x7f69f4103300 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35356/0 (socket says 172.21.15.76:35356) 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 -- 172.21.15.76:0/2081287841 learned_addr learned my addr 172.21.15.76:0/2081287841 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69ebfff640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f4102940 0x7f69f4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f3fff640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f4103300 0x7f69f419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 -- 172.21.15.76:0/2081287841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f4103300 msgr2=0x7f69f419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f4103300 0x7f69f419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 -- 172.21.15.76:0/2081287841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f4102940 msgr2=0x7f69f4197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f4102940 0x7f69f4197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 -- 172.21.15.76:0/2081287841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e4009560 con 0x7f69f4101750 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f37fe640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4197820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f69e4000c00 tx=0x7f69e4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f17fa640 1 -- 172.21.15.76:0/2081287841 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e40043d0 con 0x7f69f4101750 2024-03-23T15:43:08.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f419f8c0 con 0x7f69f4101750 2024-03-23T15:43:08.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f17fa640 1 -- 172.21.15.76:0/2081287841 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e40052d0 con 0x7f69f4101750 2024-03-23T15:43:08.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f17fa640 1 -- 172.21.15.76:0/2081287841 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4003d30 con 0x7f69f4101750 2024-03-23T15:43:08.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.811+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f419fdd0 con 0x7f69f4101750 2024-03-23T15:43:08.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.812+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f4108ac0 con 0x7f69f4101750 2024-03-23T15:43:08.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.813+0000 7f69f17fa640 1 -- 172.21.15.76:0/2081287841 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f69e4005440 con 0x7f69f4101750 2024-03-23T15:43:08.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.814+0000 7f69f17fa640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f69e00a20c0 0x7f69e00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:08.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.814+0000 7f69ebfff640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f69e00a20c0 0x7f69e00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:08.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.814+0000 7f69f17fa640 1 -- 172.21.15.76:0/2081287841 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f69e40f08a0 con 0x7f69f4101750 2024-03-23T15:43:08.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.814+0000 7f69ebfff640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f69e00a20c0 0x7f69e00a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f69f4198d30 tx=0x7f69dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:08 smithi192 ceph-mon[36653]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:08 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1066904234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:08 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2598807084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:08.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:08 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1991124982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:08.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:08.818+0000 7f69f17fa640 1 -- 172.21.15.76:0/2081287841 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f69e40b9850 con 0x7f69f4101750 2024-03-23T15:43:08.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:08 smithi183 ceph-mon[36296]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:08.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1066904234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:08.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2598807084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:08.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1991124982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:09.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.029+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69f410d1a0 con 0x7f69f4101750 2024-03-23T15:43:09.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.030+0000 7f69f17fa640 1 -- 172.21.15.76:0/2081287841 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f69e40bcf10 con 0x7f69f4101750 2024-03-23T15:43:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:09.032 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:41:22.694125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:43:09.032 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 21 2024-03-23T15:43:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f69e00a20c0 msgr2=0x7f69e00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f69e00a20c0 0x7f69e00a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f69f4198d30 tx=0x7f69dc005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 msgr2=0x7f69f4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4197820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f69e4000c00 tx=0x7f69e4005e80 comp rx=0 tx=0).stop 2024-03-23T15:43:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 shutdown_connections 2024-03-23T15:43:09.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69f4103300 0x7f69f419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f69e00a20c0 0x7f69e00a4580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69f4102940 0x7f69f4197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 --2- 172.21.15.76:0/2081287841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69f4101750 0x7f69f4197820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 >> 172.21.15.76:0/2081287841 conn(0x7f69f40fccc0 msgr2=0x7f69f4105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 shutdown_connections 2024-03-23T15:43:09.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.032+0000 7f69f9fa0640 1 -- 172.21.15.76:0/2081287841 wait complete. 2024-03-23T15:43:09.045 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:43:09.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 -- 172.21.15.76:0/1792021491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd641021e0 msgr2=0x7fcd64108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/1792021491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd641021e0 0x7fcd64108a70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009ca0 tx=0x7fcd5c02c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 -- 172.21.15.76:0/1792021491 shutdown_connections 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/1792021491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd641021e0 0x7fcd64108a70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/1792021491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd64101820 0x7fcd64101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/1792021491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64100630 0x7fcd64100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 -- 172.21.15.76:0/1792021491 >> 172.21.15.76:0/1792021491 conn(0x7fcd640fbba0 msgr2=0x7fcd640fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 -- 172.21.15.76:0/1792021491 shutdown_connections 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.145+0000 7fcd6a13e640 1 -- 172.21.15.76:0/1792021491 wait complete. 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 Processor -- start 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 -- start start 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd64100630 0x7fcd64069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64101820 0x7fcd6419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd641021e0 0x7fcd6419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6419f290 con 0x7fcd64100630 2024-03-23T15:43:09.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6419f400 con 0x7fcd64101820 2024-03-23T15:43:09.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd6a13e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6419f5c0 con 0x7fcd641021e0 2024-03-23T15:43:09.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd64100630 0x7fcd64069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd64100630 0x7fcd64069400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35366/0 (socket says 172.21.15.76:35366) 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 -- 172.21.15.76:0/3909843743 learned_addr learned my addr 172.21.15.76:0/3909843743 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd62ffd640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64101820 0x7fcd6419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd63fff640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd641021e0 0x7fcd6419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 -- 172.21.15.76:0/3909843743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd641021e0 msgr2=0x7fcd6419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd641021e0 0x7fcd6419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 -- 172.21.15.76:0/3909843743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64101820 msgr2=0x7fcd6419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64101820 0x7fcd6419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.146+0000 7fcd637fe640 1 -- 172.21.15.76:0/3909843743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c009900 con 0x7fcd64100630 2024-03-23T15:43:09.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.147+0000 7fcd637fe640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd64100630 0x7fcd64069400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400e9c0 tx=0x7fcd5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:09.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.147+0000 7fcd60ff9640 1 -- 172.21.15.76:0/3909843743 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54009700 con 0x7fcd64100630 2024-03-23T15:43:09.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.147+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6419f8a0 con 0x7fcd64100630 2024-03-23T15:43:09.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.147+0000 7fcd60ff9640 1 -- 172.21.15.76:0/3909843743 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5400cd80 con 0x7fcd64100630 2024-03-23T15:43:09.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.147+0000 7fcd60ff9640 1 -- 172.21.15.76:0/3909843743 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54010430 con 0x7fcd64100630 2024-03-23T15:43:09.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.147+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6419fe00 con 0x7fcd64100630 2024-03-23T15:43:09.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.148+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd64107950 con 0x7fcd64100630 2024-03-23T15:43:09.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.149+0000 7fcd60ff9640 1 -- 172.21.15.76:0/3909843743 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fcd54003680 con 0x7fcd64100630 2024-03-23T15:43:09.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.149+0000 7fcd60ff9640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcd4c0a2140 0x7fcd4c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.150+0000 7fcd62ffd640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcd4c0a2140 0x7fcd4c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.150+0000 7fcd60ff9640 1 -- 172.21.15.76:0/3909843743 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fcd54014070 con 0x7fcd64100630 2024-03-23T15:43:09.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.150+0000 7fcd62ffd640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcd4c0a2140 0x7fcd4c0a4600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd64069170 tx=0x7fcd58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:09.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.154+0000 7fcd60ff9640 1 -- 172.21.15.76:0/3909843743 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fcd54096f80 con 0x7fcd64100630 2024-03-23T15:43:09.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.328+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd6410c040 con 0x7fcd64100630 2024-03-23T15:43:09.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.330+0000 7fcd60ff9640 1 -- 172.21.15.76:0/3909843743 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22729 (secure 0 0 0) 0x7fcd5409a640 con 0x7fcd64100630 2024-03-23T15:43:09.332 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:09.332 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:42:16.614449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:43:09.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcd4c0a2140 msgr2=0x7fcd4c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcd4c0a2140 0x7fcd4c0a4600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd64069170 tx=0x7fcd58005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd64100630 msgr2=0x7fcd64069400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd64100630 0x7fcd64069400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400e9c0 tx=0x7fcd5400ee80 comp rx=0 tx=0).stop 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 shutdown_connections 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd641021e0 0x7fcd6419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fcd4c0a2140 0x7fcd4c0a4600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd64101820 0x7fcd6419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 --2- 172.21.15.76:0/3909843743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd64100630 0x7fcd64069400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 >> 172.21.15.76:0/3909843743 conn(0x7fcd640fbba0 msgr2=0x7fcd64104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 shutdown_connections 2024-03-23T15:43:09.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.332+0000 7fcd6a13e640 1 -- 172.21.15.76:0/3909843743 wait complete. 2024-03-23T15:43:09.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:43:09.347 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 -- 172.21.15.76:0/1796734411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8100630 msgr2=0x7f47d8100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 --2- 172.21.15.76:0/1796734411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8100630 0x7f47d8100a50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009900 tx=0x7f47d402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 -- 172.21.15.76:0/1796734411 shutdown_connections 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 --2- 172.21.15.76:0/1796734411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47d81021e0 0x7f47d8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 --2- 172.21.15.76:0/1796734411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47d8101820 0x7f47d8101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 --2- 172.21.15.76:0/1796734411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8100630 0x7f47d8100a50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 -- 172.21.15.76:0/1796734411 >> 172.21.15.76:0/1796734411 conn(0x7f47d80fbbc0 msgr2=0x7f47d80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 -- 172.21.15.76:0/1796734411 shutdown_connections 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 -- 172.21.15.76:0/1796734411 wait complete. 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.450+0000 7f47e06c3640 1 Processor -- start 2024-03-23T15:43:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 -- start start 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47d8100630 0x7f47d806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8101820 0x7f47d819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47d81021e0 0x7f47d819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f47d819f300 con 0x7f47d8101820 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47d819f470 con 0x7f47d8100630 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47d819f630 con 0x7f47d81021e0 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8101820 0x7f47d819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8101820 0x7f47d819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35370/0 (socket says 172.21.15.76:35370) 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 -- 172.21.15.76:0/2075553750 learned_addr learned my addr 172.21.15.76:0/2075553750 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47de45f640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47d8100630 0x7f47d806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47dec60640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47d81021e0 0x7f47d819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 -- 172.21.15.76:0/2075553750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47d81021e0 msgr2=0x7f47d819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47d81021e0 0x7f47d819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 -- 172.21.15.76:0/2075553750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47d8100630 msgr2=0x7f47d806a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47d8100630 0x7f47d806a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 -- 172.21.15.76:0/2075553750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d4009560 con 0x7f47d8101820 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47dec60640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47d81021e0 0x7f47d819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47de45f640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47d8100630 0x7f47d806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47ddc5e640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8101820 0x7f47d819c500 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f47c800e9f0 tx=0x7f47c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47cf7fe640 1 -- 172.21.15.76:0/2075553750 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8009720 con 0x7f47d8101820 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47cf7fe640 1 -- 172.21.15.76:0/2075553750 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c800cd80 con 0x7f47d8101820 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47cf7fe640 1 -- 172.21.15.76:0/2075553750 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8010690 con 0x7f47d8101820 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.451+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d819f8b0 con 0x7f47d8101820 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.452+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d819fd70 con 0x7f47d8101820 2024-03-23T15:43:09.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.452+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d8107950 con 0x7f47d8101820 2024-03-23T15:43:09.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.453+0000 7f47cf7fe640 1 -- 172.21.15.76:0/2075553750 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f47c80046c0 con 0x7f47d8101820 2024-03-23T15:43:09.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.454+0000 7f47cf7fe640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47c40a20c0 0x7f47c40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.454+0000 7f47de45f640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47c40a20c0 0x7f47c40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.454+0000 7f47cf7fe640 1 -- 172.21.15.76:0/2075553750 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f47c8014070 con 0x7f47d8101820 2024-03-23T15:43:09.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.454+0000 7f47de45f640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47c40a20c0 0x7f47c40a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f47d402c240 tx=0x7f47d402c7e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:09.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.458+0000 7f47cf7fe640 1 -- 172.21.15.76:0/2075553750 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f47c8097090 con 0x7f47d8101820 2024-03-23T15:43:09.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.632+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47d802d160 con 0x7f47d8101820 2024-03-23T15:43:09.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.633+0000 7f47cf7fe640 1 -- 172.21.15.76:0/2075553750 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f47c809a750 con 0x7f47d8101820 2024-03-23T15:43:09.635 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:43:09.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47c40a20c0 msgr2=0x7f47c40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47c40a20c0 0x7f47c40a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f47d402c240 tx=0x7f47d402c7e0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8101820 msgr2=0x7f47d819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8101820 0x7f47d819c500 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f47c800e9f0 tx=0x7f47c800edf0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 shutdown_connections 2024-03-23T15:43:09.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47d81021e0 0x7f47d819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47d8101820 0x7f47d819c500 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f47c40a20c0 0x7f47c40a4580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 --2- 172.21.15.76:0/2075553750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47d8100630 0x7f47d806a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 >> 172.21.15.76:0/2075553750 conn(0x7f47d80fbbc0 msgr2=0x7f47d8104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:09.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 shutdown_connections 2024-03-23T15:43:09.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.635+0000 7f47e06c3640 1 -- 172.21.15.76:0/2075553750 wait complete. 2024-03-23T15:43:09.647 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3793150907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:09.647 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2081287841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:09.647 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3909843743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:09.648 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:43:09.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 -- 172.21.15.76:0/3065017169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 msgr2=0x7fc8d4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/3065017169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d4100a40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009900 tx=0x7fc8c402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 -- 172.21.15.76:0/3065017169 shutdown_connections 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/3065017169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d41021d0 0x7fc8d4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/3065017169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8d4101810 0x7fc8d4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/3065017169 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d4100a40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 -- 172.21.15.76:0/3065017169 >> 172.21.15.76:0/3065017169 conn(0x7fc8d40fbbb0 msgr2=0x7fc8d40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.751+0000 7fc8d90a4640 1 -- 172.21.15.76:0/3065017169 shutdown_connections 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 -- 172.21.15.76:0/3065017169 wait complete. 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 Processor -- start 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 -- start start 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8d4101810 0x7fc8d419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d41021d0 0x7fc8d419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d419f2c0 con 0x7fc8d4100620 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d419f430 con 0x7fc8d4101810 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d90a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d419f5f0 con 0x7fc8d41021d0 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d406a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35376/0 (socket says 172.21.15.76:35376) 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.752+0000 7fc8d2d9d640 1 -- 172.21.15.76:0/2585344968 learned_addr learned my addr 172.21.15.76:0/2585344968 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8cbfff640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8d4101810 0x7fc8d419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d359e640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d41021d0 0x7fc8d419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d2d9d640 1 -- 172.21.15.76:0/2585344968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d41021d0 msgr2=0x7fc8d419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d2d9d640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d41021d0 0x7fc8d419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d2d9d640 1 -- 172.21.15.76:0/2585344968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8d4101810 msgr2=0x7fc8d419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d2d9d640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8d4101810 0x7fc8d419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d2d9d640 1 -- 172.21.15.76:0/2585344968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c4009560 con 0x7fc8d4100620 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d359e640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d41021d0 0x7fc8d419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d2d9d640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d406a870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005ab0 tx=0x7fc8c4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d0d99640 1 -- 172.21.15.76:0/2585344968 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40043d0 con 0x7fc8d4100620 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d0d99640 1 -- 172.21.15.76:0/2585344968 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c40052d0 con 0x7fc8d4100620 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d419f870 con 0x7fc8d4100620 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d0d99640 1 -- 172.21.15.76:0/2585344968 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4003d40 con 0x7fc8d4100620 2024-03-23T15:43:09.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.753+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d419fd80 con 0x7fc8d4100620 2024-03-23T15:43:09.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.754+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d41079c0 con 0x7fc8d4100620 2024-03-23T15:43:09.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.755+0000 7fc8d0d99640 1 -- 172.21.15.76:0/2585344968 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fc8c4005440 con 0x7fc8d4100620 2024-03-23T15:43:09.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.756+0000 7fc8d0d99640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8c00a20c0 0x7fc8c00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:09.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.756+0000 7fc8cbfff640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8c00a20c0 0x7fc8c00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:09.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.756+0000 7fc8d0d99640 1 -- 172.21.15.76:0/2585344968 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc8c40efe90 con 0x7fc8d4100620 2024-03-23T15:43:09.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.756+0000 7fc8cbfff640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8c00a20c0 0x7fc8c00a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8d406a5e0 tx=0x7fc8bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:09.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.760+0000 7fc8d0d99640 1 -- 172.21.15.76:0/2585344968 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc8c40b8e40 con 0x7fc8d4100620 2024-03-23T15:43:09.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3793150907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:09.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2081287841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:09.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3909843743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3793150907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2081287841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3909843743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:09.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.933+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8d402d1d0 con 0x7fc8d4100620 2024-03-23T15:43:09.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.933+0000 7fc8d0d99640 1 -- 172.21.15.76:0/2585344968 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fc8c40bc500 con 0x7fc8d4100620 2024-03-23T15:43:09.936 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8c00a20c0 msgr2=0x7fc8c00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8c00a20c0 0x7fc8c00a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc8d406a5e0 tx=0x7fc8bc005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 msgr2=0x7fc8d406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d406a870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005ab0 tx=0x7fc8c4005e50 comp rx=0 tx=0).stop 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 shutdown_connections 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d41021d0 0x7fc8d419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8c00a20c0 0x7fc8c00a4580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8d4101810 0x7fc8d419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 --2- 172.21.15.76:0/2585344968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8d4100620 0x7fc8d406a870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 >> 172.21.15.76:0/2585344968 conn(0x7fc8d40fbbb0 msgr2=0x7fc8d4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:09.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 shutdown_connections 2024-03-23T15:43:09.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:09.935+0000 7fc8d90a4640 1 -- 172.21.15.76:0/2585344968 wait complete. 2024-03-23T15:43:09.949 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:43:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 -- 172.21.15.76:0/2714671354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 msgr2=0x7fbee4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 --2- 172.21.15.76:0/2714671354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee4101c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009900 tx=0x7fbed802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 -- 172.21.15.76:0/2714671354 shutdown_connections 2024-03-23T15:43:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 --2- 172.21.15.76:0/2714671354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbee41021d0 0x7fbee4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 --2- 172.21.15.76:0/2714671354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee4101c90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 --2- 172.21.15.76:0/2714671354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee4100620 0x7fbee4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 -- 172.21.15.76:0/2714671354 >> 172.21.15.76:0/2714671354 conn(0x7fbee40fbb70 msgr2=0x7fbee40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 -- 172.21.15.76:0/2714671354 shutdown_connections 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 -- 172.21.15.76:0/2714671354 wait complete. 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.052+0000 7fbee9442640 1 Processor -- start 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee9442640 1 -- start start 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee9442640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbee4100620 0x7fbee406a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee9442640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee9442640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee41021d0 0x7fbee419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee9442640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbee419f270 con 0x7fbee4101810 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee9442640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbee419f3e0 con 0x7fbee4100620 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee9442640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbee419f5a0 con 0x7fbee41021d0 2024-03-23T15:43:10.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35406/0 (socket says 172.21.15.76:35406) 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 -- 172.21.15.76:0/1629332945 learned_addr learned my addr 172.21.15.76:0/1629332945 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee37fe640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee41021d0 0x7fbee419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee2ffd640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbee4100620 0x7fbee406a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 -- 172.21.15.76:0/1629332945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee41021d0 msgr2=0x7fbee419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee41021d0 0x7fbee419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 -- 172.21.15.76:0/1629332945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbee4100620 msgr2=0x7fbee406a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbee4100620 0x7fbee406a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 -- 172.21.15.76:0/1629332945 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed8009560 con 0x7fbee4101810 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.053+0000 7fbee27fc640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee419c470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006010 tx=0x7fbed8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:10.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.054+0000 7fbecbfff640 1 -- 172.21.15.76:0/1629332945 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80043d0 con 0x7fbee4101810 2024-03-23T15:43:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.054+0000 7fbecbfff640 1 -- 172.21.15.76:0/1629332945 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed80052d0 con 0x7fbee4101810 2024-03-23T15:43:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.054+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee419f820 con 0x7fbee4101810 2024-03-23T15:43:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.054+0000 7fbecbfff640 1 -- 172.21.15.76:0/1629332945 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003cc0 con 0x7fbee4101810 2024-03-23T15:43:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.054+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee419fd30 con 0x7fbee4101810 2024-03-23T15:43:10.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.055+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee4107940 con 0x7fbee4101810 2024-03-23T15:43:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.056+0000 7fbecbfff640 1 -- 172.21.15.76:0/1629332945 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fbed8005440 con 0x7fbee4101810 2024-03-23T15:43:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.056+0000 7fbecbfff640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbecc0a20c0 0x7fbecc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.056+0000 7fbee2ffd640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbecc0a20c0 0x7fbecc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.056+0000 7fbecbfff640 1 -- 172.21.15.76:0/1629332945 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fbed80efee0 con 0x7fbee4101810 2024-03-23T15:43:10.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.057+0000 7fbee2ffd640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbecc0a20c0 0x7fbecc0a4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbee4101670 tx=0x7fbed4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:10.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.061+0000 7fbecbfff640 1 -- 172.21.15.76:0/1629332945 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fbed80b8e90 con 0x7fbee4101810 2024-03-23T15:43:10.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.234+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbee402d130 con 0x7fbee4101810 2024-03-23T15:43:10.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.235+0000 7fbecbfff640 1 -- 172.21.15.76:0/1629332945 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fbed80bc550 con 0x7fbee4101810 2024-03-23T15:43:10.237 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:43:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbecc0a20c0 msgr2=0x7fbecc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbecc0a20c0 0x7fbecc0a4580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbee4101670 tx=0x7fbed4005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 msgr2=0x7fbee419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee419c470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006010 tx=0x7fbed8005e80 comp rx=0 tx=0).stop 2024-03-23T15:43:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 shutdown_connections 2024-03-23T15:43:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee41021d0 0x7fbee419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee4101810 0x7fbee419c470 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fbecc0a20c0 0x7fbecc0a4580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 --2- 172.21.15.76:0/1629332945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbee4100620 0x7fbee406a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 >> 172.21.15.76:0/1629332945 conn(0x7fbee40fbb70 msgr2=0x7fbee4104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 shutdown_connections 2024-03-23T15:43:10.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.237+0000 7fbee9442640 1 -- 172.21.15.76:0/1629332945 wait complete. 2024-03-23T15:43:10.250 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 -- 172.21.15.76:0/155711447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 msgr2=0x7f289c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/155711447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c108220 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2894009ca0 tx=0x7f289402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 -- 172.21.15.76:0/155711447 shutdown_connections 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/155711447 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c108220 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/155711447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f289c100fd0 0x7f289c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/155711447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c0ffde0 0x7f289c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 -- 172.21.15.76:0/155711447 >> 172.21.15.76:0/155711447 conn(0x7f289c0fb350 msgr2=0x7f289c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 -- 172.21.15.76:0/155711447 shutdown_connections 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.349+0000 7f28a1f6c640 1 -- 172.21.15.76:0/155711447 wait complete. 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 Processor -- start 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 -- start start 2024-03-23T15:43:10.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c0ffde0 0x7f289c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f289c100fd0 0x7f289c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f289c19f2b0 con 0x7f289c101990 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f289c19f420 con 0x7f289c0ffde0 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f28a1f6c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f289c19f5e0 con 0x7f289c100fd0 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35430/0 (socket says 172.21.15.76:35430) 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289bfff640 1 -- 172.21.15.76:0/1221229275 learned_addr learned my addr 172.21.15.76:0/1221229275 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f2892ffd640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f289c100fd0 0x7f289c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289b7fe640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c0ffde0 0x7f289c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289bfff640 1 -- 172.21.15.76:0/1221229275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f289c100fd0 msgr2=0x7f289c19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289bfff640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f289c100fd0 0x7f289c19c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289bfff640 1 -- 172.21.15.76:0/1221229275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c0ffde0 msgr2=0x7f289c0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.350+0000 7f289bfff640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c0ffde0 0x7f289c0fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f289bfff640 1 -- 172.21.15.76:0/1221229275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2894009900 con 0x7f289c101990 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f2892ffd640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f289c100fd0 0x7f289c19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f289b7fe640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c0ffde0 0x7f289c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f289bfff640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c19eb20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2894037040 tx=0x7f2894009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f28997fa640 1 -- 172.21.15.76:0/1221229275 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28940329b0 con 0x7f289c101990 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c19f860 con 0x7f289c101990 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f28997fa640 1 -- 172.21.15.76:0/1221229275 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2894003e50 con 0x7f289c101990 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f28997fa640 1 -- 172.21.15.76:0/1221229275 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28940048a0 con 0x7f289c101990 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.351+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c19fd10 con 0x7f289c101990 2024-03-23T15:43:10.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.352+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c10b7c0 con 0x7f289c101990 2024-03-23T15:43:10.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.353+0000 7f28997fa640 1 -- 172.21.15.76:0/1221229275 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f289400a040 con 0x7f289c101990 2024-03-23T15:43:10.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.353+0000 7f28997fa640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f28880a20c0 0x7f28880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.354+0000 7f289b7fe640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f28880a20c0 0x7f28880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.354+0000 7f28997fa640 1 -- 172.21.15.76:0/1221229275 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f28940f0a30 con 0x7f289c101990 2024-03-23T15:43:10.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.354+0000 7f289b7fe640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f28880a20c0 0x7f28880a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f289c100e30 tx=0x7f288c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:10.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.358+0000 7f28997fa640 1 -- 172.21.15.76:0/1221229275 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f28940b9a70 con 0x7f289c101990 2024-03-23T15:43:10.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.569+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f289c10b980 con 0x7f289c101990 2024-03-23T15:43:10.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.570+0000 7f28997fa640 1 -- 172.21.15.76:0/1221229275 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11872 (secure 0 0 0) 0x7f2894034e60 con 0x7f289c101990 2024-03-23T15:43:10.572 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:10.572 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:41:22.694125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:43:10.572 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 21 2024-03-23T15:43:10.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f28880a20c0 msgr2=0x7f28880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f28880a20c0 0x7f28880a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f289c100e30 tx=0x7f288c005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:10.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 msgr2=0x7f289c19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c19eb20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2894037040 tx=0x7f2894009360 comp rx=0 tx=0).stop 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 shutdown_connections 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f289c101990 0x7f289c19eb20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f289c100fd0 0x7f289c19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f28880a20c0 0x7f28880a4580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 --2- 172.21.15.76:0/1221229275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f289c0ffde0 0x7f289c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 >> 172.21.15.76:0/1221229275 conn(0x7f289c0fb350 msgr2=0x7f289c1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 shutdown_connections 2024-03-23T15:43:10.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.572+0000 7f28a1f6c640 1 -- 172.21.15.76:0/1221229275 wait complete. 2024-03-23T15:43:10.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:43:10.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.689+0000 7f8d3968d640 1 -- 172.21.15.76:0/16136401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d34102260 msgr2=0x7f8d34108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.689+0000 7f8d3968d640 1 --2- 172.21.15.76:0/16136401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d34102260 0x7f8d34108af0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009ca0 tx=0x7f8d2c02c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.689+0000 7f8d3968d640 1 -- 172.21.15.76:0/16136401 shutdown_connections 2024-03-23T15:43:10.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.689+0000 7f8d3968d640 1 --2- 172.21.15.76:0/16136401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d34102260 0x7f8d34108af0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.689+0000 7f8d3968d640 1 --2- 172.21.15.76:0/16136401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d341018a0 0x7f8d34101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.689+0000 7f8d3968d640 1 --2- 172.21.15.76:0/16136401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d341006b0 0x7f8d34100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.689+0000 7f8d3968d640 1 -- 172.21.15.76:0/16136401 >> 172.21.15.76:0/16136401 conn(0x7f8d340fbc40 msgr2=0x7f8d340fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 -- 172.21.15.76:0/16136401 shutdown_connections 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 -- 172.21.15.76:0/16136401 wait complete. 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 Processor -- start 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 -- start start 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d341006b0 0x7f8d34104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d341018a0 0x7f8d34102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d34102260 0x7f8d341031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34103920 con 0x7f8d341018a0 2024-03-23T15:43:10.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34103a90 con 0x7f8d34102260 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d3968d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8d341a17c0 con 0x7f8d341006b0 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d327fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d341018a0 0x7f8d34102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.690+0000 7f8d327fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d341018a0 0x7f8d34102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35450/0 (socket says 172.21.15.76:35450) 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d327fc640 1 -- 172.21.15.76:0/3695965248 learned_addr learned my addr 172.21.15.76:0/3695965248 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d337fe640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d34102260 0x7f8d341031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d32ffd640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d341006b0 0x7f8d34104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d327fc640 1 -- 172.21.15.76:0/3695965248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d341006b0 msgr2=0x7f8d34104640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d327fc640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d341006b0 0x7f8d34104640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d327fc640 1 -- 172.21.15.76:0/3695965248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d34102260 msgr2=0x7f8d341031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d327fc640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d34102260 0x7f8d341031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d327fc640 1 -- 172.21.15.76:0/3695965248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c009900 con 0x7f8d341018a0 2024-03-23T15:43:10.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d327fc640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d341018a0 0x7f8d34102c80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400d980 tx=0x7f8d2400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:10.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d1bfff640 1 -- 172.21.15.76:0/3695965248 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24009a30 con 0x7f8d341018a0 2024-03-23T15:43:10.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d341a1a70 con 0x7f8d341018a0 2024-03-23T15:43:10.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d1bfff640 1 -- 172.21.15.76:0/3695965248 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d24009b90 con 0x7f8d341018a0 2024-03-23T15:43:10.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d1bfff640 1 -- 172.21.15.76:0/3695965248 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24004a00 con 0x7f8d341018a0 2024-03-23T15:43:10.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.691+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341a1fd0 con 0x7f8d341018a0 2024-03-23T15:43:10.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.692+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3410c050 con 0x7f8d341018a0 2024-03-23T15:43:10.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.693+0000 7f8d1bfff640 1 -- 172.21.15.76:0/3695965248 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f8d24009d00 con 0x7f8d341018a0 2024-03-23T15:43:10.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.694+0000 7f8d1bfff640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8d1c0a20c0 0x7f8d1c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:10.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.694+0000 7f8d32ffd640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8d1c0a20c0 0x7f8d1c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:10.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.694+0000 7f8d1bfff640 1 -- 172.21.15.76:0/3695965248 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8d240cd610 con 0x7f8d341018a0 2024-03-23T15:43:10.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.694+0000 7f8d32ffd640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8d1c0a20c0 0x7f8d1c0a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d34101700 tx=0x7f8d28005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:10.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.698+0000 7f8d1bfff640 1 -- 172.21.15.76:0/3695965248 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f8d240965c0 con 0x7f8d341018a0 2024-03-23T15:43:10.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:10 smithi076 ceph-mon[32176]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:43:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:10 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2075553750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:10 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2585344968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:10.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:10 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1629332945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:10.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.875+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d3410c1e0 con 0x7f8d341018a0 2024-03-23T15:43:10.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.877+0000 7f8d1bfff640 1 -- 172.21.15.76:0/3695965248 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22729 (secure 0 0 0) 0x7f8d24099c80 con 0x7f8d341018a0 2024-03-23T15:43:10.879 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:10.879 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":87,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:42:16.614449+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:43:10.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8d1c0a20c0 msgr2=0x7f8d1c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8d1c0a20c0 0x7f8d1c0a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d34101700 tx=0x7f8d28005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:10.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d341018a0 msgr2=0x7f8d34102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d341018a0 0x7f8d34102c80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400d980 tx=0x7f8d2400de40 comp rx=0 tx=0).stop 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 shutdown_connections 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f8d1c0a20c0 0x7f8d1c0a4580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d34102260 0x7f8d341031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d341018a0 0x7f8d34102c80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 --2- 172.21.15.76:0/3695965248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d341006b0 0x7f8d34104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 >> 172.21.15.76:0/3695965248 conn(0x7f8d340fbc40 msgr2=0x7f8d340fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 shutdown_connections 2024-03-23T15:43:10.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.879+0000 7f8d3968d640 1 -- 172.21.15.76:0/3695965248 wait complete. 2024-03-23T15:43:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:10 smithi183 ceph-mon[36296]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:43:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:10 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2075553750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:10 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2585344968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:10.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:10 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1629332945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:10.893 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-23T15:41:17.990893+0000', 'modified': '2024-03-23T15:41:22.694125+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24479, 'mds_1': 24463, 'mds_2': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1781188743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1781188743}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1781188743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/1421961871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 1421961871}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 1421961871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6837/26718199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 26718199}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 26718199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.183:6837/3732793280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 3732793280}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 3732793280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.192:6839/2120704815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 2120704815}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 2120704815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6839/1020785546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1020785546}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1020785546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24479, 24449, 24463]} 2024-03-23T15:43:10.893 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-23T15:43:10.893 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/4238892069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec100630 msgr2=0x7f3eec100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/4238892069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec100630 0x7f3eec100a50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009900 tx=0x7f3edc02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/4238892069 shutdown_connections 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/4238892069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eec1021e0 0x7f3eec108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/4238892069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec101820 0x7f3eec101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/4238892069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec100630 0x7f3eec100a50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/4238892069 >> 172.21.15.76:0/4238892069 conn(0x7f3eec0fbbc0 msgr2=0x7f3eec0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/4238892069 shutdown_connections 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.994+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/4238892069 wait complete. 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 Processor -- start 2024-03-23T15:43:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 -- start start 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec100630 0x7f3eec1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec101820 0x7f3eec196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eec1021e0 0x7f3eec19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec19e1c0 con 0x7f3eec101820 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec19e330 con 0x7f3eec100630 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef2dc9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec19e4f0 con 0x7f3eec1021e0 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef0b65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec100630 0x7f3eec1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec101820 0x7f3eec196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef0b65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec100630 0x7f3eec1966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:53174/0 (socket says 172.21.15.76:53174) 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef0b65640 1 -- 172.21.15.76:0/1029615660 learned_addr learned my addr 172.21.15.76:0/1029615660 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.995+0000 7f3ef1366640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eec1021e0 0x7f3eec19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3eebfff640 1 -- 172.21.15.76:0/1029615660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eec1021e0 msgr2=0x7f3eec19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3eebfff640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eec1021e0 0x7f3eec19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3eebfff640 1 -- 172.21.15.76:0/1029615660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec100630 msgr2=0x7f3eec1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3eebfff640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec100630 0x7f3eec1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3eebfff640 1 -- 172.21.15.76:0/1029615660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc009560 con 0x7f3eec101820 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec100630 0x7f3eec1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3eebfff640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec101820 0x7f3eec196c10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000e9f0 tx=0x7f3ee000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3ee9ffb640 1 -- 172.21.15.76:0/1029615660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000ccc0 con 0x7f3eec101820 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec19e7d0 con 0x7f3eec101820 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3ee9ffb640 1 -- 172.21.15.76:0/1029615660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee000ce20 con 0x7f3eec101820 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3ee9ffb640 1 -- 172.21.15.76:0/1029615660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0018920 con 0x7f3eec101820 2024-03-23T15:43:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.996+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec19ed30 con 0x7f3eec101820 2024-03-23T15:43:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.998+0000 7f3ee9ffb640 1 -- 172.21.15.76:0/1029615660 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7f3ee0018a80 con 0x7f3eec101820 2024-03-23T15:43:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.998+0000 7f3ef59aa740 1 Processor -- start 2024-03-23T15:43:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef59aa740 1 -- start start 2024-03-23T15:43:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ee9ffb640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3ed40a20c0 0x7f3ed40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea925c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925c870 0x559ea925ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea9266820 0x559ea9268c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559ea91cb180 con 0x559ea925c870 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ea91c86b0 con 0x559ea9266820 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559ea91c3e90 con 0x559ea925dbe0 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef1366640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3ed40a20c0 0x7f3ed40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ee9ffb640 1 -- 172.21.15.76:0/1029615660 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3ee0014070 con 0x7f3eec101820 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925c870 0x559ea925ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925c870 0x559ea925ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35478/0 (socket says 172.21.15.76:35478) 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 -- 172.21.15.76:0/459977771 learned_addr learned my addr 172.21.15.76:0/459977771 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea925c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef1366640 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea9266820 0x559ea9268c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 -- 172.21.15.76:0/459977771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 msgr2=0x559ea925c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea925c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 -- 172.21.15.76:0/459977771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea9266820 msgr2=0x559ea9268c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea9266820 0x559ea9268c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 -- 172.21.15.76:0/459977771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea9180d80 con 0x559ea925c870 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef1366640 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea9266820 0x559ea9268c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ef1366640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3ed40a20c0 0x7f3ed40a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009fd0 tx=0x7f3ee4013b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3eebfff640 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925c870 0x559ea925ccf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00a2ec0 tx=0x7f3ee0049d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a725809a81aedac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ed2ffd640 1 -- 172.21.15.76:0/459977771 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00a33f0 con 0x559ea925c870 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ed2ffd640 1 -- 172.21.15.76:0/459977771 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee0099000 con 0x559ea925c870 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:10.999+0000 7f3ed2ffd640 1 -- 172.21.15.76:0/459977771 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0049ee0 con 0x559ea925c870 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 -- 172.21.15.76:0/459977771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925c870 msgr2=0x559ea925ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925c870 0x559ea925ccf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00a2ec0 tx=0x7f3ee0049d00 comp rx=0 tx=0).stop 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 -- 172.21.15.76:0/459977771 shutdown_connections 2024-03-23T15:43:11.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea9266820 0x559ea9268c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925c870 0x559ea925ccf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/459977771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea925c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 -- 172.21.15.76:0/459977771 >> 172.21.15.76:0/459977771 conn(0x559ea9257ad0 msgr2=0x559ea9257ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 -- 172.21.15.76:0/459977771 shutdown_connections 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 -- 172.21.15.76:0/459977771 wait complete. 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.000+0000 7f3ef59aa740 1 -- start start 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 1 Processor -- start 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925dbe0 0x559ea926adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea9266820 0x559ea9271e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559ea9168d50 con 0x559ea925dbe0 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ea8e97310 con 0x559ea925c870 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559ea91498f0 con 0x559ea9266820 2024-03-23T15:43:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef59aa740 10 client.0 _ll_register_callbacks cb 0x559ea8ed4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925dbe0 0x559ea926adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925dbe0 0x559ea926adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35490/0 (socket says 172.21.15.76:35490) 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 -- 172.21.15.76:0/222056018 learned_addr learned my addr 172.21.15.76:0/222056018 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef1366640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea9266820 0x559ea9271e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 -- 172.21.15.76:0/222056018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea9266820 msgr2=0x559ea9271e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea9266820 0x559ea9271e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 -- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 msgr2=0x559ea926a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 -- 172.21.15.76:0/222056018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec19ed30 con 0x559ea925dbe0 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3ef1366640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea9266820 0x559ea9271e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.001+0000 7f3eebfff640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925dbe0 0x559ea926adc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3ee009ac40 tx=0x7f3ee0099170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845727025a299fde server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ecbfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:11.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ecbfff640 1 -- 172.21.15.76:0/222056018 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00a33f0 con 0x559ea925dbe0 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ecbfff640 1 -- 172.21.15.76:0/222056018 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3ee0096a50 con 0x559ea925dbe0 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559ea8ec36c0 con 0x559ea925dbe0 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ecbfff640 1 -- 172.21.15.76:0/222056018 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3ee0096c10 con 0x559ea925dbe0 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 10 client.15240 fetch_fsmap learned FSMap version 21 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ea91c2510 con 0x559ea925dbe0 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ecbfff640 1 -- 172.21.15.76:0/222056018 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee00b9080 con 0x559ea925dbe0 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ecbfff640 1 -- 172.21.15.76:0/222056018 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3ee00ba460 con 0x559ea925dbe0 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 10 client.15240 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 10 client.15240 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 20 client.15240 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea9276e10 0x559ea9278280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 4 client.15240 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea8f14270 con 0x559ea9276e10 2024-03-23T15:43:11.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.002+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea9276e10 0x559ea9278280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.003+0000 7f3ecbfff640 10 client.15240 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.003+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea9276e10 0x559ea9278280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34e18e493503774a server_cookie=1983fde1eb126527 in_seq=0 out_seq=0 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.003+0000 7f3ecbfff640 1 -- 172.21.15.76:0/222056018 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559ea8f14270 con 0x559ea9276e10 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.003+0000 7f3ecbfff640 10 client.15240 handle_command_reply: tid=0 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.003+0000 7f3ef59aa740 1 client.15240 shutdown 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925dbe0 msgr2=0x559ea926adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925dbe0 0x559ea926adc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3ee009ac40 tx=0x7f3ee0099170 comp rx=0 tx=0).stop 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 shutdown_connections 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea9266820 0x559ea9271e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea9276e10 0x559ea9278280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea925dbe0 0x559ea926adc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/222056018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ecbfff640 0 client.15240 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:11.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 >> 172.21.15.76:0/222056018 conn(0x559ea9257ad0 msgr2=0x559ea9256ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 shutdown_connections 2024-03-23T15:43:11.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 1 -- 172.21.15.76:0/222056018 wait complete. 2024-03-23T15:43:11.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.004+0000 7f3ef59aa740 20 client.15240 trim_cache size 0 max 16384 2024-03-23T15:43:11.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 Processor -- start 2024-03-23T15:43:11.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 -- start start 2024-03-23T15:43:11.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925c870 0x559ea9275f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925dbe0 0x559ea9271dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea9252810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559ea9257990 con 0x559ea9266820 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ea9269810 con 0x559ea925dbe0 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559ea92784b0 con 0x559ea925c870 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea9252810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea9252810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35506/0 (socket says 172.21.15.76:35506) 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 -- 172.21.15.76:0/2680509463 learned_addr learned my addr 172.21.15.76:0/2680509463 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3eebfff640 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925dbe0 0x559ea9271dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925c870 0x559ea9275f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 -- 172.21.15.76:0/2680509463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925c870 msgr2=0x559ea9275f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925c870 0x559ea9275f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 -- 172.21.15.76:0/2680509463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925dbe0 msgr2=0x559ea9271dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925dbe0 0x559ea9271dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 -- 172.21.15.76:0/2680509463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea915ea80 con 0x559ea9266820 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ef1366640 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea9252810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4003aa0 tx=0x7f3ee401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881d62b66207574 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ed27fc640 1 -- 172.21.15.76:0/2680509463 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4018050 con 0x559ea9266820 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ed27fc640 1 -- 172.21.15.76:0/2680509463 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee4012190 con 0x559ea9266820 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.022+0000 7f3ed27fc640 1 -- 172.21.15.76:0/2680509463 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4010d90 con 0x559ea9266820 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 -- 172.21.15.76:0/2680509463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 msgr2=0x559ea9252810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea9252810 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4003aa0 tx=0x7f3ee401f000 comp rx=0 tx=0).stop 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 -- 172.21.15.76:0/2680509463 shutdown_connections 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea9252810 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925dbe0 0x559ea9271dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/2680509463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925c870 0x559ea9275f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 -- 172.21.15.76:0/2680509463 >> 172.21.15.76:0/2680509463 conn(0x559ea9257ad0 msgr2=0x559ea9256ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 -- 172.21.15.76:0/2680509463 shutdown_connections 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 -- 172.21.15.76:0/2680509463 wait complete. 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 -- start start 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.023+0000 7f3ef59aa740 1 Processor -- start 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea92d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef59aa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea92db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559ea92db6c0 con 0x559ea9266820 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ea92db830 con 0x559ea925c870 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef59aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559ea92db9f0 con 0x559ea925dbe0 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef59aa740 10 client.0 _ll_register_callbacks cb 0x7f3ee40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef0b65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea92db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea92db0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35512/0 (socket says 172.21.15.76:35512) 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 -- 172.21.15.76:0/1988011206 learned_addr learned my addr 172.21.15.76:0/1988011206 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef0b65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:53200/0 (socket says 172.21.15.76:53200) 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3eebfff640 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea92d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 -- 172.21.15.76:0/1988011206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 msgr2=0x559ea92d8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea92d8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 -- 172.21.15.76:0/1988011206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 msgr2=0x559ea926fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 -- 172.21.15.76:0/1988011206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee400fdd0 con 0x559ea9266820 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ef1366640 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea92db0b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4012070 tx=0x7f3ee4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47c8079c1bf0c20 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ecbfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.024+0000 7f3ecbfff640 1 -- 172.21.15.76:0/1988011206 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4013460 con 0x559ea9266820 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ecbfff640 1 -- 172.21.15.76:0/1988011206 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3ee4004a40 con 0x559ea9266820 2024-03-23T15:43:11.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee0099000 con 0x559ea9266820 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ecbfff640 1 -- 172.21.15.76:0/1988011206 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3ee4018020 con 0x559ea9266820 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 10 client.15246 fetch_fsmap learned FSMap version 21 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ea91847e0 con 0x559ea9266820 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ecbfff640 1 -- 172.21.15.76:0/1988011206 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee0099000 con 0x559ea9266820 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ecbfff640 1 -- 172.21.15.76:0/1988011206 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3ee4005800 con 0x559ea9266820 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 10 client.15246 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 10 client.15246 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 20 client.15246 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea92e11d0 0x559ea92dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 4 client.15246 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "status"}) -- 0x559ea8f00610 con 0x559ea92e11d0 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.025+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea92e11d0 0x559ea92dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ecbfff640 10 client.15246 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef0b65640 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea92e11d0 0x559ea92dfee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=195677ae123cfdf0 server_cookie=4fa77e8fbc2f5c48 in_seq=0 out_seq=0 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ecbfff640 1 -- 172.21.15.76:0/1988011206 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559ea8f00610 con 0x559ea92e11d0 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ecbfff640 10 client.15246 handle_command_reply: tid=0 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 client.15246 shutdown 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 msgr2=0x559ea92db0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea92db0b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4012070 tx=0x7f3ee4003f60 comp rx=0 tx=0).stop 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 shutdown_connections 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x559ea92e11d0 0x559ea92dfee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559ea9266820 0x559ea92db0b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559ea925dbe0 0x559ea92d8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ecbfff640 0 client.15246 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 --2- 172.21.15.76:0/1988011206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ea925c870 0x559ea926fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 >> 172.21.15.76:0/1988011206 conn(0x559ea9257ad0 msgr2=0x559ea9256ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 shutdown_connections 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 1 -- 172.21.15.76:0/1988011206 wait complete. 2024-03-23T15:43:11.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.026+0000 7f3ef59aa740 20 client.15246 trim_cache size 0 max 16384 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 24449, 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 109.33460691400001, 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 21, 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 87, 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T15:43:11.036 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 130.09611034400001 2024-03-23T15:43:11.037 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:11.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3ed40a20c0 msgr2=0x7f3ed40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3ed40a20c0 0x7f3ed40a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009fd0 tx=0x7f3ee4013b80 comp rx=0 tx=0).stop 2024-03-23T15:43:11.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec101820 msgr2=0x7f3eec196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec101820 0x7f3eec196c10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000e9f0 tx=0x7f3ee000eeb0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 shutdown_connections 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eec1021e0 0x7f3eec19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eec101820 0x7f3eec196c10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f3ed40a20c0 0x7f3ed40a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 --2- 172.21.15.76:0/1029615660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eec100630 0x7f3eec1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 >> 172.21.15.76:0/1029615660 conn(0x7f3eec0fbbc0 msgr2=0x7f3eec104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 shutdown_connections 2024-03-23T15:43:11.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.027+0000 7f3ef2dc9640 1 -- 172.21.15.76:0/1029615660 wait complete. 2024-03-23T15:43:11.040 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-23T15:43:11.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:10 smithi192 ceph-mon[36653]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-03-23T15:43:11.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:10 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2075553750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:11.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:10 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2585344968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:11.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:10 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1629332945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 -- 172.21.15.76:0/3757664153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8581021c0 msgr2=0x7fc858108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 --2- 172.21.15.76:0/3757664153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8581021c0 0x7fc858108a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc850009860 tx=0x7fc85002bb70 comp rx=0 tx=0).stop 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 -- 172.21.15.76:0/3757664153 shutdown_connections 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 --2- 172.21.15.76:0/3757664153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8581021c0 0x7fc858108a50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 --2- 172.21.15.76:0/3757664153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858101800 0x7fc858101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 --2- 172.21.15.76:0/3757664153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc858100610 0x7fc858100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 -- 172.21.15.76:0/3757664153 >> 172.21.15.76:0/3757664153 conn(0x7fc8580fbba0 msgr2=0x7fc8580fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 -- 172.21.15.76:0/3757664153 shutdown_connections 2024-03-23T15:43:11.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.140+0000 7fc85d488640 1 -- 172.21.15.76:0/3757664153 wait complete. 2024-03-23T15:43:11.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 Processor -- start 2024-03-23T15:43:11.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 -- start start 2024-03-23T15:43:11.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858100610 0x7fc8581966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc858101800 0x7fc858196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8581021c0 0x7fc85819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc85819e1a0 con 0x7fc858100610 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc85819e310 con 0x7fc858101800 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc85d488640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc85819e4d0 con 0x7fc8581021c0 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc856ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858100610 0x7fc8581966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc856ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858100610 0x7fc8581966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35518/0 (socket says 172.21.15.76:35518) 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc856ffd640 1 -- 172.21.15.76:0/2111940770 learned_addr learned my addr 172.21.15.76:0/2111940770 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc8567fc640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc858101800 0x7fc858196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.141+0000 7fc8577fe640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8581021c0 0x7fc85819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc856ffd640 1 -- 172.21.15.76:0/2111940770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8581021c0 msgr2=0x7fc85819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc856ffd640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8581021c0 0x7fc85819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc856ffd640 1 -- 172.21.15.76:0/2111940770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc858101800 msgr2=0x7fc858196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc856ffd640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc858101800 0x7fc858196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc856ffd640 1 -- 172.21.15.76:0/2111940770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c009560 con 0x7fc858100610 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc8567fc640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc858101800 0x7fc858196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc856ffd640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858100610 0x7fc8581966b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc84800eb70 tx=0x7fc84800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc83ffff640 1 -- 172.21.15.76:0/2111940770 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848003b70 con 0x7fc858100610 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc83ffff640 1 -- 172.21.15.76:0/2111940770 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84800ee70 con 0x7fc858100610 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc850009490 con 0x7fc858100610 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc83ffff640 1 -- 172.21.15.76:0/2111940770 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8480107f0 con 0x7fc858100610 2024-03-23T15:43:11.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.142+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85819eae0 con 0x7fc858100610 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.144+0000 7fc83ffff640 1 -- 172.21.15.76:0/2111940770 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fc848004020 con 0x7fc858100610 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.144+0000 7fc86006c740 1 Processor -- start 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc86006c740 1 -- start start 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc83ffff640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8400a20c0 0x7fc8400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc86006c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9137770 0x562cc9137b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc86006c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9138160 0x562cc913cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc86006c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc913d2d0 0x562cc913f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc86006c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562cc90a4180 con 0x562cc913d2d0 2024-03-23T15:43:11.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc86006c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cc90a16b0 con 0x562cc9137770 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc86006c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562cc909ce90 con 0x562cc9138160 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc8577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc913d2d0 0x562cc913f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9137770 0x562cc9137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc8577fe640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8400a20c0 0x7fc8400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc83ffff640 1 -- 172.21.15.76:0/2111940770 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc848014070 con 0x7fc858100610 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9137770 0x562cc9137b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:53226/0 (socket says 172.21.15.76:53226) 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 -- 172.21.15.76:0/251161968 learned_addr learned my addr 172.21.15.76:0/251161968 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc8567fc640 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9138160 0x562cc913cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 -- 172.21.15.76:0/251161968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9138160 msgr2=0x562cc913cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9138160 0x562cc913cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 -- 172.21.15.76:0/251161968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc913d2d0 msgr2=0x562cc913f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc913d2d0 0x562cc913f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 -- 172.21.15.76:0/251161968 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc9059d80 con 0x562cc9137770 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc8577fe640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8400a20c0 0x7fc8400a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc850005940 tx=0x7fc850034be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.145+0000 7fc856ffd640 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9137770 0x562cc9137b90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8480cbda0 tx=0x7fc848096bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a5b321f893976a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc83d7fa640 1 -- 172.21.15.76:0/251161968 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84809ab10 con 0x562cc9137770 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc83d7fa640 1 -- 172.21.15.76:0/251161968 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84809a930 con 0x562cc9137770 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc83d7fa640 1 -- 172.21.15.76:0/251161968 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8480cbeb0 con 0x562cc9137770 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 -- 172.21.15.76:0/251161968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9137770 msgr2=0x562cc9137b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9137770 0x562cc9137b90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8480cbda0 tx=0x7fc848096bf0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 -- 172.21.15.76:0/251161968 shutdown_connections 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc913d2d0 0x562cc913f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9138160 0x562cc913cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 --2- 172.21.15.76:0/251161968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9137770 0x562cc9137b90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 -- 172.21.15.76:0/251161968 >> 172.21.15.76:0/251161968 conn(0x562cc9130ad0 msgr2=0x562cc9130ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 -- 172.21.15.76:0/251161968 shutdown_connections 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 -- 172.21.15.76:0/251161968 wait complete. 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.146+0000 7fc86006c740 1 -- start start 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 1 Processor -- start 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9141680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc9148c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562cc9041d50 con 0x562cc9138160 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cc8d70310 con 0x562cc913d2d0 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562cc90228f0 con 0x562cc9137770 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc86006c740 10 client.0 _ll_register_callbacks cb 0x562cc8dadac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35536/0 (socket says 172.21.15.76:35536) 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 -- 172.21.15.76:0/4242895149 learned_addr learned my addr 172.21.15.76:0/4242895149 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc856ffd640 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8577fe640 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc9148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 -- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 msgr2=0x562cc9141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 -- 172.21.15.76:0/4242895149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 msgr2=0x562cc9148c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc9148c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 -- 172.21.15.76:0/4242895149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85819eae0 con 0x562cc9138160 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc8567fc640 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9141bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005fd0 tx=0x7fc84c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4827a96127277a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc83aff5640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.147+0000 7fc83aff5640 1 -- 172.21.15.76:0/4242895149 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c017070 con 0x562cc9138160 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc83aff5640 1 -- 172.21.15.76:0/4242895149 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc84c010430 con 0x562cc9138160 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cc8d9c6c0 con 0x562cc9138160 2024-03-23T15:43:11.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc83aff5640 1 -- 172.21.15.76:0/4242895149 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc84c0122e0 con 0x562cc9138160 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 10 client.15258 fetch_fsmap learned FSMap version 21 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562cc909b510 con 0x562cc9138160 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc83aff5640 1 -- 172.21.15.76:0/4242895149 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562cc8d9c6c0 con 0x562cc9138160 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc83aff5640 1 -- 172.21.15.76:0/4242895149 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc84c024020 con 0x562cc9138160 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 10 client.15258 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 20 client.15258 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc914f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 4 client.15258 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cc8ded270 con 0x562cc914dfa0 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.148+0000 7fc856ffd640 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc914f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.149+0000 7fc83aff5640 10 client.15258 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.149+0000 7fc856ffd640 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc914f410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1944c12d7cba7b39 server_cookie=7a1375f73eff8582 in_seq=0 out_seq=0 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.149+0000 7fc83aff5640 1 -- 172.21.15.76:0/4242895149 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562cc8ded270 con 0x562cc914dfa0 2024-03-23T15:43:11.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.149+0000 7fc83aff5640 10 client.15258 handle_command_reply: tid=0 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.149+0000 7fc86006c740 1 client.15258 shutdown 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 msgr2=0x562cc9141bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9141bc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005fd0 tx=0x7fc84c00f790 comp rx=0 tx=0).stop 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 shutdown_connections 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc914f410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc9148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9141bc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc83aff5640 0 client.15258 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 --2- 172.21.15.76:0/4242895149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 >> 172.21.15.76:0/4242895149 conn(0x562cc9130ad0 msgr2=0x562cc912fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 shutdown_connections 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 1 -- 172.21.15.76:0/4242895149 wait complete. 2024-03-23T15:43:11.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.150+0000 7fc86006c740 20 client.15258 trim_cache size 0 max 16384 2024-03-23T15:43:11.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.167+0000 7fc86006c740 1 Processor -- start 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc86006c740 1 -- start start 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc86006c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc86006c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc86006c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc912b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc86006c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562cc9131840 con 0x562cc9138160 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc86006c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cc914e380 con 0x562cc913d2d0 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc86006c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562cc8ded270 con 0x562cc9137770 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35542/0 (socket says 172.21.15.76:35542) 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 -- 172.21.15.76:0/118210257 learned_addr learned my addr 172.21.15.76:0/118210257 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc856ffd640 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8577fe640 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc912b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 -- 172.21.15.76:0/118210257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 msgr2=0x562cc9130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9130460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 -- 172.21.15.76:0/118210257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 msgr2=0x562cc912b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc912b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 -- 172.21.15.76:0/118210257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc9037a80 con 0x562cc9138160 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc856ffd640 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9130460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8577fe640 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc912b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc8567fc640 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9148bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc84c001eb0 tx=0x7fc84c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d7b736511c96dc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc83cff9640 1 -- 172.21.15.76:0/118210257 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c02b070 con 0x562cc9138160 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc83cff9640 1 -- 172.21.15.76:0/118210257 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c0122e0 con 0x562cc9138160 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.168+0000 7fc83cff9640 1 -- 172.21.15.76:0/118210257 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c017040 con 0x562cc9138160 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- 172.21.15.76:0/118210257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 msgr2=0x562cc9148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9148bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc84c001eb0 tx=0x7fc84c00fc90 comp rx=0 tx=0).stop 2024-03-23T15:43:11.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- 172.21.15.76:0/118210257 shutdown_connections 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc913d2d0 0x562cc912b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9138160 0x562cc9148bc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 --2- 172.21.15.76:0/118210257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc9137770 0x562cc9130460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- 172.21.15.76:0/118210257 >> 172.21.15.76:0/118210257 conn(0x562cc9130ad0 msgr2=0x562cc914cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- 172.21.15.76:0/118210257 shutdown_connections 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- 172.21.15.76:0/118210257 wait complete. 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- start start 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 Processor -- start 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9137770 0x562cc9146d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9138160 0x562cc91472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc913d2d0 0x562cc91478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562cc9147fd0 con 0x562cc9137770 2024-03-23T15:43:11.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cc9148140 con 0x562cc9138160 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.169+0000 7fc86006c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562cc9148300 con 0x562cc913d2d0 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9137770 0x562cc9146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc86006c740 10 client.0 _ll_register_callbacks cb 0x7fc84c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9137770 0x562cc9146d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35554/0 (socket says 172.21.15.76:35554) 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 -- 172.21.15.76:0/1607967417 learned_addr learned my addr 172.21.15.76:0/1607967417 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc8567fc640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9138160 0x562cc91472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc8577fe640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc913d2d0 0x562cc91478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 -- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc913d2d0 msgr2=0x562cc91478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc913d2d0 0x562cc91478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 -- 172.21.15.76:0/1607967417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9138160 msgr2=0x562cc91472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9138160 0x562cc91472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 -- 172.21.15.76:0/1607967417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc909b510 con 0x562cc9137770 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc8577fe640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc913d2d0 0x562cc91478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc856ffd640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9137770 0x562cc9146d60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc8480979a0 tx=0x7fc848009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504fb266ff559fc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc83aff5640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc83aff5640 1 -- 172.21.15.76:0/1607967417 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848003e60 con 0x562cc9137770 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc83aff5640 1 -- 172.21.15.76:0/1607967417 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc84809b6d0 con 0x562cc9137770 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84809a930 con 0x562cc9137770 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc83aff5640 1 -- 172.21.15.76:0/1607967417 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc84809afd0 con 0x562cc9137770 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc86006c740 10 client.15264 fetch_fsmap learned FSMap version 21 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562cc905d7e0 con 0x562cc9137770 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.170+0000 7fc83aff5640 1 -- 172.21.15.76:0/1607967417 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84809a930 con 0x562cc9137770 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc83aff5640 1 -- 172.21.15.76:0/1607967417 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc84801f060 con 0x562cc9137770 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 10 client.15264 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 20 client.15264 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc91b7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 4 client.15264 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T15:43:11.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "status"}) -- 0x562cc8dd9610 con 0x562cc914dfa0 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc8567fc640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc91b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc83aff5640 10 client.15264 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc8567fc640 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc91b7170 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bb4e20731f39b27 server_cookie=bfb6be0c6e1626f4 in_seq=0 out_seq=0 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc83aff5640 1 -- 172.21.15.76:0/1607967417 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x562cc8dd9610 con 0x562cc914dfa0 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc83aff5640 10 client.15264 handle_command_reply: tid=0 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 client.15264 shutdown 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9137770 msgr2=0x562cc9146d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9137770 0x562cc9146d60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc8480979a0 tx=0x7fc848009200 comp rx=0 tx=0).stop 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 shutdown_connections 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x562cc914dfa0 0x562cc91b7170 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562cc913d2d0 0x562cc91478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cc9138160 0x562cc91472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 --2- 172.21.15.76:0/1607967417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562cc9137770 0x562cc9146d60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc83aff5640 0 client.15264 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.171+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 >> 172.21.15.76:0/1607967417 conn(0x562cc9130ad0 msgr2=0x562cc912b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 shutdown_connections 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc86006c740 1 -- 172.21.15.76:0/1607967417 wait complete. 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc86006c740 20 client.15264 trim_cache size 0 max 16384 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 24463, 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-23T15:43:11.182 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-23T15:43:11.183 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-23T15:43:11.183 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 110.4935024, 2024-03-23T15:43:11.183 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 21, 2024-03-23T15:43:11.183 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 87, 2024-03-23T15:43:11.183 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T15:43:11.183 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 127.33044365400001 2024-03-23T15:43:11.183 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8400a20c0 msgr2=0x7fc8400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8400a20c0 0x7fc8400a4580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc850005940 tx=0x7fc850034be0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858100610 msgr2=0x7fc8581966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858100610 0x7fc8581966b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc84800eb70 tx=0x7fc84800c6a0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 shutdown_connections 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8581021c0 0x7fc85819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc8400a20c0 0x7fc8400a4580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc858101800 0x7fc858196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 --2- 172.21.15.76:0/2111940770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc858100610 0x7fc8581966b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 >> 172.21.15.76:0/2111940770 conn(0x7fc8580fbba0 msgr2=0x7fc858104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 shutdown_connections 2024-03-23T15:43:11.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.172+0000 7fc85d488640 1 -- 172.21.15.76:0/2111940770 wait complete. 2024-03-23T15:43:11.186 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 -- 172.21.15.76:0/474594297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 msgr2=0x7fc60c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 --2- 172.21.15.76:0/474594297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c100a50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009900 tx=0x7fc5fc02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 -- 172.21.15.76:0/474594297 shutdown_connections 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 --2- 172.21.15.76:0/474594297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc60c1021e0 0x7fc60c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 --2- 172.21.15.76:0/474594297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc60c101820 0x7fc60c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 --2- 172.21.15.76:0/474594297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c100a50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 -- 172.21.15.76:0/474594297 >> 172.21.15.76:0/474594297 conn(0x7fc60c0fbbc0 msgr2=0x7fc60c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 -- 172.21.15.76:0/474594297 shutdown_connections 2024-03-23T15:43:11.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 -- 172.21.15.76:0/474594297 wait complete. 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.287+0000 7fc611eae640 1 Processor -- start 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc611eae640 1 -- start start 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc611eae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc611eae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc60c101820 0x7fc60c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc611eae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc60c1021e0 0x7fc60c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc611eae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a14d0 con 0x7fc60c100630 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc611eae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a1640 con 0x7fc60c1021e0 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc611eae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a1800 con 0x7fc60c101820 2024-03-23T15:43:11.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35564/0 (socket says 172.21.15.76:35564) 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 -- 172.21.15.76:0/4023828385 learned_addr learned my addr 172.21.15.76:0/4023828385 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60affd640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc60c101820 0x7fc60c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60bfff640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc60c1021e0 0x7fc60c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 -- 172.21.15.76:0/4023828385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc60c101820 msgr2=0x7fc60c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc60c101820 0x7fc60c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 -- 172.21.15.76:0/4023828385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc60c1021e0 msgr2=0x7fc60c1a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc60c1021e0 0x7fc60c1a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 -- 172.21.15.76:0/4023828385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc009560 con 0x7fc60c100630 2024-03-23T15:43:11.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.288+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c1999e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005ab0 tx=0x7fc5fc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.289+0000 7fc608ff9640 1 -- 172.21.15.76:0/4023828385 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0043d0 con 0x7fc60c100630 2024-03-23T15:43:11.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.289+0000 7fc608ff9640 1 -- 172.21.15.76:0/4023828385 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc0052d0 con 0x7fc60c100630 2024-03-23T15:43:11.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.289+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c1a1a80 con 0x7fc60c100630 2024-03-23T15:43:11.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.289+0000 7fc608ff9640 1 -- 172.21.15.76:0/4023828385 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc003d40 con 0x7fc60c100630 2024-03-23T15:43:11.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.289+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c1a1fe0 con 0x7fc60c100630 2024-03-23T15:43:11.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc608ff9640 1 -- 172.21.15.76:0/4023828385 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fc5fc005440 con 0x7fc60c100630 2024-03-23T15:43:11.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 Processor -- start 2024-03-23T15:43:11.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 -- start start 2024-03-23T15:43:11.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc608ff9640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc5f40a2140 0x7fc5f40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cef5560 0x55826cef5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cefaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cefb0c0 0x55826cefd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55826ce64180 con 0x55826cef5f50 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55826ce616b0 con 0x55826cef5560 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc614b6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55826ce5ce90 con 0x55826cefb0c0 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc60bfff640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc5f40a2140 0x7fc5f40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc60affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cefaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc60affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cefaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35578/0 (socket says 172.21.15.76:35578) 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc60affd640 1 -- 172.21.15.76:0/2578375059 learned_addr learned my addr 172.21.15.76:0/2578375059 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cef5560 0x55826cef5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc608ff9640 1 -- 172.21.15.76:0/4023828385 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc5fc0efd60 con 0x7fc60c100630 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.291+0000 7fc60bfff640 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cefb0c0 0x55826cefd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc60affd640 1 -- 172.21.15.76:0/2578375059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cefb0c0 msgr2=0x55826cefd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc60affd640 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cefb0c0 0x55826cefd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc60affd640 1 -- 172.21.15.76:0/2578375059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cef5560 msgr2=0x55826cef5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc60affd640 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cef5560 0x55826cef5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc60affd640 1 -- 172.21.15.76:0/2578375059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826ce19d80 con 0x55826cef5f50 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc60affd640 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cefaa50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc600009c90 tx=0x7fc600011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f406037e9d8a8a82 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc60bfff640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc5f40a2140 0x7fc5f40a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc604005fd0 tx=0x7fc604009bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc5f1ffb640 1 -- 172.21.15.76:0/2578375059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600019630 con 0x55826cef5f50 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc5f1ffb640 1 -- 172.21.15.76:0/2578375059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc600009e40 con 0x55826cef5f50 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.292+0000 7fc5f1ffb640 1 -- 172.21.15.76:0/2578375059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60000eb30 con 0x55826cef5f50 2024-03-23T15:43:11.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- 172.21.15.76:0/2578375059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 msgr2=0x55826cefaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cefaa50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc600009c90 tx=0x7fc600011810 comp rx=0 tx=0).stop 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- 172.21.15.76:0/2578375059 shutdown_connections 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cefb0c0 0x55826cefd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cefaa50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 --2- 172.21.15.76:0/2578375059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cef5560 0x55826cef5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- 172.21.15.76:0/2578375059 >> 172.21.15.76:0/2578375059 conn(0x55826cef0ad0 msgr2=0x55826cef0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- 172.21.15.76:0/2578375059 shutdown_connections 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- 172.21.15.76:0/2578375059 wait complete. 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- start start 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 Processor -- start 2024-03-23T15:43:11.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826ceffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf06a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55826ce01d50 con 0x55826cef5560 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55826cb30310 con 0x55826cefb0c0 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55826cde28f0 con 0x55826cef5f50 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc614b6c740 10 client.0 _ll_register_callbacks cb 0x55826cb6dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.293+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35588/0 (socket says 172.21.15.76:35588) 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3568475459 learned_addr learned my addr 172.21.15.76:0/3568475459 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60bfff640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf06a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60affd640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826ceffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3568475459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 msgr2=0x55826ceffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826ceffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 msgr2=0x55826cf06a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf06a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3568475459 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6000098f0 con 0x55826cef5560 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60bfff640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf06a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff500 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ed840 tx=0x7fc5fc0bc900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8b04a543922266 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc5eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc5eaffd640 1 -- 172.21.15.76:0/3568475459 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0bca70 con 0x55826cef5560 2024-03-23T15:43:11.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc5eaffd640 1 -- 172.21.15.76:0/3568475459 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc5fc0c0bc0 con 0x55826cef5560 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55826cb5c6c0 con 0x55826cef5560 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc5eaffd640 1 -- 172.21.15.76:0/3568475459 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc5fc0c5210 con 0x55826cef5560 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc614b6c740 10 client.15276 fetch_fsmap learned FSMap version 21 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55826ce5b510 con 0x55826cef5560 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc5eaffd640 1 -- 172.21.15.76:0/3568475459 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55826cb5c6c0 con 0x55826cef5560 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc5eaffd640 1 -- 172.21.15.76:0/3568475459 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc5fc03c070 con 0x55826cef5560 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc614b6c740 10 client.15276 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc614b6c740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.294+0000 7fc614b6c740 20 client.15276 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.295+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf0bb50 0x55826cf0cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.295+0000 7fc614b6c740 4 client.15276 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:11.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.295+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55826cbad270 con 0x55826cf0bb50 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.295+0000 7fc60affd640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf0bb50 0x55826cf0cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.295+0000 7fc5eaffd640 10 client.15276 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.295+0000 7fc60affd640 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf0bb50 0x55826cf0cfc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d3d11d3d405a59 server_cookie=367a52dad7c2bf7c in_seq=0 out_seq=0 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc5eaffd640 1 -- 172.21.15.76:0/3568475459 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55826cbad270 con 0x55826cf0bb50 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc5eaffd640 10 client.15276 handle_command_reply: tid=0 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 client.15276 shutdown 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 msgr2=0x55826ceff500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff500 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ed840 tx=0x7fc5fc0bc900 comp rx=0 tx=0).stop 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 shutdown_connections 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf0bb50 0x55826cf0cfc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf06a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826ceffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc5eaffd640 0 client.15276 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3568475459 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff500 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 >> 172.21.15.76:0/3568475459 conn(0x55826cef0ad0 msgr2=0x55826ceefce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 shutdown_connections 2024-03-23T15:43:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 1 -- 172.21.15.76:0/3568475459 wait complete. 2024-03-23T15:43:11.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.296+0000 7fc614b6c740 20 client.15276 trim_cache size 0 max 16384 2024-03-23T15:43:11.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 Processor -- start 2024-03-23T15:43:11.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 -- start start 2024-03-23T15:43:11.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826cf06a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826ceff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55826cef6870 con 0x55826cef5560 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55826cef15c0 con 0x55826cefb0c0 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc614b6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55826cefdc10 con 0x55826cef5f50 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35590/0 (socket says 172.21.15.76:35590) 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3550921830 learned_addr learned my addr 172.21.15.76:0/3550921830 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60bfff640 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826ceff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60affd640 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826cf06a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3550921830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 msgr2=0x55826cf06a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826cf06a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3550921830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 msgr2=0x55826ceff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826ceff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 -- 172.21.15.76:0/3550921830 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826cdf7a80 con 0x55826cef5560 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60bfff640 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826ceff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.314+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff110 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0bca00 tx=0x7fc5fc0b74d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74c105ea860c5cb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc5f17fa640 1 -- 172.21.15.76:0/3550921830 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0bc3d0 con 0x55826cef5560 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc5f17fa640 1 -- 172.21.15.76:0/3550921830 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc0c0bc0 con 0x55826cef5560 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc5f17fa640 1 -- 172.21.15.76:0/3550921830 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0bbec0 con 0x55826cef5560 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 -- 172.21.15.76:0/3550921830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 msgr2=0x55826ceff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff110 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0bca00 tx=0x7fc5fc0b74d0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 -- 172.21.15.76:0/3550921830 shutdown_connections 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826ceff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5f50 0x55826cf06a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 --2- 172.21.15.76:0/3550921830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5560 0x55826ceff110 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 -- 172.21.15.76:0/3550921830 >> 172.21.15.76:0/3550921830 conn(0x55826cef0ad0 msgr2=0x55826cef0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 -- 172.21.15.76:0/3550921830 shutdown_connections 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.315+0000 7fc614b6c740 1 -- 172.21.15.76:0/3550921830 wait complete. 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 -- start start 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 Processor -- start 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5560 0x55826cf026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cf02c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf05c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55826cf06190 con 0x55826cef5f50 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55826cf06300 con 0x55826cefb0c0 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55826cf064c0 con 0x55826cef5560 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc614b6c740 10 client.0 _ll_register_callbacks cb 0x7fc5fc0b8d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cf02c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cf02c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35598/0 (socket says 172.21.15.76:35598) 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 -- 172.21.15.76:0/862610739 learned_addr learned my addr 172.21.15.76:0/862610739 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5560 0x55826cf026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60bfff640 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf05c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 -- 172.21.15.76:0/862610739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5560 msgr2=0x55826cf026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5560 0x55826cf026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 -- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 msgr2=0x55826cf05c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf05c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 -- 172.21.15.76:0/862610739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826ce5b510 con 0x55826cef5f50 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc60affd640 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cf02c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc600005fd0 tx=0x7fc600005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55cb7f93ef55661 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.316+0000 7fc5eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc5eaffd640 1 -- 172.21.15.76:0/862610739 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600005460 con 0x55826cef5f50 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc5eaffd640 1 -- 172.21.15.76:0/862610739 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc6000055c0 con 0x55826cef5f50 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc600009e40 con 0x55826cef5f50 2024-03-23T15:43:11.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc5eaffd640 1 -- 172.21.15.76:0/862610739 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc6000035f0 con 0x55826cef5f50 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 10 client.15282 fetch_fsmap learned FSMap version 21 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55826ce1d7e0 con 0x55826cef5f50 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc5eaffd640 1 -- 172.21.15.76:0/862610739 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc600009e40 con 0x55826cef5f50 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc5eaffd640 1 -- 172.21.15.76:0/862610739 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc600037050 con 0x55826cef5f50 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 10 client.15282 fetch_fsmap finished waiting for FSMap version 21 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 20 client.15282 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf76530 0x55826cf75240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 4 client.15282 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T15:43:11.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "status"}) -- 0x55826cb99610 con 0x55826cf76530 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.317+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf76530 0x55826cf75240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc60b7fe640 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf76530 0x55826cf75240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da8211d1b8d11b2 server_cookie=1c9f7145dd0eb44 in_seq=0 out_seq=0 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc5eaffd640 10 client.15282 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc5eaffd640 1 -- 172.21.15.76:0/862610739 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55826cb99610 con 0x55826cf76530 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc5eaffd640 10 client.15282 handle_command_reply: tid=0 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 client.15282 shutdown 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 msgr2=0x55826cf02c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cf02c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc600005fd0 tx=0x7fc600005d30 comp rx=0 tx=0).stop 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 shutdown_connections 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55826cf76530 0x55826cf75240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55826cefb0c0 0x55826cf05c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc5eaffd640 0 client.15282 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:11.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55826cef5f50 0x55826cf02c30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 --2- 172.21.15.76:0/862610739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826cef5560 0x55826cf026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 >> 172.21.15.76:0/862610739 conn(0x55826cef0ad0 msgr2=0x55826ceff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 shutdown_connections 2024-03-23T15:43:11.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 1 -- 172.21.15.76:0/862610739 wait complete. 2024-03-23T15:43:11.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.318+0000 7fc614b6c740 20 client.15282 trim_cache size 0 max 16384 2024-03-23T15:43:11.330 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "89c3e582-e92a-11ee-95cd-87774f69a715", 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 24479, 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 112.666186827, 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 21, 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 87, 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 85, 2024-03-23T15:43:11.331 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 118.85025625 2024-03-23T15:43:11.332 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:11.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc5f40a2140 msgr2=0x7fc5f40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc5f40a2140 0x7fc5f40a4600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc604005fd0 tx=0x7fc604009bf0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 msgr2=0x7fc60c1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c1999e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005ab0 tx=0x7fc5fc005e50 comp rx=0 tx=0).stop 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 shutdown_connections 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fc5f40a2140 0x7fc5f40a4600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc60c1021e0 0x7fc60c1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc60c101820 0x7fc60c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 --2- 172.21.15.76:0/4023828385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc60c100630 0x7fc60c1999e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 >> 172.21.15.76:0/4023828385 conn(0x7fc60c0fbbc0 msgr2=0x7fc60c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 shutdown_connections 2024-03-23T15:43:11.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.319+0000 7fc611eae640 1 -- 172.21.15.76:0/4023828385 wait complete. 2024-03-23T15:43:11.335 INFO:teuthology.run_tasks:Running task sequential... 2024-03-23T15:43:11.347 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-23T15:43:11.347 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-23T15:43:11.347 INFO:teuthology.task.print:Enabling mgr modules 2024-03-23T15:43:11.348 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-03-23T15:43:11.348 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T15:43:11.348 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:43:11.349 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-23T15:43:11.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/2666029879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e8101820 msgr2=0x7fd0e8101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/2666029879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e8101820 0x7fd0e8101ca0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009900 tx=0x7fd0dc02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/2666029879 shutdown_connections 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/2666029879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e81021e0 0x7fd0e8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/2666029879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e8101820 0x7fd0e8101ca0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/2666029879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e8100630 0x7fd0e8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/2666029879 >> 172.21.15.76:0/2666029879 conn(0x7fd0e80fbba0 msgr2=0x7fd0e80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.448+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/2666029879 shutdown_connections 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/2666029879 wait complete. 2024-03-23T15:43:11.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 Processor -- start 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 -- start start 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e8100630 0x7fd0e8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e8101820 0x7fd0e8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e81021e0 0x7fd0e819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e819f200 con 0x7fd0e81021e0 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e819f370 con 0x7fd0e8100630 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ee2a4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e819f530 con 0x7fd0e8101820 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ec841640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e81021e0 0x7fd0e819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e8100630 0x7fd0e8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.449+0000 7fd0ec841640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e81021e0 0x7fd0e819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35616/0 (socket says 172.21.15.76:35616) 2024-03-23T15:43:11.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e8101820 0x7fd0e8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0ec841640 1 -- 172.21.15.76:0/793067048 learned_addr learned my addr 172.21.15.76:0/793067048 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e77fe640 1 -- 172.21.15.76:0/793067048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e8100630 msgr2=0x7fd0e8197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e77fe640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e8100630 0x7fd0e8197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e77fe640 1 -- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e81021e0 msgr2=0x7fd0e819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e77fe640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e81021e0 0x7fd0e819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e77fe640 1 -- 172.21.15.76:0/793067048 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d8009490 con 0x7fd0e8101820 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0ec841640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e81021e0 0x7fd0e819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e77fe640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e8101820 0x7fd0e8197ca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc000c00 tx=0x7fd0dc005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0043d0 con 0x7fd0e8101820 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc009560 con 0x7fd0e8101820 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0dc0052d0 con 0x7fd0e8101820 2024-03-23T15:43:11.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e819fb50 con 0x7fd0e8101820 2024-03-23T15:43:11.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.450+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc003ce0 con 0x7fd0e8101820 2024-03-23T15:43:11.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.451+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e810c040 con 0x7fd0e8101820 2024-03-23T15:43:11.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.452+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 136756+0+0 (secure 0 0 0) 0x7fd0dc005440 con 0x7fd0e8101820 2024-03-23T15:43:11.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.453+0000 7fd0e57fa640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd0d00a2160 0x7fd0d00a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:11.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.453+0000 7fd0e7fff640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd0d00a2160 0x7fd0d00a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:11.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.453+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd0dc0f0230 con 0x7fd0e8101820 2024-03-23T15:43:11.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.453+0000 7fd0e7fff640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd0d00a2160 0x7fd0d00a4620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8101680 tx=0x7fd0d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:11.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.457+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd0dc0b91e0 con 0x7fd0e8101820 2024-03-23T15:43:11.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:11.679+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fd0e819fd70 con 0x7fd0e8101820 2024-03-23T15:43:11.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:11 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1221229275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:11.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:11 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3695965248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:11 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1221229275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:11 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3695965248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:12.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:11 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1221229275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:12.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:11 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3695965248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:12.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.587+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mgrmap(e 19) ==== 136773+0+0 (secure 0 0 0) 0x7fd0dc0b95b0 con 0x7fd0e8101820 2024-03-23T15:43:12.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.588+0000 7fd0e57fa640 1 -- 172.21.15.76:0/793067048 <== mon.1 v2:172.21.15.192:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fd0dc0f1670 con 0x7fd0e8101820 2024-03-23T15:43:12.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd0d00a2160 msgr2=0x7fd0d00a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:12.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd0d00a2160 0x7fd0d00a4620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8101680 tx=0x7fd0d8005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e8101820 msgr2=0x7fd0e8197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e8101820 0x7fd0e8197ca0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc000c00 tx=0x7fd0dc005e50 comp rx=0 tx=0).stop 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 shutdown_connections 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0e81021e0 0x7fd0e819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e8101820 0x7fd0e8197ca0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fd0d00a2160 0x7fd0d00a4620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 --2- 172.21.15.76:0/793067048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e8100630 0x7fd0e8197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 >> 172.21.15.76:0/793067048 conn(0x7fd0e80fbba0 msgr2=0x7fd0e8068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 shutdown_connections 2024-03-23T15:43:12.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.591+0000 7fd0ee2a4640 1 -- 172.21.15.76:0/793067048 wait complete. 2024-03-23T15:43:12.608 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-23T15:43:12.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.715+0000 7fadf0676640 1 -- 172.21.15.76:0/2610478491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8101820 msgr2=0x7fade8101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.715+0000 7fadf0676640 1 --2- 172.21.15.76:0/2610478491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8101820 0x7fade8101ca0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009900 tx=0x7fadd802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- 172.21.15.76:0/2610478491 shutdown_connections 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 --2- 172.21.15.76:0/2610478491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade81021e0 0x7fade8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 --2- 172.21.15.76:0/2610478491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8101820 0x7fade8101ca0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 --2- 172.21.15.76:0/2610478491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade8100630 0x7fade8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- 172.21.15.76:0/2610478491 >> 172.21.15.76:0/2610478491 conn(0x7fade80fbba0 msgr2=0x7fade80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- 172.21.15.76:0/2610478491 shutdown_connections 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- 172.21.15.76:0/2610478491 wait complete. 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 Processor -- start 2024-03-23T15:43:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- start start 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8100630 0x7fade8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade8101820 0x7fade8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade81021e0 0x7fade819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fade819f220 con 0x7fade8100630 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fade819f390 con 0x7fade81021e0 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadf0676640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fade819f550 con 0x7fade8101820 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadee412640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8100630 0x7fade8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadee412640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8100630 0x7fade8197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35656/0 (socket says 172.21.15.76:35656) 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadee412640 1 -- 172.21.15.76:0/2165615253 learned_addr learned my addr 172.21.15.76:0/2165615253 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.716+0000 7fadeec13640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade81021e0 0x7fade819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadedc11640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade8101820 0x7fade8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadee412640 1 -- 172.21.15.76:0/2165615253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade8101820 msgr2=0x7fade8197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadee412640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade8101820 0x7fade8197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadee412640 1 -- 172.21.15.76:0/2165615253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade81021e0 msgr2=0x7fade819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadee412640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade81021e0 0x7fade819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadee412640 1 -- 172.21.15.76:0/2165615253 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd8009560 con 0x7fade8100630 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadedc11640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade8101820 0x7fade8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadee412640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8100630 0x7fade8197780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fade000eb70 tx=0x7fade000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7faddf7fe640 1 -- 172.21.15.76:0/2165615253 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0003b70 con 0x7fade8100630 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade819f830 con 0x7fade8100630 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7faddf7fe640 1 -- 172.21.15.76:0/2165615253 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade000ee70 con 0x7fade8100630 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7faddf7fe640 1 -- 172.21.15.76:0/2165615253 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade00107f0 con 0x7fade8100630 2024-03-23T15:43:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.717+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade819fcf0 con 0x7fade8100630 2024-03-23T15:43:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.718+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade810c040 con 0x7fade8100630 2024-03-23T15:43:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.719+0000 7faddf7fe640 1 -- 172.21.15.76:0/2165615253 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 19) ==== 136773+0+0 (secure 0 0 0) 0x7fade0004020 con 0x7fade8100630 2024-03-23T15:43:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.720+0000 7faddf7fe640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fadd40a21b0 0x7fadd40a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.720+0000 7fadedc11640 1 -- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fadd40a21b0 msgr2=0x7fadd40a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.720+0000 7faddf7fe640 1 -- 172.21.15.76:0/2165615253 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fade0014070 con 0x7fade8100630 2024-03-23T15:43:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.720+0000 7fadedc11640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fadd40a21b0 0x7fadd40a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:43:12.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.724+0000 7faddf7fe640 1 -- 172.21.15.76:0/2165615253 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fade0097380 con 0x7fade8100630 2024-03-23T15:43:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:12 smithi183 ceph-mon[36296]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:43:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:12 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/793067048' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T15:43:12.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:12 smithi183 ceph-mon[36296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T15:43:12.884 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: ignoring --setuser ceph since I am not root 2024-03-23T15:43:12.884 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: ignoring --setgroup ceph since I am not root 2024-03-23T15:43:12.884 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:12 smithi183 ceph-mgr[37689]: -- 172.21.15.183:0/3493750586 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffc15801a0 con 0x55ffc1556c00 2024-03-23T15:43:12.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.904+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fade810c1d0 con 0x7fade8100630 2024-03-23T15:43:12.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.908+0000 7faddf7fe640 1 -- 172.21.15.76:0/2165615253 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7fade0003e60 con 0x7fade8100630 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fadd40a21b0 msgr2=0x7fadd40a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fadd40a21b0 0x7fadd40a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8100630 msgr2=0x7fade8197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8100630 0x7fade8197780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fade000eb70 tx=0x7fade000c6a0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 shutdown_connections 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7fadd40a21b0 0x7fadd40a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fade81021e0 0x7fade819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade8101820 0x7fade8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 --2- 172.21.15.76:0/2165615253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fade8100630 0x7fade8197780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 >> 172.21.15.76:0/2165615253 conn(0x7fade80fbba0 msgr2=0x7fade8068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 shutdown_connections 2024-03-23T15:43:12.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:12.910+0000 7fadf0676640 1 -- 172.21.15.76:0/2165615253 wait complete. 2024-03-23T15:43:12.924 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: ignoring --setuser ceph since I am not root 2024-03-23T15:43:12.924 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: ignoring --setgroup ceph since I am not root 2024-03-23T15:43:12.924 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:12 smithi076 ceph-mon[32176]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:43:12.924 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:12 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/793067048' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T15:43:12.924 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:12 smithi076 ceph-mon[32176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T15:43:12.925 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-23T15:43:13.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 -- 172.21.15.76:0/3633848514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8101820 msgr2=0x7f76c8101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 --2- 172.21.15.76:0/3633848514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8101820 0x7f76c8101ca0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009900 tx=0x7f76b002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 -- 172.21.15.76:0/3633848514 shutdown_connections 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 --2- 172.21.15.76:0/3633848514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c81021e0 0x7f76c8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 --2- 172.21.15.76:0/3633848514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8101820 0x7f76c8101ca0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 --2- 172.21.15.76:0/3633848514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c8100630 0x7f76c8100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 -- 172.21.15.76:0/3633848514 >> 172.21.15.76:0/3633848514 conn(0x7f76c80fbba0 msgr2=0x7f76c80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 -- 172.21.15.76:0/3633848514 shutdown_connections 2024-03-23T15:43:13.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.027+0000 7f76cca4e640 1 -- 172.21.15.76:0/3633848514 wait complete. 2024-03-23T15:43:13.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 Processor -- start 2024-03-23T15:43:13.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 -- start start 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8100630 0x7f76c8196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c8101820 0x7f76c8196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c81021e0 0x7f76c819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f76c819e120 con 0x7f76c8100630 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76c819e290 con 0x7f76c81021e0 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76cca4e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76c819e450 con 0x7f76c8101820 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8100630 0x7f76c8196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8100630 0x7f76c8196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35666/0 (socket says 172.21.15.76:35666) 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c659c640 1 -- 172.21.15.76:0/1493104114 learned_addr learned my addr 172.21.15.76:0/1493104114 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76bffff640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c8101820 0x7f76c8196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c6d9d640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c81021e0 0x7f76c819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c659c640 1 -- 172.21.15.76:0/1493104114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c8101820 msgr2=0x7f76c8196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:13.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c659c640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c8101820 0x7f76c8196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c659c640 1 -- 172.21.15.76:0/1493104114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c81021e0 msgr2=0x7f76c819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.028+0000 7f76c659c640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c81021e0 0x7f76c819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.029+0000 7f76c659c640 1 -- 172.21.15.76:0/1493104114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b0009560 con 0x7f76c8100630 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.029+0000 7f76c659c640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8100630 0x7f76c8196630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e9c0 tx=0x7f76b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.029+0000 7f76bf7fe640 1 -- 172.21.15.76:0/1493104114 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8009700 con 0x7f76c8100630 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.029+0000 7f76bf7fe640 1 -- 172.21.15.76:0/1493104114 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b800cd80 con 0x7f76c8100630 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.029+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c819e730 con 0x7f76c8100630 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.029+0000 7f76bf7fe640 1 -- 172.21.15.76:0/1493104114 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8010430 con 0x7f76c8100630 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.029+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c819ec10 con 0x7f76c8100630 2024-03-23T15:43:13.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.030+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c81079d0 con 0x7f76c8100630 2024-03-23T15:43:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.031+0000 7f76bf7fe640 1 -- 172.21.15.76:0/1493104114 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 19) ==== 136773+0+0 (secure 0 0 0) 0x7f76b8003680 con 0x7f76c8100630 2024-03-23T15:43:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.032+0000 7f76bf7fe640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 0x7f76b40a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.032+0000 7f76bffff640 1 -- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 msgr2=0x7f76b40a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.032+0000 7f76bf7fe640 1 -- 172.21.15.76:0/1493104114 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f76b8014070 con 0x7f76c8100630 2024-03-23T15:43:13.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.032+0000 7f76bffff640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 0x7f76b40a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:43:13.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.036+0000 7f76bf7fe640 1 -- 172.21.15.76:0/1493104114 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f76b8097080 con 0x7f76c8100630 2024-03-23T15:43:13.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:12 smithi192 ceph-mon[36653]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-23T15:43:13.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:12 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/793067048' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T15:43:13.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:12 smithi192 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-23T15:43:13.070 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:12 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: ignoring --setuser ceph since I am not root 2024-03-23T15:43:13.070 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:12 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: ignoring --setgroup ceph since I am not root 2024-03-23T15:43:13.070 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:12 smithi192 ceph-mgr[38510]: -- 172.21.15.192:0/1342028286 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613c518c1a0 con 0x5613c5163000 2024-03-23T15:43:13.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.233+0000 7f76bffff640 1 -- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 msgr2=0x7f76b40a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:13.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.233+0000 7f76bffff640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 0x7f76b40a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:43:13.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.236+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f76c810c040 con 0x7f76c8100630 2024-03-23T15:43:13.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.239+0000 7f76bf7fe640 1 -- 172.21.15.76:0/1493104114 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f76b809a740 con 0x7f76c8100630 2024-03-23T15:43:13.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 msgr2=0x7f76b40a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:43:13.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 0x7f76b40a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8100630 msgr2=0x7f76c8196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:13.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8100630 0x7f76c8196630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f76b800e9c0 tx=0x7f76b800ee80 comp rx=0 tx=0).stop 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 shutdown_connections 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f76b40a21b0 0x7f76b40a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c81021e0 0x7f76c819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c8101820 0x7f76c8196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 --2- 172.21.15.76:0/1493104114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76c8100630 0x7f76c8196630 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 >> 172.21.15.76:0/1493104114 conn(0x7f76c80fbba0 msgr2=0x7f76c8104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 shutdown_connections 2024-03-23T15:43:13.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:13.242+0000 7f76cca4e640 1 -- 172.21.15.76:0/1493104114 wait complete. 2024-03-23T15:43:13.256 INFO:teuthology.run_tasks:Running task kclient... 2024-03-23T15:43:13.280 INFO:tasks.kclient:Mounting kernel clients... 2024-03-23T15:43:13.280 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-23T15:43:13.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:13.280 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-23T15:43:13.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:13.295 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-03-23T15:43:13.365 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2024-03-23T15:43:13.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:43:13.367 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-23T15:43:13.367 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T15:43:13.367 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:13.367 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-23T15:43:13.569 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:13.150+0000 7f2472774200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:43:13.569 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:13.244+0000 7f2472774200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:43:13.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:13.189+0000 7fc6dc91e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:43:13.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:13.288+0000 7fc6dc91e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:43:13.596 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:13.167+0000 7fe49f449200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T15:43:13.596 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:13.264+0000 7fe49f449200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T15:43:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:43:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:13 smithi183 ceph-mon[36296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-23T15:43:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:13 smithi183 ceph-mon[36296]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-23T15:43:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:13 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2165615253' entity='client.admin' 2024-03-23T15:43:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:13 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1493104114' entity='client.admin' 2024-03-23T15:43:13.894 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-mon[36653]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-23T15:43:13.894 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-mon[36653]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-23T15:43:13.894 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2165615253' entity='client.admin' 2024-03-23T15:43:13.894 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1493104114' entity='client.admin' 2024-03-23T15:43:13.950 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:13 smithi076 ceph-mon[32176]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-23T15:43:13.950 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:13 smithi076 ceph-mon[32176]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-23T15:43:13.950 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:13 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2165615253' entity='client.admin' 2024-03-23T15:43:13.950 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:13 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1493104114' entity='client.admin' 2024-03-23T15:43:14.217 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:13.972+0000 7fc6dc91e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:43:14.217 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.070+0000 7fc6dc91e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:43:14.217 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.213+0000 7fc6dc91e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:43:14.246 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:13.891+0000 7f2472774200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:43:14.246 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:13.985+0000 7f2472774200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:43:14.246 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:14.128+0000 7f2472774200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:43:14.246 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:14.243+0000 7f2472774200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:43:14.314 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.056+0000 7fe49f449200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T15:43:14.314 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.168+0000 7fe49f449200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T15:43:14.569 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:14.328+0000 7f2472774200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:43:14.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.310+0000 7fc6dc91e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:43:14.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.395+0000 7fc6dc91e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:43:14.633 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.312+0000 7fe49f449200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T15:43:14.633 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.409+0000 7fe49f449200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T15:43:14.633 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.493+0000 7fe49f449200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T15:43:14.955 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:14.605+0000 7f2472774200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:43:14.955 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:14.747+0000 7f2472774200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:43:14.955 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:14.855+0000 7f2472774200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:43:14.963 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.664+0000 7fc6dc91e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:43:14.963 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.774+0000 7fc6dc91e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:43:14.963 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.863+0000 7fc6dc91e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:43:15.091 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.785+0000 7fe49f449200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-23T15:43:15.091 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.904+0000 7fe49f449200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T15:43:15.091 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:14.992+0000 7fe49f449200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T15:43:15.232 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:14.960+0000 7fc6dc91e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:43:15.233 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:15.145+0000 7fc6dc91e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:43:15.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:14.952+0000 7f2472774200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:43:15.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:15 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:15.156+0000 7f2472774200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:43:15.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:15.088+0000 7fe49f449200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T15:43:15.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:15.267+0000 7fe49f449200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T15:43:15.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 -- 172.21.15.76:0/1329448940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 msgr2=0x7f668010e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:15.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 --2- 172.21.15.76:0/1329448940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f668010e6c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6670009990 tx=0x7f6670009ca0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 -- 172.21.15.76:0/1329448940 shutdown_connections 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 --2- 172.21.15.76:0/1329448940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f668010ec00 0x7f6680110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 --2- 172.21.15.76:0/1329448940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6680074430 0x7f6680074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 --2- 172.21.15.76:0/1329448940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f668010e6c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 -- 172.21.15.76:0/1329448940 >> 172.21.15.76:0/1329448940 conn(0x7f6680079360 msgr2=0x7f6680079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 -- 172.21.15.76:0/1329448940 shutdown_connections 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.617+0000 7f6687cf3700 1 -- 172.21.15.76:0/1329448940 wait complete. 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 Processor -- start 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 -- start start 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6680074430 0x7f66801a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f66801a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:15.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f668010ec00 0x7f668019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f668019ae80 con 0x7f6680074d50 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f668019aff0 con 0x7f6680074430 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6687cf3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f668019b1b0 con 0x7f668010ec00 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f668528e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f66801a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f668528e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f66801a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49184/0 (socket says 172.21.15.76:49184) 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f668528e700 1 -- 172.21.15.76:0/2971536340 learned_addr learned my addr 172.21.15.76:0/2971536340 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6685a8f700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6680074430 0x7f66801a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.618+0000 7f6686290700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f668010ec00 0x7f668019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f668528e700 1 -- 172.21.15.76:0/2971536340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f668010ec00 msgr2=0x7f668019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f668528e700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f668010ec00 0x7f668019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f668528e700 1 -- 172.21.15.76:0/2971536340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6680074430 msgr2=0x7f66801a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f668528e700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6680074430 0x7f66801a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f668528e700 1 -- 172.21.15.76:0/2971536340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66700095f0 con 0x7f6680074d50 2024-03-23T15:43:15.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f668528e700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f66801a0b40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6670000c00 tx=0x7f6670004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:15.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f6676ffd700 1 -- 172.21.15.76:0/2971536340 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667001d070 con 0x7f6680074d50 2024-03-23T15:43:15.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668019b430 con 0x7f6680074d50 2024-03-23T15:43:15.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801a7630 con 0x7f6680074d50 2024-03-23T15:43:15.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f6676ffd700 1 -- 172.21.15.76:0/2971536340 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6670022470 con 0x7f6680074d50 2024-03-23T15:43:15.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.619+0000 7f6676ffd700 1 -- 172.21.15.76:0/2971536340 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670017650 con 0x7f6680074d50 2024-03-23T15:43:15.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.620+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6680075b60 con 0x7f6680074d50 2024-03-23T15:43:15.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.621+0000 7f6676ffd700 1 -- 172.21.15.76:0/2971536340 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 19) ==== 136773+0+0 (secure 0 0 0) 0x7f66700225e0 con 0x7f6680074d50 2024-03-23T15:43:15.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.622+0000 7f6676ffd700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 0x7f666c0a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:15.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.622+0000 7f6685a8f700 1 -- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 msgr2=0x7f666c0a4660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:15.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.622+0000 7f6676ffd700 1 -- 172.21.15.76:0/2971536340 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f66700cf470 con 0x7f6680074d50 2024-03-23T15:43:15.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.622+0000 7f6685a8f700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 0x7f666c0a4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:43:15.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.628+0000 7f6676ffd700 1 -- 172.21.15.76:0/2971536340 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f66700984b0 con 0x7f6680074d50 2024-03-23T15:43:15.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.823+0000 7f6685a8f700 1 -- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 msgr2=0x7f666c0a4660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:15.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.823+0000 7f6685a8f700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 0x7f666c0a4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:43:15.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.873+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6680068ec0 con 0x7f6680074d50 2024-03-23T15:43:15.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.874+0000 7f6676ffd700 1 -- 172.21.15.76:0/2971536340 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7f6670027070 con 0x7f6680074d50 2024-03-23T15:43:15.877 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T15:43:15.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 msgr2=0x7f666c0a4660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:43:15.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 0x7f666c0a4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 msgr2=0x7f66801a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:15.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f66801a0b40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6670000c00 tx=0x7f6670004730 comp rx=0 tx=0).stop 2024-03-23T15:43:15.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 shutdown_connections 2024-03-23T15:43:15.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f666c0a21a0 0x7f666c0a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f668010ec00 0x7f668019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6680074430 0x7f66801a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 --2- 172.21.15.76:0/2971536340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6680074d50 0x7f66801a0b40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:15.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 >> 172.21.15.76:0/2971536340 conn(0x7f6680079360 msgr2=0x7f66800ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:15.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 shutdown_connections 2024-03-23T15:43:15.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:15.876+0000 7f6687cf3700 1 -- 172.21.15.76:0/2971536340 wait complete. 2024-03-23T15:43:16.165 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:15 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2971536340' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T15:43:16.259 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:16.125+0000 7f2472774200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:43:16.259 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:15 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2971536340' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T15:43:16.346 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:15 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2971536340' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T15:43:16.569 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:16.256+0000 7f2472774200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:43:16.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:16.161+0000 7fc6dc91e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:43:16.575 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:16.296+0000 7fc6dc91e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:43:16.632 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T15:43:16.633 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T15:43:16.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:16.633 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-03-23T15:43:16.634 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:16.343+0000 7fe49f449200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-23T15:43:16.634 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:16.476+0000 7fe49f449200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T15:43:16.649 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T15:43:16.649 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T15:43:16.649 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T15:43:16.649 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T15:43:16.650 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42136sec preferred_lft 42136sec 2024-03-23T15:43:16.651 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-03-23T15:43:16.651 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-03-23T15:43:16.651 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T15:43:16.651 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T15:43:16.651 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-23T15:43:16.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2024-03-23T15:43:16.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2024-03-23T15:43:16.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2024-03-23T15:43:16.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T15:43:16.652 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-23T15:43:16.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T15:43:16.850 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:16.645+0000 7fc6dc91e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:43:16.850 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:16.740+0000 7fc6dc91e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:43:16.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:16.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:16.861 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T15:43:16.889 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-03-23T15:43:16.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:16.891 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-03-23T15:43:16.897 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:16.606+0000 7f2472774200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:43:16.897 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:16.697+0000 7f2472774200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:43:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-03-23T15:43:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T15:43:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T15:43:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T15:43:16.961 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T15:43:16.962 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T15:43:16.962 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-23T15:43:16.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T15:43:16.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T15:43:16.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T15:43:16.962 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-23T15:43:17.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T15:43:17.133 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:16.836+0000 7fe49f449200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T15:43:17.133 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:16.928+0000 7fe49f449200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T15:43:17.154 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:16.954+0000 7fc6dc91e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:43:17.154 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:17 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:17.151+0000 7fc6dc91e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:43:17.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:17.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:17.291 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-23T15:43:17.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:17.306 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-03-23T15:43:17.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:16.895+0000 7f2472774200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:43:17.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:17.083+0000 7f2472774200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:43:17.362 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T15:43:17.362 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-23T15:43:17.362 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T15:43:17.362 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T15:43:17.362 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-23T15:43:17.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T15:43:17.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:17.486 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T15:43:17.486 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T15:43:17.486 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-23T15:43:17.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T15:43:17.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T15:43:17.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T15:43:17.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T15:43:17.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T15:43:17.487 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-23T15:43:17.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T15:43:17.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:17.633 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T15:43:17.633 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-03-23T15:43:17.633 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-03-23T15:43:17.633 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T15:43:17.634 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-03-23T15:43:17.634 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:17.147+0000 7fe49f449200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:43:17.634 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:17.366+0000 7fe49f449200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T15:43:17.641 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:17.371+0000 7f2472774200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:43:17.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T15:43:17.712 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:17 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:17.457+0000 7fc6dc91e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:43:17.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:17.719 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T15:43:17.719 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:17.734 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T15:43:17.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T15:43:17.735 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:17.790 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-23T15:43:17.791 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-23T15:43:17.791 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4 2024-03-23T15:43:17.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:17.791 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-03-23T15:43:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-03-23T15:43:17.877 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-03-23T15:43:17.920 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:17.638+0000 7f2472774200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:43:17.950 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:17.673+0000 7fe49f449200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T15:43:17.984 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:17 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:17.736+0000 7fc6dc91e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:43:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-03-23T15:43:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-03-23T15:43:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.76:3300,172.21.15.183:3300,172.21.15.192:3300" 2024-03-23T15:43:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 0@89c3e582-e92a-11ee-95cd-87774f69a715.cephfs=/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4 2024-03-23T15:43:17.993 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.76:3300/172.21.15.183:3300/172.21.15.192:3300" will pass to kernel 2024-03-23T15:43:17.994 INFO:teuthology.orchestra.run.smithi076.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-23T15:43:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: You just mounted a file system that supports labels which does not 2024-03-23T15:43:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-23T15:43:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: applications will generate AVC messages and not be allowed access to 2024-03-23T15:43:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-23T15:43:17.996 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-23T15:43:17.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:17.996 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:18.025 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T15:43:18.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T15:43:18.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:18.123 INFO:teuthology.orchestra.run.smithi076.stdout:client15327 2024-03-23T15:43:18.123 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:43:18.123 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T15:43:18.140 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T15:43:18.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T15:43:18.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:18.236 INFO:teuthology.orchestra.run.smithi076.stdout:client15327 2024-03-23T15:43:18.237 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-23T15:43:18.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T15:43:18.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:18.294 INFO:teuthology.orchestra.run.smithi076.stdout:client15327 2024-03-23T15:43:18.294 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/89c3e582-e92a-11ee-95cd-87774f69a715.client15327/status') 2024-03-23T15:43:18.319 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:17.917+0000 7f2472774200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:43:18.325 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:18.005+0000 7fc6dc91e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:43:18.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T15:43:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:instance: client.15327 (3)192.168.0.1:0/1716252318 2024-03-23T15:43:18.355 INFO:teuthology.orchestra.run.smithi076.stdout:blocklisted: false 2024-03-23T15:43:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records in 2024-03-23T15:43:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records out 2024-03-23T15:43:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:70 bytes copied, 2.2919e-05 s, 3.1 MB/s 2024-03-23T15:43:18.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T15:43:18.363 INFO:teuthology.run_tasks:Running task exec... 2024-03-23T15:43:18.372 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T15:43:18.372 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:43:18.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-23T15:43:18.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:17.948+0000 7fe49f449200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T15:43:18.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:18.189+0000 7fe49f449200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.504+0000 7f17f9015640 1 -- 172.21.15.76:0/2740984842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4104ba0 msgr2=0x7f17f4106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.504+0000 7f17f9015640 1 --2- 172.21.15.76:0/2740984842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4104ba0 0x7f17f4106f90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009900 tx=0x7f17e402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 -- 172.21.15.76:0/2740984842 shutdown_connections 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 --2- 172.21.15.76:0/2740984842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17f41074d0 0x7f17f41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 --2- 172.21.15.76:0/2740984842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4104ba0 0x7f17f4106f90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 --2- 172.21.15.76:0/2740984842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f4102270 0x7f17f4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 -- 172.21.15.76:0/2740984842 >> 172.21.15.76:0/2740984842 conn(0x7f17f40fbba0 msgr2=0x7f17f40fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:18.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 -- 172.21.15.76:0/2740984842 shutdown_connections 2024-03-23T15:43:18.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 -- 172.21.15.76:0/2740984842 wait complete. 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.505+0000 7f17f9015640 1 Processor -- start 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f9015640 1 -- start start 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f9015640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4102270 0x7f17f4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f9015640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17f4104ba0 0x7f17f4197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f9015640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f41074d0 0x7f17f419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f9015640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17f419f280 con 0x7f17f4102270 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f9015640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17f419f3f0 con 0x7f17f4104ba0 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f9015640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f17f419f5b0 con 0x7f17f41074d0 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4102270 0x7f17f4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4102270 0x7f17f4197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49190/0 (socket says 172.21.15.76:49190) 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 -- 172.21.15.76:0/2442720380 learned_addr learned my addr 172.21.15.76:0/2442720380 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f359e640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f41074d0 0x7f17f419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:18.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f259c640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17f4104ba0 0x7f17f4197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 -- 172.21.15.76:0/2442720380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f41074d0 msgr2=0x7f17f419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f41074d0 0x7f17f419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 -- 172.21.15.76:0/2442720380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17f4104ba0 msgr2=0x7f17f4197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17f4104ba0 0x7f17f4197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.506+0000 7f17f2d9d640 1 -- 172.21.15.76:0/2442720380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e4009560 con 0x7f17f4102270 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.507+0000 7f17f2d9d640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4102270 0x7f17f4197790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f17e800e9f0 tx=0x7f17e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.507+0000 7f17dbfff640 1 -- 172.21.15.76:0/2442720380 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8009720 con 0x7f17f4102270 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.507+0000 7f17dbfff640 1 -- 172.21.15.76:0/2442720380 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e800cd80 con 0x7f17f4102270 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.507+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f419f890 con 0x7f17f4102270 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.507+0000 7f17dbfff640 1 -- 172.21.15.76:0/2442720380 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8010690 con 0x7f17f4102270 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.507+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f419fd20 con 0x7f17f4102270 2024-03-23T15:43:18.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.508+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f40fd690 con 0x7f17f4102270 2024-03-23T15:43:18.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.509+0000 7f17dbfff640 1 -- 172.21.15.76:0/2442720380 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 19) ==== 136773+0+0 (secure 0 0 0) 0x7f17e80046c0 con 0x7f17f4102270 2024-03-23T15:43:18.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.509+0000 7f17dbfff640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 0x7f17dc0a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.509+0000 7f17f259c640 1 -- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 msgr2=0x7f17dc0a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:18.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.510+0000 7f17dbfff640 1 -- 172.21.15.76:0/2442720380 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f17e8014070 con 0x7f17f4102270 2024-03-23T15:43:18.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.510+0000 7f17f259c640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 0x7f17dc0a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:43:18.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.514+0000 7f17dbfff640 1 -- 172.21.15.76:0/2442720380 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f17e8097040 con 0x7f17f4102270 2024-03-23T15:43:18.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.711+0000 7f17f259c640 1 -- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 msgr2=0x7f17dc0a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:18.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.711+0000 7f17f259c640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 0x7f17dc0a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:43:18.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.736+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f17f402d130 con 0x7f17f4102270 2024-03-23T15:43:18.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.787+0000 7f17dbfff640 1 -- 172.21.15.76:0/2442720380 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7f17e809a700 con 0x7f17f4102270 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 msgr2=0x7f17dc0a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 0x7f17dc0a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4102270 msgr2=0x7f17f4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4102270 0x7f17f4197790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f17e800e9f0 tx=0x7f17e800edf0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 shutdown_connections 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f41074d0 0x7f17f419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f17dc0a21b0 0x7f17dc0a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17f4104ba0 0x7f17f4197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 --2- 172.21.15.76:0/2442720380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f4102270 0x7f17f4197790 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 >> 172.21.15.76:0/2442720380 conn(0x7f17f40fbba0 msgr2=0x7f17f4102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:18.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 shutdown_connections 2024-03-23T15:43:18.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.790+0000 7f17f9015640 1 -- 172.21.15.76:0/2442720380 wait complete. 2024-03-23T15:43:18.805 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 -- 172.21.15.76:0/3800170403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94100630 msgr2=0x7f1a94100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 --2- 172.21.15.76:0/3800170403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94100630 0x7f1a94100a50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009900 tx=0x7f1a8002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 -- 172.21.15.76:0/3800170403 shutdown_connections 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 --2- 172.21.15.76:0/3800170403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a941021e0 0x7f1a94108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 --2- 172.21.15.76:0/3800170403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a94101820 0x7f1a94101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 --2- 172.21.15.76:0/3800170403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94100630 0x7f1a94100a50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 -- 172.21.15.76:0/3800170403 >> 172.21.15.76:0/3800170403 conn(0x7f1a940fbba0 msgr2=0x7f1a940fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:18.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 -- 172.21.15.76:0/3800170403 shutdown_connections 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 -- 172.21.15.76:0/3800170403 wait complete. 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 Processor -- start 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.910+0000 7f1a9ae64640 1 -- start start 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a9ae64640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a94100630 0x7f1a94069920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a9ae64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94101820 0x7f1a9419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a9ae64640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a941021e0 0x7f1a9419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a9ae64640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9419f260 con 0x7f1a94101820 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a9ae64640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9419f3d0 con 0x7f1a941021e0 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a9ae64640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9419f590 con 0x7f1a94100630 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94101820 0x7f1a9419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94101820 0x7f1a9419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49194/0 (socket says 172.21.15.76:49194) 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 -- 172.21.15.76:0/4098672133 learned_addr learned my addr 172.21.15.76:0/4098672133 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a98c00640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a94100630 0x7f1a94069920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a99401640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a941021e0 0x7f1a9419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:18.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a94100630 msgr2=0x7f1a94069920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a94100630 0x7f1a94069920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a941021e0 msgr2=0x7f1a9419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a941021e0 0x7f1a9419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 -- 172.21.15.76:0/4098672133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80009560 con 0x7f1a94101820 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a93fff640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94101820 0x7f1a9419c490 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800e9f0 tx=0x7f1a8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88009700 con 0x7f1a94101820 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8800cd80 con 0x7f1a94101820 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a9ae64640 1 -- 172.21.15.76:0/4098672133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9419f870 con 0x7f1a94101820 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.911+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88018920 con 0x7f1a94101820 2024-03-23T15:43:18.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.912+0000 7f1a9ae64640 1 -- 172.21.15.76:0/4098672133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9419fd80 con 0x7f1a94101820 2024-03-23T15:43:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.913+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a74005290 con 0x7f1a94101820 2024-03-23T15:43:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.914+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 19) ==== 136773+0+0 (secure 0 0 0) 0x7f1a88018a80 con 0x7f1a94101820 2024-03-23T15:43:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.915+0000 7f1a91ffb640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 0x7f1a840a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.915+0000 7f1a98c00640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 msgr2=0x7f1a840a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.915+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f1a88014070 con 0x7f1a94101820 2024-03-23T15:43:18.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.915+0000 7f1a98c00640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 0x7f1a840a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T15:43:18.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:18.919+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1a88053280 con 0x7f1a94101820 2024-03-23T15:43:19.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:18 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2442720380' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-23T15:43:19.069 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:18 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:18.735+0000 7f2472774200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:43:19.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:18 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2442720380' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-23T15:43:19.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:18 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2442720380' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-23T15:43:19.076 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:18.861+0000 7fc6dc91e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:43:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.116+0000 7f1a98c00640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 msgr2=0x7f1a840a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.116+0000 7f1a98c00640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 0x7f1a840a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T15:43:19.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.140+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 --> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1a74000c60 con 0x7f1a840a21b0 2024-03-23T15:43:19.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:19.050+0000 7fe49f449200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T15:43:19.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.517+0000 7f1a98c00640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 msgr2=0x7f1a840a4670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:6800/2479514030 2024-03-23T15:43:19.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.517+0000 7f1a98c00640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 0x7f1a840a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T15:43:19.645 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:19.325+0000 7f2472774200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:43:19.646 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:19.424+0000 7f2472774200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:43:19.646 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z[38506]: 2024-03-23T15:43:19.643+0000 7f2472774200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:43:19.777 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:19.450+0000 7fc6dc91e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:43:19.777 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:19.551+0000 7fc6dc91e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:43:19.778 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:19.774+0000 7fc6dc91e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:43:19.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.797+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 20) ==== 136441+0+0 (secure 0 0 0) 0x7f1a880974c0 con 0x7f1a94101820 2024-03-23T15:43:19.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.798+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 msgr2=0x7f1a840a4670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T15:43:19.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:19.798+0000 7f1a91ffb640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/2479514030,v1:172.21.15.76:6801/2479514030] conn(0x7f1a840a21b0 0x7f1a840a4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:19.947 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:19.632+0000 7fe49f449200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:19.755+0000 7fe49f449200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2442720380' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: Standby manager daemon z restarted 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: Standby manager daemon z started 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: Active manager daemon x restarted 2024-03-23T15:43:19.948 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:19 smithi183 ceph-mon[36296]: Activating manager daemon x 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2442720380' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: Standby manager daemon z restarted 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: Standby manager daemon z started 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: Active manager daemon x restarted 2024-03-23T15:43:19.949 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:19 smithi192 ceph-mon[36653]: Activating manager daemon x 2024-03-23T15:43:20.056 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2442720380' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-23T15:43:20.056 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: Standby manager daemon z restarted 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: Standby manager daemon z started 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.192:0/319203999' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: Active manager daemon x restarted 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:19 smithi076 ceph-mon[32176]: Activating manager daemon x 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63f3be700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63f3be700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63f3be700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.057 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63f3be700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.058 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63f3be700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.058 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63c3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.058 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63c3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.058 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63c3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.058 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63c3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.058 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.023+0000 7fc63c3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:20.383 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y[37685]: 2024-03-23T15:43:20.020+0000 7fe49f449200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T15:43:20.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.818+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mon.0 v2:172.21.15.76:3300/0 8 ==== mgrmap(e 21) ==== 136568+0+0 (secure 0 0 0) 0x7f1a880cbbc0 con 0x7f1a94101820 2024-03-23T15:43:20.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.819+0000 7f1a91ffb640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a84051810 0x7f1a84051bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:20.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.819+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1a74000c60 con 0x7f1a84051810 2024-03-23T15:43:20.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.820+0000 7f1a98c00640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a84051810 0x7f1a84051bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:20.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.821+0000 7f1a98c00640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a84051810 0x7f1a84051bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005ab0 tx=0x7f1a80005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:20.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.837+0000 7f1a91ffb640 1 -- 172.21.15.76:0/4098672133 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f1a74000c60 con 0x7f1a84051810 2024-03-23T15:43:20.840 INFO:teuthology.orchestra.run.smithi076.stdout:{} 2024-03-23T15:43:20.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a84051810 msgr2=0x7f1a84051bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a84051810 0x7f1a84051bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a80005ab0 tx=0x7f1a80005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94101820 msgr2=0x7f1a9419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 --2- 172.21.15.76:0/4098672133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a94101820 0x7f1a9419c490 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800e9f0 tx=0x7f1a8800eeb0 comp rx=0 tx=0).stop 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a99401640 1 -- 172.21.15.76:0/4098672133 reap_dead start 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 shutdown_connections 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 >> 172.21.15.76:0/4098672133 conn(0x7f1a940fbba0 msgr2=0x7f1a94104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 shutdown_connections 2024-03-23T15:43:20.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.839+0000 7f1a7f7fe640 1 -- 172.21.15.76:0/4098672133 wait complete. 2024-03-23T15:43:20.854 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-23T15:43:20.865 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-23T15:43:20.881 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:43:20.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 -- 172.21.15.76:0/2925175033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a601006b0 msgr2=0x7f7a60100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:20.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 --2- 172.21.15.76:0/2925175033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a601006b0 0x7f7a60100ad0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009900 tx=0x7f7a5802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:20.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 -- 172.21.15.76:0/2925175033 shutdown_connections 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 --2- 172.21.15.76:0/2925175033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a60102260 0x7f7a60108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 --2- 172.21.15.76:0/2925175033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601018a0 0x7f7a60101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 --2- 172.21.15.76:0/2925175033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a601006b0 0x7f7a60100ad0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 -- 172.21.15.76:0/2925175033 >> 172.21.15.76:0/2925175033 conn(0x7f7a600fbc40 msgr2=0x7f7a600fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 -- 172.21.15.76:0/2925175033 shutdown_connections 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.982+0000 7f7a67381640 1 -- 172.21.15.76:0/2925175033 wait complete. 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 Processor -- start 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 -- start start 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601006b0 0x7f7a601977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a601018a0 0x7f7a60197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60102260 0x7f7a6019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6019f2d0 con 0x7f7a60102260 2024-03-23T15:43:20.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6019f440 con 0x7f7a601006b0 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a67381640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6019f600 con 0x7f7a601018a0 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a6591e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60102260 0x7f7a6019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a6511d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601006b0 0x7f7a601977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a6511d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601006b0 0x7f7a601977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:58774/0 (socket says 172.21.15.76:58774) 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a6511d640 1 -- 172.21.15.76:0/3716819037 learned_addr learned my addr 172.21.15.76:0/3716819037 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.983+0000 7f7a6491c640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a601018a0 0x7f7a60197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a6591e640 1 -- 172.21.15.76:0/3716819037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a601018a0 msgr2=0x7f7a60197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a6591e640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a601018a0 0x7f7a60197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a6591e640 1 -- 172.21.15.76:0/3716819037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601006b0 msgr2=0x7f7a601977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a6591e640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601006b0 0x7f7a601977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a6591e640 1 -- 172.21.15.76:0/3716819037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a54009490 con 0x7f7a60102260 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a6511d640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601006b0 0x7f7a601977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:20.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a6591e640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60102260 0x7f7a6019ed90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009fd0 tx=0x7f7a5000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:20.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a5e7fc640 1 -- 172.21.15.76:0/3716819037 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000ce20 con 0x7f7a60102260 2024-03-23T15:43:20.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a5e7fc640 1 -- 172.21.15.76:0/3716819037 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a50005020 con 0x7f7a60102260 2024-03-23T15:43:20.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a58009560 con 0x7f7a60102260 2024-03-23T15:43:20.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a5e7fc640 1 -- 172.21.15.76:0/3716819037 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50010690 con 0x7f7a60102260 2024-03-23T15:43:20.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.984+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6019fcb0 con 0x7f7a60102260 2024-03-23T15:43:20.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.985+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6010c050 con 0x7f7a60102260 2024-03-23T15:43:20.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.986+0000 7f7a5e7fc640 1 -- 172.21.15.76:0/3716819037 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 21) ==== 136568+0+0 (secure 0 0 0) 0x7f7a50005500 con 0x7f7a60102260 2024-03-23T15:43:20.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.987+0000 7f7a5e7fc640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a480a1f40 0x7f7a480a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:20.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.987+0000 7f7a6511d640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a480a1f40 0x7f7a480a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:20.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.987+0000 7f7a5e7fc640 1 -- 172.21.15.76:0/3716819037 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7a50014070 con 0x7f7a60102260 2024-03-23T15:43:20.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.987+0000 7f7a6511d640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a480a1f40 0x7f7a480a4400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7a5802c840 tx=0x7f7a5802cd10 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:20.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:20.990+0000 7f7a5e7fc640 1 -- 172.21.15.76:0/3716819037 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f7a50097190 con 0x7f7a60102260 2024-03-23T15:43:21.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: mgrmap e20: x(active, starting, since 0.012011s), standbys: y, z 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:43:21.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:43:21.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: Manager daemon x is now available 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: Standby manager daemon y restarted 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: Standby manager daemon y started 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T15:43:21.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:43:21.074 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T15:43:21.074 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:43:21.074 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: [23/Mar/2024:15:43:20] ENGINE Bus STARTING 2024-03-23T15:43:21.074 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: [23/Mar/2024:15:43:20] ENGINE Serving on https://172.21.15.76:7150 2024-03-23T15:43:21.074 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: [23/Mar/2024:15:43:20] ENGINE Serving on http://172.21.15.76:8765 2024-03-23T15:43:21.074 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:20 smithi192 ceph-mon[36653]: [23/Mar/2024:15:43:20] ENGINE Bus STARTED 2024-03-23T15:43:21.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.828+0000 7fc6393b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:21.075 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.828+0000 7fc6393b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:21.076 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.828+0000 7fc6393b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:21.076 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.828+0000 7fc6393b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:21.076 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:43:20.828+0000 7fc6393b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:43:21.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: mgrmap e20: x(active, starting, since 0.012011s), standbys: y, z 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:43:21.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:43:21.078 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: Manager daemon x is now available 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T15:43:21.079 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: Standby manager daemon y restarted 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: Standby manager daemon y started 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: [23/Mar/2024:15:43:20] ENGINE Bus STARTING 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: [23/Mar/2024:15:43:20] ENGINE Serving on https://172.21.15.76:7150 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: [23/Mar/2024:15:43:20] ENGINE Serving on http://172.21.15.76:8765 2024-03-23T15:43:21.080 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:20 smithi076 ceph-mon[32176]: [23/Mar/2024:15:43:20] ENGINE Bus STARTED 2024-03-23T15:43:21.093 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: osdmap e88: 12 total, 12 up, 12 in 2024-03-23T15:43:21.093 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: mgrmap e20: x(active, starting, since 0.012011s), standbys: y, z 2024-03-23T15:43:21.093 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-23T15:43:21.094 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-23T15:43:21.094 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-23T15:43:21.094 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-23T15:43:21.094 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-23T15:43:21.094 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:43:21.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-23T15:43:21.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: Manager daemon x is now available 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: Standby manager daemon y restarted 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: Standby manager daemon y started 2024-03-23T15:43:21.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-23T15:43:21.098 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T15:43:21.098 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-23T15:43:21.098 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: from='mgr.? 172.21.15.183:0/1915464365' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T15:43:21.098 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: [23/Mar/2024:15:43:20] ENGINE Bus STARTING 2024-03-23T15:43:21.098 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: [23/Mar/2024:15:43:20] ENGINE Serving on https://172.21.15.76:7150 2024-03-23T15:43:21.098 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: [23/Mar/2024:15:43:20] ENGINE Serving on http://172.21.15.76:8765 2024-03-23T15:43:21.098 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:20 smithi183 ceph-mon[36296]: [23/Mar/2024:15:43:20] ENGINE Bus STARTED 2024-03-23T15:43:21.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.176+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a601a01b0 con 0x7f7a60102260 2024-03-23T15:43:21.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.178+0000 7f7a5e7fc640 1 -- 172.21.15.76:0/3716819037 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23107 (secure 0 0 0) 0x7f7a50005190 con 0x7f7a60102260 2024-03-23T15:43:21.181 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:21.181 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":88,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:43:19.788338+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/4023925236":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6801/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6800/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/1911361196":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/4249446235":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/3432469273":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:43:21.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a480a1f40 msgr2=0x7f7a480a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a480a1f40 0x7f7a480a4400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7a5802c840 tx=0x7f7a5802cd10 comp rx=0 tx=0).stop 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60102260 msgr2=0x7f7a6019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60102260 0x7f7a6019ed90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009fd0 tx=0x7f7a5000ef10 comp rx=0 tx=0).stop 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 shutdown_connections 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a480a1f40 0x7f7a480a4400 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a60102260 0x7f7a6019ed90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a601018a0 0x7f7a60197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 --2- 172.21.15.76:0/3716819037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a601006b0 0x7f7a601977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 >> 172.21.15.76:0/3716819037 conn(0x7f7a600fbc40 msgr2=0x7f7a60069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 shutdown_connections 2024-03-23T15:43:21.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.180+0000 7f7a67381640 1 -- 172.21.15.76:0/3716819037 wait complete. 2024-03-23T15:43:21.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:43:21.195 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:43:21.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 -- 172.21.15.76:0/10216225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0103340 msgr2=0x7fd6a0109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 --2- 172.21.15.76:0/10216225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0103340 0x7fd6a0109bd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd698009ca0 tx=0x7fd69802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 -- 172.21.15.76:0/10216225 shutdown_connections 2024-03-23T15:43:21.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 --2- 172.21.15.76:0/10216225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0103340 0x7fd6a0109bd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 --2- 172.21.15.76:0/10216225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a0102980 0x7fd6a0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 --2- 172.21.15.76:0/10216225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6a0101790 0x7fd6a0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 -- 172.21.15.76:0/10216225 >> 172.21.15.76:0/10216225 conn(0x7fd6a00fcce0 msgr2=0x7fd6a00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 -- 172.21.15.76:0/10216225 shutdown_connections 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 -- 172.21.15.76:0/10216225 wait complete. 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.300+0000 7fd6a7057640 1 Processor -- start 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 -- start start 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a0101790 0x7fd6a00754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0102980 0x7fd6a0073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6a0103340 0x7fd6a0074040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a00747a0 con 0x7fd6a0102980 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0074910 con 0x7fd6a0101790 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01a1800 con 0x7fd6a0103340 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0102980 0x7fd6a0073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0102980 0x7fd6a0073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49378/0 (socket says 172.21.15.76:49378) 2024-03-23T15:43:21.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 -- 172.21.15.76:0/916307080 learned_addr learned my addr 172.21.15.76:0/916307080 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a55f4640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6a0103340 0x7fd6a0074040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a4df3640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a0101790 0x7fd6a00754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 -- 172.21.15.76:0/916307080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6a0103340 msgr2=0x7fd6a0074040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6a0103340 0x7fd6a0074040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 -- 172.21.15.76:0/916307080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a0101790 msgr2=0x7fd6a00754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a0101790 0x7fd6a00754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 -- 172.21.15.76:0/916307080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd698009900 con 0x7fd6a0102980 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69ffff640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0102980 0x7fd6a0073b00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00ea20 tx=0x7fd68c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69dffb640 1 -- 172.21.15.76:0/916307080 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c009700 con 0x7fd6a0102980 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69dffb640 1 -- 172.21.15.76:0/916307080 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68c00cd80 con 0x7fd6a0102980 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a01a1ab0 con 0x7fd6a0102980 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.301+0000 7fd69dffb640 1 -- 172.21.15.76:0/916307080 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c018840 con 0x7fd6a0102980 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.302+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a01a1fc0 con 0x7fd6a0102980 2024-03-23T15:43:21.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.303+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a0108a50 con 0x7fd6a0102980 2024-03-23T15:43:21.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.303+0000 7fd69dffb640 1 -- 172.21.15.76:0/916307080 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 21) ==== 136568+0+0 (secure 0 0 0) 0x7fd68c010a80 con 0x7fd6a0102980 2024-03-23T15:43:21.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.304+0000 7fd69dffb640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd6900a1ec0 0x7fd6900a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.304+0000 7fd6a4df3640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd6900a1ec0 0x7fd6900a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.304+0000 7fd69dffb640 1 -- 172.21.15.76:0/916307080 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd68c014070 con 0x7fd6a0102980 2024-03-23T15:43:21.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.305+0000 7fd6a4df3640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd6900a1ec0 0x7fd6900a4380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6a01027e0 tx=0x7fd694005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:21.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.309+0000 7fd69dffb640 1 -- 172.21.15.76:0/916307080 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd68c053280 con 0x7fd6a0102980 2024-03-23T15:43:21.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.484+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6a002d160 con 0x7fd6a0102980 2024-03-23T15:43:21.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.484+0000 7fd69dffb640 1 -- 172.21.15.76:0/916307080 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd68c096ef0 con 0x7fd6a0102980 2024-03-23T15:43:21.488 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:43:21.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd6900a1ec0 msgr2=0x7fd6900a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd6900a1ec0 0x7fd6900a4380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6a01027e0 tx=0x7fd694005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0102980 msgr2=0x7fd6a0073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0102980 0x7fd6a0073b00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00ea20 tx=0x7fd68c00eee0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 shutdown_connections 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd6900a1ec0 0x7fd6900a4380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6a0103340 0x7fd6a0074040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6a0102980 0x7fd6a0073b00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 --2- 172.21.15.76:0/916307080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a0101790 0x7fd6a00754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 >> 172.21.15.76:0/916307080 conn(0x7fd6a00fcce0 msgr2=0x7fd6a0105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:21.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 shutdown_connections 2024-03-23T15:43:21.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.487+0000 7fd6a7057640 1 -- 172.21.15.76:0/916307080 wait complete. 2024-03-23T15:43:21.503 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:43:21.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.614+0000 7f5423fb5640 1 -- 172.21.15.76:0/3321302906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c102190 msgr2=0x7f541c104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.614+0000 7f5423fb5640 1 --2- 172.21.15.76:0/3321302906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c102190 0x7f541c104580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f540c009860 tx=0x7f540c02bb70 comp rx=0 tx=0).stop 2024-03-23T15:43:21.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.614+0000 7f5423fb5640 1 -- 172.21.15.76:0/3321302906 shutdown_connections 2024-03-23T15:43:21.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.614+0000 7f5423fb5640 1 --2- 172.21.15.76:0/3321302906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f541c1073f0 0x7f541c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.614+0000 7f5423fb5640 1 --2- 172.21.15.76:0/3321302906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c104ac0 0x7f541c106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.615+0000 7f5423fb5640 1 --2- 172.21.15.76:0/3321302906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c102190 0x7f541c104580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.615+0000 7f5423fb5640 1 -- 172.21.15.76:0/3321302906 >> 172.21.15.76:0/3321302906 conn(0x7f541c0fbba0 msgr2=0x7f541c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:21.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.615+0000 7f5423fb5640 1 -- 172.21.15.76:0/3321302906 shutdown_connections 2024-03-23T15:43:21.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.615+0000 7f5423fb5640 1 -- 172.21.15.76:0/3321302906 wait complete. 2024-03-23T15:43:21.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.615+0000 7f5423fb5640 1 Processor -- start 2024-03-23T15:43:21.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.615+0000 7f5423fb5640 1 -- start start 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5423fb5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f541c102190 0x7f541c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5423fb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c104ac0 0x7f541c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5423fb5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1073f0 0x7f541c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5423fb5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f541c19f310 con 0x7f541c104ac0 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5423fb5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f541c19f480 con 0x7f541c1073f0 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5423fb5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f541c19f640 con 0x7f541c102190 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c104ac0 0x7f541c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c104ac0 0x7f541c197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49404/0 (socket says 172.21.15.76:49404) 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 -- 172.21.15.76:0/2375531180 learned_addr learned my addr 172.21.15.76:0/2375531180 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421d51640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f541c102190 0x7f541c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5422552640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1073f0 0x7f541c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 -- 172.21.15.76:0/2375531180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f541c102190 msgr2=0x7f541c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f541c102190 0x7f541c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 -- 172.21.15.76:0/2375531180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1073f0 msgr2=0x7f541c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1073f0 0x7f541c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 -- 172.21.15.76:0/2375531180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c009490 con 0x7f541c104ac0 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5421550640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c104ac0 0x7f541c197d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f541800e9c0 tx=0x7f541800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.616+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418009700 con 0x7f541c104ac0 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.617+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541800cd80 con 0x7f541c104ac0 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.617+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c19f920 con 0x7f541c104ac0 2024-03-23T15:43:21.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.617+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418010430 con 0x7f541c104ac0 2024-03-23T15:43:21.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.617+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c19fe80 con 0x7f541c104ac0 2024-03-23T15:43:21.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.618+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c0fd690 con 0x7f541c104ac0 2024-03-23T15:43:21.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.619+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 21) ==== 136568+0+0 (secure 0 0 0) 0x7f5418005000 con 0x7f541c104ac0 2024-03-23T15:43:21.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.619+0000 7f5412ffd640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54080a1f40 0x7f54080a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.619+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5418014070 con 0x7f541c104ac0 2024-03-23T15:43:21.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.619+0000 7f5421d51640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54080a1f40 0x7f54080a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.620+0000 7f5421d51640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54080a1f40 0x7f54080a4400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f540c02c180 tx=0x7f540c034c00 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:21.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.624+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f5418096ef0 con 0x7f541c104ac0 2024-03-23T15:43:21.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.819+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f541c02d130 con 0x7f541c104ac0 2024-03-23T15:43:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.829+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7f54180972c0 con 0x7f541c104ac0 2024-03-23T15:43:21.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.830+0000 7f5412ffd640 1 -- 172.21.15.76:0/2375531180 <== mon.0 v2:172.21.15.76:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f54180916f0 con 0x7f541c104ac0 2024-03-23T15:43:21.833 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54080a1f40 msgr2=0x7f54080a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54080a1f40 0x7f54080a4400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f540c02c180 tx=0x7f540c034c00 comp rx=0 tx=0).stop 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c104ac0 msgr2=0x7f541c197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c104ac0 0x7f541c197d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f541800e9c0 tx=0x7f541800ee80 comp rx=0 tx=0).stop 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 shutdown_connections 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54080a1f40 0x7f54080a4400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f541c1073f0 0x7f541c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f541c104ac0 0x7f541c197d60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 --2- 172.21.15.76:0/2375531180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f541c102190 0x7f541c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 >> 172.21.15.76:0/2375531180 conn(0x7f541c0fbba0 msgr2=0x7f541c102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 shutdown_connections 2024-03-23T15:43:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.833+0000 7f5423fb5640 1 -- 172.21.15.76:0/2375531180 wait complete. 2024-03-23T15:43:21.849 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 -- 172.21.15.76:0/3746921678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c100610 msgr2=0x7f467c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 --2- 172.21.15.76:0/3746921678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c100610 0x7f467c100a30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4670009900 tx=0x7f467002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 -- 172.21.15.76:0/3746921678 shutdown_connections 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 --2- 172.21.15.76:0/3746921678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f467c1021c0 0x7f467c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 --2- 172.21.15.76:0/3746921678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f467c101800 0x7f467c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 --2- 172.21.15.76:0/3746921678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c100610 0x7f467c100a30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 -- 172.21.15.76:0/3746921678 >> 172.21.15.76:0/3746921678 conn(0x7f467c0fbb80 msgr2=0x7f467c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:21.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 -- 172.21.15.76:0/3746921678 shutdown_connections 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.965+0000 7f468324e640 1 -- 172.21.15.76:0/3746921678 wait complete. 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 Processor -- start 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 -- start start 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f467c100610 0x7f467c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f467c101800 0x7f467c197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c1021c0 0x7f467c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f467c19f2d0 con 0x7f467c1021c0 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f467c19f440 con 0x7f467c101800 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f468324e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f467c19f600 con 0x7f467c100610 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c1021c0 0x7f467c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c1021c0 0x7f467c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49448/0 (socket says 172.21.15.76:49448) 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 -- 172.21.15.76:0/1406729329 learned_addr learned my addr 172.21.15.76:0/1406729329 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:21.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f4680fea640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f467c100610 0x7f467c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f467bfff640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f467c101800 0x7f467c197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 -- 172.21.15.76:0/1406729329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f467c100610 msgr2=0x7f467c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f467c100610 0x7f467c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 -- 172.21.15.76:0/1406729329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f467c101800 msgr2=0x7f467c197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f467c101800 0x7f467c197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.966+0000 7f46817eb640 1 -- 172.21.15.76:0/1406729329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4670009560 con 0x7f467c1021c0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.967+0000 7f46817eb640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c1021c0 0x7f467c19ed90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f467400eaa0 tx=0x7f467400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.967+0000 7f4679ffb640 1 -- 172.21.15.76:0/1406729329 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467400cdb0 con 0x7f467c1021c0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.967+0000 7f4679ffb640 1 -- 172.21.15.76:0/1406729329 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4674005020 con 0x7f467c1021c0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.967+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c19f8e0 con 0x7f467c1021c0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.967+0000 7f4679ffb640 1 -- 172.21.15.76:0/1406729329 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740116b0 con 0x7f467c1021c0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.967+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c19fda0 con 0x7f467c1021c0 2024-03-23T15:43:21.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.968+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467c10bff0 con 0x7f467c1021c0 2024-03-23T15:43:21.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.969+0000 7f4679ffb640 1 -- 172.21.15.76:0/1406729329 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7f4674005520 con 0x7f467c1021c0 2024-03-23T15:43:21.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.969+0000 7f4679ffb640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f466c0a22c0 0x7f466c0a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:21.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.969+0000 7f4680fea640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f466c0a22c0 0x7f466c0a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:21.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.969+0000 7f4679ffb640 1 -- 172.21.15.76:0/1406729329 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f46740ce3c0 con 0x7f467c1021c0 2024-03-23T15:43:21.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.970+0000 7f4680fea640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f466c0a22c0 0x7f466c0a4780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4670005ab0 tx=0x7f4670005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:21.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:21.974+0000 7f4679ffb640 1 -- 172.21.15.76:0/1406729329 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4674096f90 con 0x7f467c1021c0 2024-03-23T15:43:22.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:21 smithi076 ceph-mon[32176]: mgrmap e21: x(active, since 1.03339s), standbys: y, z 2024-03-23T15:43:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:21 smithi076 ceph-mon[32176]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:21 smithi076 ceph-mon[32176]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:21 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3716819037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:21 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/916307080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:21 smithi183 ceph-mon[36296]: mgrmap e21: x(active, since 1.03339s), standbys: y, z 2024-03-23T15:43:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:21 smithi183 ceph-mon[36296]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:21 smithi183 ceph-mon[36296]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:21 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3716819037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:22.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:21 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/916307080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:22.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.162+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f467c1a01e0 con 0x7f467c1021c0 2024-03-23T15:43:22.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.162+0000 7f4679ffb640 1 -- 172.21.15.76:0/1406729329 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f467401f0a0 con 0x7f467c1021c0 2024-03-23T15:43:22.165 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f466c0a22c0 msgr2=0x7f466c0a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f466c0a22c0 0x7f466c0a4780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4670005ab0 tx=0x7f4670005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c1021c0 msgr2=0x7f467c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c1021c0 0x7f467c19ed90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f467400eaa0 tx=0x7f467400ef60 comp rx=0 tx=0).stop 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 shutdown_connections 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f466c0a22c0 0x7f466c0a4780 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467c1021c0 0x7f467c19ed90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f467c101800 0x7f467c197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 --2- 172.21.15.76:0/1406729329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f467c100610 0x7f467c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.164+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 >> 172.21.15.76:0/1406729329 conn(0x7f467c0fbb80 msgr2=0x7f467c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:22.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.165+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 shutdown_connections 2024-03-23T15:43:22.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.165+0000 7f468324e640 1 -- 172.21.15.76:0/1406729329 wait complete. 2024-03-23T15:43:22.179 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:43:22.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 -- 172.21.15.76:0/1719553308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570104ae0 msgr2=0x7fa570106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 --2- 172.21.15.76:0/1719553308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570104ae0 0x7fa570106ed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa564009900 tx=0x7fa56402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:22.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 -- 172.21.15.76:0/1719553308 shutdown_connections 2024-03-23T15:43:22.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 --2- 172.21.15.76:0/1719553308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa570107410 0x7fa570109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 --2- 172.21.15.76:0/1719553308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570104ae0 0x7fa570106ed0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 --2- 172.21.15.76:0/1719553308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5701021b0 0x7fa5701045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 -- 172.21.15.76:0/1719553308 >> 172.21.15.76:0/1719553308 conn(0x7fa5700fbba0 msgr2=0x7fa5700fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:22.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 -- 172.21.15.76:0/1719553308 shutdown_connections 2024-03-23T15:43:22.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 -- 172.21.15.76:0/1719553308 wait complete. 2024-03-23T15:43:22.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.280+0000 7fa5748f4640 1 Processor -- start 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 -- start start 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5701021b0 0x7fa570101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa570104ae0 0x7fa570100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570107410 0x7fa5701006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa570100e00 con 0x7fa570107410 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa570100f70 con 0x7fa5701021b0 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701a1820 con 0x7fa570104ae0 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570107410 0x7fa5701006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570107410 0x7fa5701006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49458/0 (socket says 172.21.15.76:49458) 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 -- 172.21.15.76:0/72236953 learned_addr learned my addr 172.21.15.76:0/72236953 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:22.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56e59c640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5701021b0 0x7fa570101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56dd9b640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa570104ae0 0x7fa570100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 -- 172.21.15.76:0/72236953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa570104ae0 msgr2=0x7fa570100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa570104ae0 0x7fa570100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 -- 172.21.15.76:0/72236953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5701021b0 msgr2=0x7fa570101b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5701021b0 0x7fa570101b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 -- 172.21.15.76:0/72236953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564009560 con 0x7fa570107410 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa56ed9d640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570107410 0x7fa5701006a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa56800eb40 tx=0x7fa56800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa55f7fe640 1 -- 172.21.15.76:0/72236953 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568004b20 con 0x7fa570107410 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5701a1ad0 con 0x7fa570107410 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa55f7fe640 1 -- 172.21.15.76:0/72236953 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa568004c80 con 0x7fa570107410 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.281+0000 7fa55f7fe640 1 -- 172.21.15.76:0/72236953 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568010670 con 0x7fa570107410 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.282+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5701a1f60 con 0x7fa570107410 2024-03-23T15:43:22.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.282+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5700fd690 con 0x7fa570107410 2024-03-23T15:43:22.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.283+0000 7fa55f7fe640 1 -- 172.21.15.76:0/72236953 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7fa568005020 con 0x7fa570107410 2024-03-23T15:43:22.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.284+0000 7fa55f7fe640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa5580a22c0 0x7fa5580a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.284+0000 7fa56e59c640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa5580a22c0 0x7fa5580a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.284+0000 7fa55f7fe640 1 -- 172.21.15.76:0/72236953 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa568014070 con 0x7fa570107410 2024-03-23T15:43:22.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.285+0000 7fa56e59c640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa5580a22c0 0x7fa5580a4780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa560009bd0 tx=0x7fa560005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:22.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.289+0000 7fa55f7fe640 1 -- 172.21.15.76:0/72236953 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa568097320 con 0x7fa570107410 2024-03-23T15:43:22.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:21 smithi192 ceph-mon[36653]: mgrmap e21: x(active, since 1.03339s), standbys: y, z 2024-03-23T15:43:22.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:21 smithi192 ceph-mon[36653]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:22.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:21 smithi192 ceph-mon[36653]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:21 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3716819037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:22.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:21 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/916307080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:22.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.487+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa57010d340 con 0x7fa570107410 2024-03-23T15:43:22.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.489+0000 7fa55f7fe640 1 -- 172.21.15.76:0/72236953 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23107 (secure 0 0 0) 0x7fa56809a9e0 con 0x7fa570107410 2024-03-23T15:43:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:22.493 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":88,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:43:19.788338+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/4023925236":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6801/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6800/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/1911361196":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/4249446235":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/3432469273":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:43:22.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa5580a22c0 msgr2=0x7fa5580a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa5580a22c0 0x7fa5580a4780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa560009bd0 tx=0x7fa560005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570107410 msgr2=0x7fa5701006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570107410 0x7fa5701006a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa56800eb40 tx=0x7fa56800c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 shutdown_connections 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa5580a22c0 0x7fa5580a4780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa570107410 0x7fa5701006a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa570104ae0 0x7fa570100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 --2- 172.21.15.76:0/72236953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5701021b0 0x7fa570101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 >> 172.21.15.76:0/72236953 conn(0x7fa5700fbba0 msgr2=0x7fa570102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 shutdown_connections 2024-03-23T15:43:22.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.491+0000 7fa5748f4640 1 -- 172.21.15.76:0/72236953 wait complete. 2024-03-23T15:43:22.507 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:43:22.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.612+0000 7f42a913e640 1 -- 172.21.15.76:0/248244986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a4104ac0 msgr2=0x7f42a4106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.612+0000 7f42a913e640 1 --2- 172.21.15.76:0/248244986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a4104ac0 0x7f42a4106eb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f428c009900 tx=0x7f428c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- 172.21.15.76:0/248244986 shutdown_connections 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 --2- 172.21.15.76:0/248244986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a41073f0 0x7f42a4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 --2- 172.21.15.76:0/248244986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a4104ac0 0x7f42a4106eb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 --2- 172.21.15.76:0/248244986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42a4102190 0x7f42a4104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- 172.21.15.76:0/248244986 >> 172.21.15.76:0/248244986 conn(0x7f42a40fbb80 msgr2=0x7f42a40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- 172.21.15.76:0/248244986 shutdown_connections 2024-03-23T15:43:22.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- 172.21.15.76:0/248244986 wait complete. 2024-03-23T15:43:22.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 Processor -- start 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- start start 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a4102190 0x7f42a41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42a4104ac0 0x7f42a4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a41073f0 0x7f42a419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42a419f2e0 con 0x7f42a41073f0 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42a419f450 con 0x7f42a4102190 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.613+0000 7f42a913e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f42a419f610 con 0x7f42a4104ac0 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a41073f0 0x7f42a419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f429bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42a4104ac0 0x7f42a4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a41073f0 0x7f42a419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49482/0 (socket says 172.21.15.76:49482) 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a4102190 0x7f42a41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 -- 172.21.15.76:0/206587037 learned_addr learned my addr 172.21.15.76:0/206587037 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:22.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 -- 172.21.15.76:0/206587037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42a4104ac0 msgr2=0x7f42a4197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42a4104ac0 0x7f42a4197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 -- 172.21.15.76:0/206587037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a4102190 msgr2=0x7f42a41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a4102190 0x7f42a41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 -- 172.21.15.76:0/206587037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c009560 con 0x7f42a41073f0 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f429bfff640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42a4104ac0 0x7f42a4197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a359e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a41073f0 0x7f42a419eda0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f429c00eb10 tx=0x7f429c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a0d99640 1 -- 172.21.15.76:0/206587037 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c004b20 con 0x7f42a41073f0 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a0d99640 1 -- 172.21.15.76:0/206587037 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f429c004c80 con 0x7f42a41073f0 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a0d99640 1 -- 172.21.15.76:0/206587037 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c010650 con 0x7f42a41073f0 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a419f8f0 con 0x7f42a41073f0 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.614+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a419fd80 con 0x7f42a41073f0 2024-03-23T15:43:22.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.615+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a40fd670 con 0x7f42a41073f0 2024-03-23T15:43:22.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.616+0000 7f42a0d99640 1 -- 172.21.15.76:0/206587037 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7f429c005020 con 0x7f42a41073f0 2024-03-23T15:43:22.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.617+0000 7f42a0d99640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f42900a22c0 0x7f42900a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.617+0000 7f42a0d99640 1 -- 172.21.15.76:0/206587037 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f429c014070 con 0x7f42a41073f0 2024-03-23T15:43:22.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.617+0000 7f42a2d9d640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f42900a22c0 0x7f42900a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.617+0000 7f42a2d9d640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f42900a22c0 0x7f42900a4780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4294009bd0 tx=0x7f4294005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:22.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.625+0000 7f42a0d99640 1 -- 172.21.15.76:0/206587037 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f429c097320 con 0x7f42a41073f0 2024-03-23T15:43:22.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.808+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42a402d1d0 con 0x7f42a41073f0 2024-03-23T15:43:22.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.808+0000 7f42a0d99640 1 -- 172.21.15.76:0/206587037 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f429c09a9e0 con 0x7f42a41073f0 2024-03-23T15:43:22.812 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:43:22.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f42900a22c0 msgr2=0x7f42900a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f42900a22c0 0x7f42900a4780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4294009bd0 tx=0x7f4294005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:22.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a41073f0 msgr2=0x7f42a419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a41073f0 0x7f42a419eda0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f429c00eb10 tx=0x7f429c00c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 shutdown_connections 2024-03-23T15:43:22.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f42900a22c0 0x7f42900a4780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42a41073f0 0x7f42a419eda0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42a4104ac0 0x7f42a4197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 --2- 172.21.15.76:0/206587037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a4102190 0x7f42a41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.810+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 >> 172.21.15.76:0/206587037 conn(0x7f42a40fbb80 msgr2=0x7f42a4102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.811+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 shutdown_connections 2024-03-23T15:43:22.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.811+0000 7f42a913e640 1 -- 172.21.15.76:0/206587037 wait complete. 2024-03-23T15:43:22.826 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:43:22.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 -- 172.21.15.76:0/3109679218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 msgr2=0x7f9c8c101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 --2- 172.21.15.76:0/3109679218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c101c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009900 tx=0x7f9c7c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 -- 172.21.15.76:0/3109679218 shutdown_connections 2024-03-23T15:43:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 --2- 172.21.15.76:0/3109679218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8c1021c0 0x7f9c8c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 --2- 172.21.15.76:0/3109679218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c101c80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 --2- 172.21.15.76:0/3109679218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8c100610 0x7f9c8c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 -- 172.21.15.76:0/3109679218 >> 172.21.15.76:0/3109679218 conn(0x7f9c8c0fbba0 msgr2=0x7f9c8c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 -- 172.21.15.76:0/3109679218 shutdown_connections 2024-03-23T15:43:22.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.931+0000 7f9c90efb640 1 -- 172.21.15.76:0/3109679218 wait complete. 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 Processor -- start 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 -- start start 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8c100610 0x7f9c8c1045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8c1021c0 0x7f9c8c103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c103880 con 0x7f9c8c101800 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1039f0 con 0x7f9c8c100610 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c90efb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c1a17f0 con 0x7f9c8c1021c0 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c102be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49498/0 (socket says 172.21.15.76:49498) 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 -- 172.21.15.76:0/4142234672 learned_addr learned my addr 172.21.15.76:0/4142234672 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c8a59c640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8c100610 0x7f9c8c1045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c8ad9d640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8c1021c0 0x7f9c8c103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 -- 172.21.15.76:0/4142234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8c1021c0 msgr2=0x7f9c8c103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8c1021c0 0x7f9c8c103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 -- 172.21.15.76:0/4142234672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8c100610 msgr2=0x7f9c8c1045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8c100610 0x7f9c8c1045a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 -- 172.21.15.76:0/4142234672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c009560 con 0x7f9c8c101800 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c89d9b640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c102be0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c036040 tx=0x7f9c7c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.932+0000 7f9c7b7fe640 1 -- 172.21.15.76:0/4142234672 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0043d0 con 0x7f9c8c101800 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.933+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c1a1a40 con 0x7f9c8c101800 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.933+0000 7f9c7b7fe640 1 -- 172.21.15.76:0/4142234672 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c7c0052d0 con 0x7f9c8c101800 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.933+0000 7f9c7b7fe640 1 -- 172.21.15.76:0/4142234672 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c003cc0 con 0x7f9c8c101800 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.933+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c1a1f50 con 0x7f9c8c101800 2024-03-23T15:43:22.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.934+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c10c020 con 0x7f9c8c101800 2024-03-23T15:43:22.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.935+0000 7f9c7b7fe640 1 -- 172.21.15.76:0/4142234672 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7f9c7c005440 con 0x7f9c8c101800 2024-03-23T15:43:22.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.935+0000 7f9c7b7fe640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9c740a22c0 0x7f9c740a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:22.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.935+0000 7f9c8a59c640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9c740a22c0 0x7f9c740a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:22.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.935+0000 7f9c7b7fe640 1 -- 172.21.15.76:0/4142234672 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9c7c0f03c0 con 0x7f9c8c101800 2024-03-23T15:43:22.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.936+0000 7f9c8a59c640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9c740a22c0 0x7f9c740a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c101660 tx=0x7f9c80005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:22.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:22.940+0000 7f9c7b7fe640 1 -- 172.21.15.76:0/4142234672 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9c7c0b8ff0 con 0x7f9c8c101800 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2375531180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1406729329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/72236953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:23.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:22 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/206587037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.122+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c8c1a2140 con 0x7f9c8c101800 2024-03-23T15:43:23.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.122+0000 7f9c7b7fe640 1 -- 172.21.15.76:0/4142234672 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f9c7c0bc6b0 con 0x7f9c8c101800 2024-03-23T15:43:23.126 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:43:23.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.124+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9c740a22c0 msgr2=0x7f9c740a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.124+0000 7f9c90efb640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9c740a22c0 0x7f9c740a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c101660 tx=0x7f9c80005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:23.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.124+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 msgr2=0x7f9c8c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.124+0000 7f9c90efb640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c102be0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c036040 tx=0x7f9c7c005e80 comp rx=0 tx=0).stop 2024-03-23T15:43:23.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 shutdown_connections 2024-03-23T15:43:23.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9c740a22c0 0x7f9c740a4780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c8c1021c0 0x7f9c8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c8c101800 0x7f9c8c102be0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 --2- 172.21.15.76:0/4142234672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c8c100610 0x7f9c8c1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 >> 172.21.15.76:0/4142234672 conn(0x7f9c8c0fbba0 msgr2=0x7f9c8c0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:23.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 shutdown_connections 2024-03-23T15:43:23.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.125+0000 7f9c90efb640 1 -- 172.21.15.76:0/4142234672 wait complete. 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2375531180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1406729329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/72236953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:23.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:22 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/206587037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.140 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 -- 172.21.15.76:0/366408996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4101800 msgr2=0x7f18c4101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/366408996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4101800 0x7f18c4101c80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009900 tx=0x7f18b402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 -- 172.21.15.76:0/366408996 shutdown_connections 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/366408996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18c41021c0 0x7f18c4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/366408996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4101800 0x7f18c4101c80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/366408996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c4100610 0x7f18c4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 -- 172.21.15.76:0/366408996 >> 172.21.15.76:0/366408996 conn(0x7f18c40fbb80 msgr2=0x7f18c40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 -- 172.21.15.76:0/366408996 shutdown_connections 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.246+0000 7f18cc4c9640 1 -- 172.21.15.76:0/366408996 wait complete. 2024-03-23T15:43:23.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 Processor -- start 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 -- start start 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4100610 0x7f18c4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18c4101800 0x7f18c4196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c41021c0 0x7f18c419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f18c419e180 con 0x7f18c4100610 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18c419e2f0 con 0x7f18c41021c0 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18cc4c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18c419e4b0 con 0x7f18c4101800 2024-03-23T15:43:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4100610 0x7f18c4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4100610 0x7f18c4196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49510/0 (socket says 172.21.15.76:49510) 2024-03-23T15:43:23.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 -- 172.21.15.76:0/3213391654 learned_addr learned my addr 172.21.15.76:0/3213391654 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:23.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18caa66640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c41021c0 0x7f18c419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18c9a64640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18c4101800 0x7f18c4196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 -- 172.21.15.76:0/3213391654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18c4101800 msgr2=0x7f18c4196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18c4101800 0x7f18c4196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 -- 172.21.15.76:0/3213391654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c41021c0 msgr2=0x7f18c419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c41021c0 0x7f18c419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 -- 172.21.15.76:0/3213391654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc009490 con 0x7f18c4100610 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.247+0000 7f18ca265640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4100610 0x7f18c4196690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00e9c0 tx=0x7f18bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.248+0000 7f18bb7fe640 1 -- 172.21.15.76:0/3213391654 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00cde0 con 0x7f18c4100610 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.248+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b4009560 con 0x7f18c4100610 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.248+0000 7f18bb7fe640 1 -- 172.21.15.76:0/3213391654 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18bc010910 con 0x7f18c4100610 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.248+0000 7f18bb7fe640 1 -- 172.21.15.76:0/3213391654 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc018960 con 0x7f18c4100610 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.248+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c419eb30 con 0x7f18c4100610 2024-03-23T15:43:23.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.249+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c4068500 con 0x7f18c4100610 2024-03-23T15:43:23.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.250+0000 7f18bb7fe640 1 -- 172.21.15.76:0/3213391654 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7f18bc010a80 con 0x7f18c4100610 2024-03-23T15:43:23.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.250+0000 7f18bb7fe640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f18b00a22c0 0x7f18b00a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.250+0000 7f18bb7fe640 1 -- 172.21.15.76:0/3213391654 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f18bc014070 con 0x7f18c4100610 2024-03-23T15:43:23.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.250+0000 7f18c9a64640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f18b00a22c0 0x7f18b00a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.251+0000 7f18c9a64640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f18b00a22c0 0x7f18b00a4780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009230 tx=0x7f18b4005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:23.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.255+0000 7f18bb7fe640 1 -- 172.21.15.76:0/3213391654 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f18bc097310 con 0x7f18c4100610 2024-03-23T15:43:23.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:23.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2375531180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1406729329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/72236953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:22 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/206587037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.441+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18c4107930 con 0x7f18c4100610 2024-03-23T15:43:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.441+0000 7f18bb7fe640 1 -- 172.21.15.76:0/3213391654 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f18bc09a9d0 con 0x7f18c4100610 2024-03-23T15:43:23.444 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:43:23.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f18b00a22c0 msgr2=0x7f18b00a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f18b00a22c0 0x7f18b00a4780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009230 tx=0x7f18b4005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4100610 msgr2=0x7f18c4196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4100610 0x7f18c4196690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00e9c0 tx=0x7f18bc00ee80 comp rx=0 tx=0).stop 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 shutdown_connections 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f18b00a22c0 0x7f18b00a4780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c41021c0 0x7f18c419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18c4101800 0x7f18c4196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 --2- 172.21.15.76:0/3213391654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18c4100610 0x7f18c4196690 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.443+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 >> 172.21.15.76:0/3213391654 conn(0x7f18c40fbb80 msgr2=0x7f18c4104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.444+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 shutdown_connections 2024-03-23T15:43:23.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.444+0000 7f18cc4c9640 1 -- 172.21.15.76:0/3213391654 wait complete. 2024-03-23T15:43:23.458 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-23T15:43:23.458 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:43:23.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 -- 172.21.15.76:0/1231660615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101c20 msgr2=0x7f2db8102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/1231660615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101c20 0x7f2db8102040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009900 tx=0x7f2da802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:23.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 -- 172.21.15.76:0/1231660615 shutdown_connections 2024-03-23T15:43:23.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/1231660615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db8101280 0x7f2db8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/1231660615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 0x7f2db8100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/1231660615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101c20 0x7f2db8102040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 -- 172.21.15.76:0/1231660615 >> 172.21.15.76:0/1231660615 conn(0x7f2db80fb2f0 msgr2=0x7f2db80fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 -- 172.21.15.76:0/1231660615 shutdown_connections 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 -- 172.21.15.76:0/1231660615 wait complete. 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.577+0000 7f2dbddaf640 1 Processor -- start 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2dbddaf640 1 -- start start 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2dbddaf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 0x7f2db8100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2dbddaf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101280 0x7f2db819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2dbddaf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db8101c20 0x7f2db819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2dbddaf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2db819f320 con 0x7f2db81008c0 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2dbddaf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2db819f490 con 0x7f2db8101280 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2dbddaf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2db819f650 con 0x7f2db8101c20 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 0x7f2db8100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 0x7f2db8100320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49530/0 (socket says 172.21.15.76:49530) 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101280 0x7f2db819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 -- 172.21.15.76:0/2788240890 learned_addr learned my addr 172.21.15.76:0/2788240890 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db7fff640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db8101c20 0x7f2db819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 -- 172.21.15.76:0/2788240890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db8101c20 msgr2=0x7f2db819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db8101c20 0x7f2db819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 -- 172.21.15.76:0/2788240890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101280 msgr2=0x7f2db819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101280 0x7f2db819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 -- 172.21.15.76:0/2788240890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac009490 con 0x7f2db81008c0 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db6ffd640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101280 0x7f2db819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.578+0000 7f2db77fe640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 0x7f2db8100320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2da8000c00 tx=0x7f2da8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.579+0000 7f2db4ff9640 1 -- 172.21.15.76:0/2788240890 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8004aa0 con 0x7f2db81008c0 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.579+0000 7f2db4ff9640 1 -- 172.21.15.76:0/2788240890 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da80052d0 con 0x7f2db81008c0 2024-03-23T15:43:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.579+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da8009560 con 0x7f2db81008c0 2024-03-23T15:43:23.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.579+0000 7f2db4ff9640 1 -- 172.21.15.76:0/2788240890 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8003e60 con 0x7f2db81008c0 2024-03-23T15:43:23.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.579+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db819fbf0 con 0x7f2db81008c0 2024-03-23T15:43:23.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.580+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db8107a00 con 0x7f2db81008c0 2024-03-23T15:43:23.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.581+0000 7f2db4ff9640 1 -- 172.21.15.76:0/2788240890 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7f2da8005440 con 0x7f2db81008c0 2024-03-23T15:43:23.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.581+0000 7f2db4ff9640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2da00a22c0 0x7f2da00a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.582+0000 7f2db4ff9640 1 -- 172.21.15.76:0/2788240890 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f2da80f0b30 con 0x7f2db81008c0 2024-03-23T15:43:23.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.582+0000 7f2db6ffd640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2da00a22c0 0x7f2da00a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.582+0000 7f2db6ffd640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2da00a22c0 0x7f2da00a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2db8100090 tx=0x7f2dac005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:23.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.588+0000 7f2db4ff9640 1 -- 172.21.15.76:0/2788240890 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2da80b9760 con 0x7f2db81008c0 2024-03-23T15:43:23.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.819+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2db802d1d0 con 0x7f2db81008c0 2024-03-23T15:43:23.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.820+0000 7f2db4ff9640 1 -- 172.21.15.76:0/2788240890 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f2da80bce20 con 0x7f2db81008c0 2024-03-23T15:43:23.823 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:23.823 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:43:18.737936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24463,24449]},"id":1}]} 2024-03-23T15:43:23.823 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 22 2024-03-23T15:43:23.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.822+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2da00a22c0 msgr2=0x7f2da00a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.822+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2da00a22c0 0x7f2da00a4780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2db8100090 tx=0x7f2dac005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:23.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.822+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 msgr2=0x7f2db8100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.822+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 0x7f2db8100320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2da8000c00 tx=0x7f2da8004270 comp rx=0 tx=0).stop 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 shutdown_connections 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2da00a22c0 0x7f2da00a4780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2db8101c20 0x7f2db819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db8101280 0x7f2db819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 --2- 172.21.15.76:0/2788240890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2db81008c0 0x7f2db8100320 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 >> 172.21.15.76:0/2788240890 conn(0x7f2db80fb2f0 msgr2=0x7f2db8104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 shutdown_connections 2024-03-23T15:43:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.823+0000 7f2dbddaf640 1 -- 172.21.15.76:0/2788240890 wait complete. 2024-03-23T15:43:23.837 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-23T15:43:23.838 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-23T15:43:23.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 -- 172.21.15.76:0/367858612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa338100640 msgr2=0x7fa338100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/367858612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa338100640 0x7fa338100a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009900 tx=0x7fa32c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 -- 172.21.15.76:0/367858612 shutdown_connections 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/367858612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3381021f0 0x7fa338108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/367858612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa338101830 0x7fa338101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/367858612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa338100640 0x7fa338100a60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 -- 172.21.15.76:0/367858612 >> 172.21.15.76:0/367858612 conn(0x7fa3380fbbf0 msgr2=0x7fa3380fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 -- 172.21.15.76:0/367858612 shutdown_connections 2024-03-23T15:43:23.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.940+0000 7fa33e6c3640 1 -- 172.21.15.76:0/367858612 wait complete. 2024-03-23T15:43:23.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 Processor -- start 2024-03-23T15:43:23.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 -- start start 2024-03-23T15:43:23.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa338100640 0x7fa338197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa338101830 0x7fa338197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3381021f0 0x7fa33819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa33819f2d0 con 0x7fa3381021f0 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa33819f440 con 0x7fa338101830 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33e6c3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa33819f600 con 0x7fa338100640 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33cc60640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3381021f0 0x7fa33819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33cc60640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3381021f0 0x7fa33819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49546/0 (socket says 172.21.15.76:49546) 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa33cc60640 1 -- 172.21.15.76:0/567904426 learned_addr learned my addr 172.21.15.76:0/567904426 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.941+0000 7fa3377fe640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa338101830 0x7fa338197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa337fff640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa338100640 0x7fa338197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33cc60640 1 -- 172.21.15.76:0/567904426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa338100640 msgr2=0x7fa338197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33cc60640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa338100640 0x7fa338197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33cc60640 1 -- 172.21.15.76:0/567904426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa338101830 msgr2=0x7fa338197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33cc60640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa338101830 0x7fa338197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33cc60640 1 -- 172.21.15.76:0/567904426 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c009560 con 0x7fa3381021f0 2024-03-23T15:43:23.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33cc60640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3381021f0 0x7fa33819ed90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa33000eb10 tx=0x7fa33000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:23.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa3357fa640 1 -- 172.21.15.76:0/567904426 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330004b20 con 0x7fa3381021f0 2024-03-23T15:43:23.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33819f8e0 con 0x7fa3381021f0 2024-03-23T15:43:23.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa3357fa640 1 -- 172.21.15.76:0/567904426 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330004c80 con 0x7fa3381021f0 2024-03-23T15:43:23.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa3357fa640 1 -- 172.21.15.76:0/567904426 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330010650 con 0x7fa3381021f0 2024-03-23T15:43:23.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.942+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33819fda0 con 0x7fa3381021f0 2024-03-23T15:43:23.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.944+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33810c050 con 0x7fa3381021f0 2024-03-23T15:43:23.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.945+0000 7fa3357fa640 1 -- 172.21.15.76:0/567904426 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) ==== 136927+0+0 (secure 0 0 0) 0x7fa330005020 con 0x7fa3381021f0 2024-03-23T15:43:23.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.945+0000 7fa3357fa640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3200a22c0 0x7fa3200a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:23.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.945+0000 7fa337fff640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3200a22c0 0x7fa3200a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:23.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.946+0000 7fa3357fa640 1 -- 172.21.15.76:0/567904426 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa330014070 con 0x7fa3381021f0 2024-03-23T15:43:23.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.946+0000 7fa337fff640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3200a22c0 0x7fa3200a4780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa32c005ab0 tx=0x7fa32c005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:23.950+0000 7fa3357fa640 1 -- 172.21.15.76:0/567904426 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa330097320 con 0x7fa3381021f0 2024-03-23T15:43:24.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.133+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa33806a3f0 con 0x7fa3200a22c0 2024-03-23T15:43:24.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.141+0000 7fa3357fa640 1 -- 172.21.15.76:0/567904426 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258021 (secure 0 0 0) 0x7fa33806a3f0 con 0x7fa3200a22c0 2024-03-23T15:43:24.147 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:24.148 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3200a22c0 msgr2=0x7fa3200a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3200a22c0 0x7fa3200a4780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa32c005ab0 tx=0x7fa32c005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3381021f0 msgr2=0x7fa33819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3381021f0 0x7fa33819ed90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa33000eb10 tx=0x7fa33000c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 shutdown_connections 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3200a22c0 0x7fa3200a4780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3381021f0 0x7fa33819ed90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa338101830 0x7fa338197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 --2- 172.21.15.76:0/567904426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa338100640 0x7fa338197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 >> 172.21.15.76:0/567904426 conn(0x7fa3380fbbf0 msgr2=0x7fa338069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 shutdown_connections 2024-03-23T15:43:24.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.144+0000 7fa33e6c3640 1 -- 172.21.15.76:0/567904426 wait complete. 2024-03-23T15:43:24.158 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.158 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.158 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.158 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Unable to set osd_memory_target on smithi192 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/4142234672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi076 to 176.2M 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Unable to set osd_memory_target on smithi076 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.159 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi183 to 174.1M 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Unable to set osd_memory_target on smithi183 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Updating smithi076:/etc/ceph/ceph.conf 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3213391654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.160 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.161 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.161 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:24 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2788240890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:24.161 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-03-23T15:43:23.805884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188100,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":966,"num_read_kb":637,"num_write":468,"num_write_kb":4375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369360,"kb_used_data":11840,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572464,"statfs":{"total":1151940427776,"available":1151562203136,"internally_reserved":0,"allocated":12124160,"data_stored":9129175,"data_compressed":293061,"data_compressed_allocated":393216,"data_compressed_original":3637248,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.975010"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795300+0000","last_change":"2024-03-23T15:41:18.651731+0000","last_active":"2024-03-23T15:43:19.795300+0000","last_peered":"2024-03-23T15:43:19.795300+0000","last_clean":"2024-03-23T15:43:19.795300+0000","last_became_active":"2024-03-23T15:41:18.651312+0000","last_became_peered":"2024-03-23T15:41:18.651312+0000","last_unstale":"2024-03-23T15:43:19.795300+0000","last_undegraded":"2024-03-23T15:43:19.795300+0000","last_fullsized":"2024-03-23T15:43:19.795300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:39:20.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855216+0000","last_change":"2024-03-23T15:41:18.655736+0000","last_active":"2024-03-23T15:43:19.855216+0000","last_peered":"2024-03-23T15:43:19.855216+0000","last_clean":"2024-03-23T15:43:19.855216+0000","last_became_active":"2024-03-23T15:41:18.655389+0000","last_became_peered":"2024-03-23T15:41:18.655389+0000","last_unstale":"2024-03-23T15:43:19.855216+0000","last_undegraded":"2024-03-23T15:43:19.855216+0000","last_fullsized":"2024-03-23T15:43:19.855216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:35:56.260056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855070+0000","last_change":"2024-03-23T15:41:18.650709+0000","last_active":"2024-03-23T15:43:19.855070+0000","last_peered":"2024-03-23T15:43:19.855070+0000","last_clean":"2024-03-23T15:43:19.855070+0000","last_became_active":"2024-03-23T15:41:18.650316+0000","last_became_peered":"2024-03-23T15:41:18.650316+0000","last_unstale":"2024-03-23T15:43:19.855070+0000","last_undegraded":"2024-03-23T15:43:19.855070+0000","last_fullsized":"2024-03-23T15:43:19.855070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:50:57.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830666+0000","last_change":"2024-03-23T15:41:18.652216+0000","last_active":"2024-03-23T15:43:19.830666+0000","last_peered":"2024-03-23T15:43:19.830666+0000","last_clean":"2024-03-23T15:43:19.830666+0000","last_became_active":"2024-03-23T15:41:18.650525+0000","last_became_peered":"2024-03-23T15:41:18.650525+0000","last_unstale":"2024-03-23T15:43:19.830666+0000","last_undegraded":"2024-03-23T15:43:19.830666+0000","last_fullsized":"2024-03-23T15:43:19.830666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:16.458257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854905+0000","last_change":"2024-03-23T15:41:18.649146+0000","last_active":"2024-03-23T15:43:19.854905+0000","last_peered":"2024-03-23T15:43:19.854905+0000","last_clean":"2024-03-23T15:43:19.854905+0000","last_became_active":"2024-03-23T15:41:18.648728+0000","last_became_peered":"2024-03-23T15:41:18.648728+0000","last_unstale":"2024-03-23T15:43:19.854905+0000","last_undegraded":"2024-03-23T15:43:19.854905+0000","last_fullsized":"2024-03-23T15:43:19.854905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:56.251250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794938+0000","last_change":"2024-03-23T15:41:18.652957+0000","last_active":"2024-03-23T15:43:19.794938+0000","last_peered":"2024-03-23T15:43:19.794938+0000","last_clean":"2024-03-23T15:43:19.794938+0000","last_became_active":"2024-03-23T15:41:18.652578+0000","last_became_peered":"2024-03-23T15:41:18.652578+0000","last_unstale":"2024-03-23T15:43:19.794938+0000","last_undegraded":"2024-03-23T15:43:19.794938+0000","last_fullsized":"2024-03-23T15:43:19.794938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:46:38.961554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855251+0000","last_change":"2024-03-23T15:41:18.646134+0000","last_active":"2024-03-23T15:43:19.855251+0000","last_peered":"2024-03-23T15:43:19.855251+0000","last_clean":"2024-03-23T15:43:19.855251+0000","last_became_active":"2024-03-23T15:41:18.645680+0000","last_became_peered":"2024-03-23T15:41:18.645680+0000","last_unstale":"2024-03-23T15:43:19.855251+0000","last_undegraded":"2024-03-23T15:43:19.855251+0000","last_fullsized":"2024-03-23T15:43:19.855251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:34:31.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902586+0000","last_change":"2024-03-23T15:41:18.646143+0000","last_active":"2024-03-23T15:43:19.902586+0000","last_peered":"2024-03-23T15:43:19.902586+0000","last_clean":"2024-03-23T15:43:19.902586+0000","last_became_active":"2024-03-23T15:41:18.645771+0000","last_became_peered":"2024-03-23T15:41:18.645771+0000","last_unstale":"2024-03-23T15:43:19.902586+0000","last_undegraded":"2024-03-23T15:43:19.902586+0000","last_fullsized":"2024-03-23T15:43:19.902586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:39.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855670+0000","last_change":"2024-03-23T15:41:18.665083+0000","last_active":"2024-03-23T15:43:19.855670+0000","last_peered":"2024-03-23T15:43:19.855670+0000","last_clean":"2024-03-23T15:43:19.855670+0000","last_became_active":"2024-03-23T15:41:18.663143+0000","last_became_peered":"2024-03-23T15:41:18.663143+0000","last_unstale":"2024-03-23T15:43:19.855670+0000","last_undegraded":"2024-03-23T15:43:19.855670+0000","last_fullsized":"2024-03-23T15:43:19.855670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:52:23.421645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795361+0000","last_change":"2024-03-23T15:41:18.650014+0000","last_active":"2024-03-23T15:43:19.795361+0000","last_peered":"2024-03-23T15:43:19.795361+0000","last_clean":"2024-03-23T15:43:19.795361+0000","last_became_active":"2024-03-23T15:41:18.649673+0000","last_became_peered":"2024-03-23T15:41:18.649673+0000","last_unstale":"2024-03-23T15:43:19.795361+0000","last_undegraded":"2024-03-23T15:43:19.795361+0000","last_fullsized":"2024-03-23T15:43:19.795361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:30.387993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855606+0000","last_change":"2024-03-23T15:41:18.665622+0000","last_active":"2024-03-23T15:43:19.855606+0000","last_peered":"2024-03-23T15:43:19.855606+0000","last_clean":"2024-03-23T15:43:19.855606+0000","last_became_active":"2024-03-23T15:41:18.665274+0000","last_became_peered":"2024-03-23T15:41:18.665274+0000","last_unstale":"2024-03-23T15:43:19.855606+0000","last_undegraded":"2024-03-23T15:43:19.855606+0000","last_fullsized":"2024-03-23T15:43:19.855606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:12:18.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830547+0000","last_change":"2024-03-23T15:41:18.650888+0000","last_active":"2024-03-23T15:43:19.830547+0000","last_peered":"2024-03-23T15:43:19.830547+0000","last_clean":"2024-03-23T15:43:19.830547+0000","last_became_active":"2024-03-23T15:41:18.650363+0000","last_became_peered":"2024-03-23T15:41:18.650363+0000","last_unstale":"2024-03-23T15:43:19.830547+0000","last_undegraded":"2024-03-23T15:43:19.830547+0000","last_fullsized":"2024-03-23T15:43:19.830547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:41.289697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828315+0000","last_change":"2024-03-23T15:41:18.634183+0000","last_active":"2024-03-23T15:43:19.828315+0000","last_peered":"2024-03-23T15:43:19.828315+0000","last_clean":"2024-03-23T15:43:19.828315+0000","last_became_active":"2024-03-23T15:41:18.633849+0000","last_became_peered":"2024-03-23T15:41:18.633849+0000","last_unstale":"2024-03-23T15:43:19.828315+0000","last_undegraded":"2024-03-23T15:43:19.828315+0000","last_fullsized":"2024-03-23T15:43:19.828315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:13.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854833+0000","last_change":"2024-03-23T15:41:18.645269+0000","last_active":"2024-03-23T15:43:19.854833+0000","last_peered":"2024-03-23T15:43:19.854833+0000","last_clean":"2024-03-23T15:43:19.854833+0000","last_became_active":"2024-03-23T15:41:18.644940+0000","last_became_peered":"2024-03-23T15:41:18.644940+0000","last_unstale":"2024-03-23T15:43:19.854833+0000","last_undegraded":"2024-03-23T15:43:19.854833+0000","last_fullsized":"2024-03-23T15:43:19.854833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:26.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854878+0000","last_change":"2024-03-23T15:41:18.652789+0000","last_active":"2024-03-23T15:43:19.854878+0000","last_peered":"2024-03-23T15:43:19.854878+0000","last_clean":"2024-03-23T15:43:19.854878+0000","last_became_active":"2024-03-23T15:41:18.651520+0000","last_became_peered":"2024-03-23T15:41:18.651520+0000","last_unstale":"2024-03-23T15:43:19.854878+0000","last_undegraded":"2024-03-23T15:43:19.854878+0000","last_fullsized":"2024-03-23T15:43:19.854878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:00.388217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854859+0000","last_change":"2024-03-23T15:41:18.660398+0000","last_active":"2024-03-23T15:43:19.854859+0000","last_peered":"2024-03-23T15:43:19.854859+0000","last_clean":"2024-03-23T15:43:19.854859+0000","last_became_active":"2024-03-23T15:41:18.660084+0000","last_became_peered":"2024-03-23T15:41:18.660084+0000","last_unstale":"2024-03-23T15:43:19.854859+0000","last_undegraded":"2024-03-23T15:43:19.854859+0000","last_fullsized":"2024-03-23T15:43:19.854859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:04:33.115173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795310+0000","last_change":"2024-03-23T15:41:18.663192+0000","last_active":"2024-03-23T15:43:19.795310+0000","last_peered":"2024-03-23T15:43:19.795310+0000","last_clean":"2024-03-23T15:43:19.795310+0000","last_became_active":"2024-03-23T15:41:18.662844+0000","last_became_peered":"2024-03-23T15:41:18.662844+0000","last_unstale":"2024-03-23T15:43:19.795310+0000","last_undegraded":"2024-03-23T15:43:19.795310+0000","last_fullsized":"2024-03-23T15:43:19.795310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:24.260706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793526+0000","last_change":"2024-03-23T15:41:18.646405+0000","last_active":"2024-03-23T15:43:19.793526+0000","last_peered":"2024-03-23T15:43:19.793526+0000","last_clean":"2024-03-23T15:43:19.793526+0000","last_became_active":"2024-03-23T15:41:18.645516+0000","last_became_peered":"2024-03-23T15:41:18.645516+0000","last_unstale":"2024-03-23T15:43:19.793526+0000","last_undegraded":"2024-03-23T15:43:19.793526+0000","last_fullsized":"2024-03-23T15:43:19.793526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:29:18.588190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855413+0000","last_change":"2024-03-23T15:42:16.843159+0000","last_active":"2024-03-23T15:43:19.855413+0000","last_peered":"2024-03-23T15:43:19.855413+0000","last_clean":"2024-03-23T15:43:19.855413+0000","last_became_active":"2024-03-23T15:42:16.842729+0000","last_became_peered":"2024-03-23T15:42:16.842729+0000","last_unstale":"2024-03-23T15:43:19.855413+0000","last_undegraded":"2024-03-23T15:43:19.855413+0000","last_fullsized":"2024-03-23T15:43:19.855413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:57:18.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793618+0000","last_change":"2024-03-23T15:41:17.655443+0000","last_active":"2024-03-23T15:43:19.793618+0000","last_peered":"2024-03-23T15:43:19.793618+0000","last_clean":"2024-03-23T15:43:19.793618+0000","last_became_active":"2024-03-23T15:41:17.654834+0000","last_became_peered":"2024-03-23T15:41:17.654834+0000","last_unstale":"2024-03-23T15:43:19.793618+0000","last_undegraded":"2024-03-23T15:43:19.793618+0000","last_fullsized":"2024-03-23T15:43:19.793618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:49:43.215644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903720+0000","last_change":"2024-03-23T15:41:18.650241+0000","last_active":"2024-03-23T15:43:19.903720+0000","last_peered":"2024-03-23T15:43:19.903720+0000","last_clean":"2024-03-23T15:43:19.903720+0000","last_became_active":"2024-03-23T15:41:18.649914+0000","last_became_peered":"2024-03-23T15:41:18.649914+0000","last_unstale":"2024-03-23T15:43:19.903720+0000","last_undegraded":"2024-03-23T15:43:19.903720+0000","last_fullsized":"2024-03-23T15:43:19.903720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:35:18.892649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903809+0000","last_change":"2024-03-23T15:41:17.663308+0000","last_active":"2024-03-23T15:43:19.903809+0000","last_peered":"2024-03-23T15:43:19.903809+0000","last_clean":"2024-03-23T15:43:19.903809+0000","last_became_active":"2024-03-23T15:41:17.662958+0000","last_became_peered":"2024-03-23T15:41:17.662958+0000","last_unstale":"2024-03-23T15:43:19.903809+0000","last_undegraded":"2024-03-23T15:43:19.903809+0000","last_fullsized":"2024-03-23T15:43:19.903809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:43:37.512893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855316+0000","last_change":"2024-03-23T15:41:18.651225+0000","last_active":"2024-03-23T15:43:19.855316+0000","last_peered":"2024-03-23T15:43:19.855316+0000","last_clean":"2024-03-23T15:43:19.855316+0000","last_became_active":"2024-03-23T15:41:18.650884+0000","last_became_peered":"2024-03-23T15:41:18.650884+0000","last_unstale":"2024-03-23T15:43:19.855316+0000","last_undegraded":"2024-03-23T15:43:19.855316+0000","last_fullsized":"2024-03-23T15:43:19.855316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:00.445203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855793+0000","last_change":"2024-03-23T15:41:17.662753+0000","last_active":"2024-03-23T15:43:19.855793+0000","last_peered":"2024-03-23T15:43:19.855793+0000","last_clean":"2024-03-23T15:43:19.855793+0000","last_became_active":"2024-03-23T15:41:17.662378+0000","last_became_peered":"2024-03-23T15:41:17.662378+0000","last_unstale":"2024-03-23T15:43:19.855793+0000","last_undegraded":"2024-03-23T15:43:19.855793+0000","last_fullsized":"2024-03-23T15:43:19.855793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:58:55.674739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854935+0000","last_change":"2024-03-23T15:41:18.659171+0000","last_active":"2024-03-23T15:43:19.854935+0000","last_peered":"2024-03-23T15:43:19.854935+0000","last_clean":"2024-03-23T15:43:19.854935+0000","last_became_active":"2024-03-23T15:41:18.658836+0000","last_became_peered":"2024-03-23T15:41:18.658836+0000","last_unstale":"2024-03-23T15:43:19.854935+0000","last_undegraded":"2024-03-23T15:43:19.854935+0000","last_fullsized":"2024-03-23T15:43:19.854935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:15.056528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794998+0000","last_change":"2024-03-23T15:42:16.631598+0000","last_active":"2024-03-23T15:43:19.794998+0000","last_peered":"2024-03-23T15:43:19.794998+0000","last_clean":"2024-03-23T15:43:19.794998+0000","last_became_active":"2024-03-23T15:42:16.626457+0000","last_became_peered":"2024-03-23T15:42:16.626457+0000","last_unstale":"2024-03-23T15:43:19.794998+0000","last_undegraded":"2024-03-23T15:43:19.794998+0000","last_fullsized":"2024-03-23T15:43:19.794998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:29.776204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829447+0000","last_change":"2024-03-23T15:41:18.648500+0000","last_active":"2024-03-23T15:43:19.829447+0000","last_peered":"2024-03-23T15:43:19.829447+0000","last_clean":"2024-03-23T15:43:19.829447+0000","last_became_active":"2024-03-23T15:41:18.648139+0000","last_became_peered":"2024-03-23T15:41:18.648139+0000","last_unstale":"2024-03-23T15:43:19.829447+0000","last_undegraded":"2024-03-23T15:43:19.829447+0000","last_fullsized":"2024-03-23T15:43:19.829447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:38.071057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:20.032863+0000","last_change":"2024-03-23T15:41:17.658471+0000","last_active":"2024-03-23T15:43:20.032863+0000","last_peered":"2024-03-23T15:43:20.032863+0000","last_clean":"2024-03-23T15:43:20.032863+0000","last_became_active":"2024-03-23T15:41:17.658142+0000","last_became_peered":"2024-03-23T15:41:17.658142+0000","last_unstale":"2024-03-23T15:43:20.032863+0000","last_undegraded":"2024-03-23T15:43:20.032863+0000","last_fullsized":"2024-03-23T15:43:20.032863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:23.224790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855346+0000","last_change":"2024-03-23T15:41:18.646046+0000","last_active":"2024-03-23T15:43:19.855346+0000","last_peered":"2024-03-23T15:43:19.855346+0000","last_clean":"2024-03-23T15:43:19.855346+0000","last_became_active":"2024-03-23T15:41:18.645727+0000","last_became_peered":"2024-03-23T15:41:18.645727+0000","last_unstale":"2024-03-23T15:43:19.855346+0000","last_undegraded":"2024-03-23T15:43:19.855346+0000","last_fullsized":"2024-03-23T15:43:19.855346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:55.552079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794899+0000","last_change":"2024-03-23T15:41:17.657285+0000","last_active":"2024-03-23T15:43:19.794899+0000","last_peered":"2024-03-23T15:43:19.794899+0000","last_clean":"2024-03-23T15:43:19.794899+0000","last_became_active":"2024-03-23T15:41:17.656942+0000","last_became_peered":"2024-03-23T15:41:17.656942+0000","last_unstale":"2024-03-23T15:43:19.794899+0000","last_undegraded":"2024-03-23T15:43:19.794899+0000","last_fullsized":"2024-03-23T15:43:19.794899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:18.107467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829486+0000","last_change":"2024-03-23T15:41:18.655175+0000","last_active":"2024-03-23T15:43:19.829486+0000","last_peered":"2024-03-23T15:43:19.829486+0000","last_clean":"2024-03-23T15:43:19.829486+0000","last_became_active":"2024-03-23T15:41:18.654846+0000","last_became_peered":"2024-03-23T15:41:18.654846+0000","last_unstale":"2024-03-23T15:43:19.829486+0000","last_undegraded":"2024-03-23T15:43:19.829486+0000","last_fullsized":"2024-03-23T15:43:19.829486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:43:10.008538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828140+0000","last_change":"2024-03-23T15:41:17.657391+0000","last_active":"2024-03-23T15:43:19.828140+0000","last_peered":"2024-03-23T15:43:19.828140+0000","last_clean":"2024-03-23T15:43:19.828140+0000","last_became_active":"2024-03-23T15:41:17.657081+0000","last_became_peered":"2024-03-23T15:41:17.657081+0000","last_unstale":"2024-03-23T15:43:19.828140+0000","last_undegraded":"2024-03-23T15:43:19.828140+0000","last_fullsized":"2024-03-23T15:43:19.828140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:08.744477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902586+0000","last_change":"2024-03-23T15:41:18.656268+0000","last_active":"2024-03-23T15:43:19.902586+0000","last_peered":"2024-03-23T15:43:19.902586+0000","last_clean":"2024-03-23T15:43:19.902586+0000","last_became_active":"2024-03-23T15:41:18.655910+0000","last_became_peered":"2024-03-23T15:41:18.655910+0000","last_unstale":"2024-03-23T15:43:19.902586+0000","last_undegraded":"2024-03-23T15:43:19.902586+0000","last_fullsized":"2024-03-23T15:43:19.902586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:38:57.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829296+0000","last_change":"2024-03-23T15:41:17.656555+0000","last_active":"2024-03-23T15:43:19.829296+0000","last_peered":"2024-03-23T15:43:19.829296+0000","last_clean":"2024-03-23T15:43:19.829296+0000","last_became_active":"2024-03-23T15:41:17.655732+0000","last_became_peered":"2024-03-23T15:41:17.655732+0000","last_unstale":"2024-03-23T15:43:19.829296+0000","last_undegraded":"2024-03-23T15:43:19.829296+0000","last_fullsized":"2024-03-23T15:43:19.829296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:53.839841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793401+0000","last_change":"2024-03-23T15:41:18.647334+0000","last_active":"2024-03-23T15:43:19.793401+0000","last_peered":"2024-03-23T15:43:19.793401+0000","last_clean":"2024-03-23T15:43:19.793401+0000","last_became_active":"2024-03-23T15:41:18.646679+0000","last_became_peered":"2024-03-23T15:41:18.646679+0000","last_unstale":"2024-03-23T15:43:19.793401+0000","last_undegraded":"2024-03-23T15:43:19.793401+0000","last_fullsized":"2024-03-23T15:43:19.793401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.710759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.827515+0000","last_change":"2024-03-23T15:41:17.655400+0000","last_active":"2024-03-23T15:43:19.827515+0000","last_peered":"2024-03-23T15:43:19.827515+0000","last_clean":"2024-03-23T15:43:19.827515+0000","last_became_active":"2024-03-23T15:41:17.655034+0000","last_became_peered":"2024-03-23T15:41:17.655034+0000","last_unstale":"2024-03-23T15:43:19.827515+0000","last_undegraded":"2024-03-23T15:43:19.827515+0000","last_fullsized":"2024-03-23T15:43:19.827515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:30.973048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855488+0000","last_change":"2024-03-23T15:41:18.660248+0000","last_active":"2024-03-23T15:43:19.855488+0000","last_peered":"2024-03-23T15:43:19.855488+0000","last_clean":"2024-03-23T15:43:19.855488+0000","last_became_active":"2024-03-23T15:41:18.659728+0000","last_became_peered":"2024-03-23T15:41:18.659728+0000","last_unstale":"2024-03-23T15:43:19.855488+0000","last_undegraded":"2024-03-23T15:43:19.855488+0000","last_fullsized":"2024-03-23T15:43:19.855488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:03.681499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855657+0000","last_change":"2024-03-23T15:41:17.659784+0000","last_active":"2024-03-23T15:43:19.855657+0000","last_peered":"2024-03-23T15:43:19.855657+0000","last_clean":"2024-03-23T15:43:19.855657+0000","last_became_active":"2024-03-23T15:41:17.659422+0000","last_became_peered":"2024-03-23T15:41:17.659422+0000","last_unstale":"2024-03-23T15:43:19.855657+0000","last_undegraded":"2024-03-23T15:43:19.855657+0000","last_fullsized":"2024-03-23T15:43:19.855657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:48.607487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855290+0000","last_change":"2024-03-23T15:41:18.651777+0000","last_active":"2024-03-23T15:43:19.855290+0000","last_peered":"2024-03-23T15:43:19.855290+0000","last_clean":"2024-03-23T15:43:19.855290+0000","last_became_active":"2024-03-23T15:41:18.651457+0000","last_became_peered":"2024-03-23T15:41:18.651457+0000","last_unstale":"2024-03-23T15:43:19.855290+0000","last_undegraded":"2024-03-23T15:43:19.855290+0000","last_fullsized":"2024-03-23T15:43:19.855290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:25:28.831493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829549+0000","last_change":"2024-03-23T15:41:17.656357+0000","last_active":"2024-03-23T15:43:19.829549+0000","last_peered":"2024-03-23T15:43:19.829549+0000","last_clean":"2024-03-23T15:43:19.829549+0000","last_became_active":"2024-03-23T15:41:17.655977+0000","last_became_peered":"2024-03-23T15:41:17.655977+0000","last_unstale":"2024-03-23T15:43:19.829549+0000","last_undegraded":"2024-03-23T15:43:19.829549+0000","last_fullsized":"2024-03-23T15:43:19.829549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:07:45.248954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855083+0000","last_change":"2024-03-23T15:42:16.626757+0000","last_active":"2024-03-23T15:43:19.855083+0000","last_peered":"2024-03-23T15:43:19.855083+0000","last_clean":"2024-03-23T15:43:19.855083+0000","last_became_active":"2024-03-23T15:42:16.626409+0000","last_became_peered":"2024-03-23T15:42:16.626409+0000","last_unstale":"2024-03-23T15:43:19.855083+0000","last_undegraded":"2024-03-23T15:43:19.855083+0000","last_fullsized":"2024-03-23T15:43:19.855083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:46.969160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":301,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828033+0000","last_change":"2024-03-23T15:41:17.659838+0000","last_active":"2024-03-23T15:43:19.828033+0000","last_peered":"2024-03-23T15:43:19.828033+0000","last_clean":"2024-03-23T15:43:19.828033+0000","last_became_active":"2024-03-23T15:41:17.659326+0000","last_became_peered":"2024-03-23T15:41:17.659326+0000","last_unstale":"2024-03-23T15:43:19.828033+0000","last_undegraded":"2024-03-23T15:43:19.828033+0000","last_fullsized":"2024-03-23T15:43:19.828033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:07.651340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":183,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855649+0000","last_change":"2024-03-23T15:41:18.656629+0000","last_active":"2024-03-23T15:43:19.855649+0000","last_peered":"2024-03-23T15:43:19.855649+0000","last_clean":"2024-03-23T15:43:19.855649+0000","last_became_active":"2024-03-23T15:41:18.656196+0000","last_became_peered":"2024-03-23T15:41:18.656196+0000","last_unstale":"2024-03-23T15:43:19.855649+0000","last_undegraded":"2024-03-23T15:43:19.855649+0000","last_fullsized":"2024-03-23T15:43:19.855649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:30.013411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855049+0000","last_change":"2024-03-23T15:41:17.663999+0000","last_active":"2024-03-23T15:43:19.855049+0000","last_peered":"2024-03-23T15:43:19.855049+0000","last_clean":"2024-03-23T15:43:19.855049+0000","last_became_active":"2024-03-23T15:41:17.663039+0000","last_became_peered":"2024-03-23T15:41:17.663039+0000","last_unstale":"2024-03-23T15:43:19.855049+0000","last_undegraded":"2024-03-23T15:43:19.855049+0000","last_fullsized":"2024-03-23T15:43:19.855049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:00.271821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828098+0000","last_change":"2024-03-23T15:41:18.646822+0000","last_active":"2024-03-23T15:43:19.828098+0000","last_peered":"2024-03-23T15:43:19.828098+0000","last_clean":"2024-03-23T15:43:19.828098+0000","last_became_active":"2024-03-23T15:41:18.646476+0000","last_became_peered":"2024-03-23T15:41:18.646476+0000","last_unstale":"2024-03-23T15:43:19.828098+0000","last_undegraded":"2024-03-23T15:43:19.828098+0000","last_fullsized":"2024-03-23T15:43:19.828098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:18.352065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829326+0000","last_change":"2024-03-23T15:41:17.663507+0000","last_active":"2024-03-23T15:43:19.829326+0000","last_peered":"2024-03-23T15:43:19.829326+0000","last_clean":"2024-03-23T15:43:19.829326+0000","last_became_active":"2024-03-23T15:41:17.663122+0000","last_became_peered":"2024-03-23T15:41:17.663122+0000","last_unstale":"2024-03-23T15:43:19.829326+0000","last_undegraded":"2024-03-23T15:43:19.829326+0000","last_fullsized":"2024-03-23T15:43:19.829326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:32.202576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855780+0000","last_change":"2024-03-23T15:41:18.650110+0000","last_active":"2024-03-23T15:43:19.855780+0000","last_peered":"2024-03-23T15:43:19.855780+0000","last_clean":"2024-03-23T15:43:19.855780+0000","last_became_active":"2024-03-23T15:41:18.649737+0000","last_became_peered":"2024-03-23T15:41:18.649737+0000","last_unstale":"2024-03-23T15:43:19.855780+0000","last_undegraded":"2024-03-23T15:43:19.855780+0000","last_fullsized":"2024-03-23T15:43:19.855780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:31.610961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830442+0000","last_change":"2024-03-23T15:41:17.653930+0000","last_active":"2024-03-23T15:43:19.830442+0000","last_peered":"2024-03-23T15:43:19.830442+0000","last_clean":"2024-03-23T15:43:19.830442+0000","last_became_active":"2024-03-23T15:41:17.652677+0000","last_became_peered":"2024-03-23T15:41:17.652677+0000","last_unstale":"2024-03-23T15:43:19.830442+0000","last_undegraded":"2024-03-23T15:43:19.830442+0000","last_fullsized":"2024-03-23T15:43:19.830442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:47.167052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855179+0000","last_change":"2024-03-23T15:42:16.843231+0000","last_active":"2024-03-23T15:43:19.855179+0000","last_peered":"2024-03-23T15:43:19.855179+0000","last_clean":"2024-03-23T15:43:19.855179+0000","last_became_active":"2024-03-23T15:42:16.842785+0000","last_became_peered":"2024-03-23T15:42:16.842785+0000","last_unstale":"2024-03-23T15:43:19.855179+0000","last_undegraded":"2024-03-23T15:43:19.855179+0000","last_fullsized":"2024-03-23T15:43:19.855179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:24:07.377828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854899+0000","last_change":"2024-03-23T15:41:17.662091+0000","last_active":"2024-03-23T15:43:19.854899+0000","last_peered":"2024-03-23T15:43:19.854899+0000","last_clean":"2024-03-23T15:43:19.854899+0000","last_became_active":"2024-03-23T15:41:17.661747+0000","last_became_peered":"2024-03-23T15:41:17.661747+0000","last_unstale":"2024-03-23T15:43:19.854899+0000","last_undegraded":"2024-03-23T15:43:19.854899+0000","last_fullsized":"2024-03-23T15:43:19.854899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:08.661262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829293+0000","last_change":"2024-03-23T15:42:16.627776+0000","last_active":"2024-03-23T15:43:19.829293+0000","last_peered":"2024-03-23T15:43:19.829293+0000","last_clean":"2024-03-23T15:43:19.829293+0000","last_became_active":"2024-03-23T15:42:16.627331+0000","last_became_peered":"2024-03-23T15:42:16.627331+0000","last_unstale":"2024-03-23T15:43:19.829293+0000","last_undegraded":"2024-03-23T15:43:19.829293+0000","last_fullsized":"2024-03-23T15:43:19.829293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:02:57.288827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855572+0000","last_change":"2024-03-23T15:41:17.660898+0000","last_active":"2024-03-23T15:43:19.855572+0000","last_peered":"2024-03-23T15:43:19.855572+0000","last_clean":"2024-03-23T15:43:19.855572+0000","last_became_active":"2024-03-23T15:41:17.660515+0000","last_became_peered":"2024-03-23T15:41:17.660515+0000","last_unstale":"2024-03-23T15:43:19.855572+0000","last_undegraded":"2024-03-23T15:43:19.855572+0000","last_fullsized":"2024-03-23T15:43:19.855572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:58.911321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795271+0000","last_change":"2024-03-23T15:41:18.653582+0000","last_active":"2024-03-23T15:43:19.795271+0000","last_peered":"2024-03-23T15:43:19.795271+0000","last_clean":"2024-03-23T15:43:19.795271+0000","last_became_active":"2024-03-23T15:41:18.653184+0000","last_became_peered":"2024-03-23T15:41:18.653184+0000","last_unstale":"2024-03-23T15:43:19.795271+0000","last_undegraded":"2024-03-23T15:43:19.795271+0000","last_fullsized":"2024-03-23T15:43:19.795271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:59:16.726981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793476+0000","last_change":"2024-03-23T15:41:17.651886+0000","last_active":"2024-03-23T15:43:19.793476+0000","last_peered":"2024-03-23T15:43:19.793476+0000","last_clean":"2024-03-23T15:43:19.793476+0000","last_became_active":"2024-03-23T15:41:17.651472+0000","last_became_peered":"2024-03-23T15:41:17.651472+0000","last_unstale":"2024-03-23T15:43:19.793476+0000","last_undegraded":"2024-03-23T15:43:19.793476+0000","last_fullsized":"2024-03-23T15:43:19.793476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:37.098333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795196+0000","last_change":"2024-03-23T15:41:18.644205+0000","last_active":"2024-03-23T15:43:19.795196+0000","last_peered":"2024-03-23T15:43:19.795196+0000","last_clean":"2024-03-23T15:43:19.795196+0000","last_became_active":"2024-03-23T15:41:18.643850+0000","last_became_peered":"2024-03-23T15:41:18.643850+0000","last_unstale":"2024-03-23T15:43:19.795196+0000","last_undegraded":"2024-03-23T15:43:19.795196+0000","last_fullsized":"2024-03-23T15:43:19.795196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:48:10.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855322+0000","last_change":"2024-03-23T15:41:17.661051+0000","last_active":"2024-03-23T15:43:19.855322+0000","last_peered":"2024-03-23T15:43:19.855322+0000","last_clean":"2024-03-23T15:43:19.855322+0000","last_became_active":"2024-03-23T15:41:17.659673+0000","last_became_peered":"2024-03-23T15:41:17.659673+0000","last_unstale":"2024-03-23T15:43:19.855322+0000","last_undegraded":"2024-03-23T15:43:19.855322+0000","last_fullsized":"2024-03-23T15:43:19.855322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.028104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795266+0000","last_change":"2024-03-23T15:42:16.843248+0000","last_active":"2024-03-23T15:43:19.795266+0000","last_peered":"2024-03-23T15:43:19.795266+0000","last_clean":"2024-03-23T15:43:19.795266+0000","last_became_active":"2024-03-23T15:42:16.842800+0000","last_became_peered":"2024-03-23T15:42:16.842800+0000","last_unstale":"2024-03-23T15:43:19.795266+0000","last_undegraded":"2024-03-23T15:43:19.795266+0000","last_fullsized":"2024-03-23T15:43:19.795266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829351+0000","last_change":"2024-03-23T15:41:18.653560+0000","last_active":"2024-03-23T15:43:19.829351+0000","last_peered":"2024-03-23T15:43:19.829351+0000","last_clean":"2024-03-23T15:43:19.829351+0000","last_became_active":"2024-03-23T15:41:18.653249+0000","last_became_peered":"2024-03-23T15:41:18.653249+0000","last_unstale":"2024-03-23T15:43:19.829351+0000","last_undegraded":"2024-03-23T15:43:19.829351+0000","last_fullsized":"2024-03-23T15:43:19.829351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:22.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828395+0000","last_change":"2024-03-23T15:41:17.656318+0000","last_active":"2024-03-23T15:43:19.828395+0000","last_peered":"2024-03-23T15:43:19.828395+0000","last_clean":"2024-03-23T15:43:19.828395+0000","last_became_active":"2024-03-23T15:41:17.655958+0000","last_became_peered":"2024-03-23T15:41:17.655958+0000","last_unstale":"2024-03-23T15:43:19.828395+0000","last_undegraded":"2024-03-23T15:43:19.828395+0000","last_fullsized":"2024-03-23T15:43:19.828395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:12.333504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903433+0000","last_change":"2024-03-23T15:41:18.657616+0000","last_active":"2024-03-23T15:43:19.903433+0000","last_peered":"2024-03-23T15:43:19.903433+0000","last_clean":"2024-03-23T15:43:19.903433+0000","last_became_active":"2024-03-23T15:41:18.657271+0000","last_became_peered":"2024-03-23T15:41:18.657271+0000","last_unstale":"2024-03-23T15:43:19.903433+0000","last_undegraded":"2024-03-23T15:43:19.903433+0000","last_fullsized":"2024-03-23T15:43:19.903433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:02.241087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830384+0000","last_change":"2024-03-23T15:41:17.654358+0000","last_active":"2024-03-23T15:43:19.830384+0000","last_peered":"2024-03-23T15:43:19.830384+0000","last_clean":"2024-03-23T15:43:19.830384+0000","last_became_active":"2024-03-23T15:41:17.653558+0000","last_became_peered":"2024-03-23T15:41:17.653558+0000","last_unstale":"2024-03-23T15:43:19.830384+0000","last_undegraded":"2024-03-23T15:43:19.830384+0000","last_fullsized":"2024-03-23T15:43:19.830384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:32.377883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855332+0000","last_change":"2024-03-23T15:40:05.871801+0000","last_active":"2024-03-23T15:43:19.855332+0000","last_peered":"2024-03-23T15:43:19.855332+0000","last_clean":"2024-03-23T15:43:19.855332+0000","last_became_active":"2024-03-23T15:40:05.871417+0000","last_became_peered":"2024-03-23T15:40:05.871417+0000","last_unstale":"2024-03-23T15:43:19.855332+0000","last_undegraded":"2024-03-23T15:43:19.855332+0000","last_fullsized":"2024-03-23T15:43:19.855332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_clean_scrub_stamp":"2024-03-23T15:36:39.745753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:55.235176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.827993+0000","last_change":"2024-03-23T15:41:18.652251+0000","last_active":"2024-03-23T15:43:19.827993+0000","last_peered":"2024-03-23T15:43:19.827993+0000","last_clean":"2024-03-23T15:43:19.827993+0000","last_became_active":"2024-03-23T15:41:18.651876+0000","last_became_peered":"2024-03-23T15:41:18.651876+0000","last_unstale":"2024-03-23T15:43:19.827993+0000","last_undegraded":"2024-03-23T15:43:19.827993+0000","last_fullsized":"2024-03-23T15:43:19.827993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:45.235243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855564+0000","last_change":"2024-03-23T15:42:16.692783+0000","last_active":"2024-03-23T15:43:19.855564+0000","last_peered":"2024-03-23T15:43:19.855564+0000","last_clean":"2024-03-23T15:43:19.855564+0000","last_became_active":"2024-03-23T15:42:16.690169+0000","last_became_peered":"2024-03-23T15:42:16.690169+0000","last_unstale":"2024-03-23T15:43:19.855564+0000","last_undegraded":"2024-03-23T15:43:19.855564+0000","last_fullsized":"2024-03-23T15:43:19.855564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:20.679040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855712+0000","last_change":"2024-03-23T15:41:17.660703+0000","last_active":"2024-03-23T15:43:19.855712+0000","last_peered":"2024-03-23T15:43:19.855712+0000","last_clean":"2024-03-23T15:43:19.855712+0000","last_became_active":"2024-03-23T15:41:17.660343+0000","last_became_peered":"2024-03-23T15:41:17.660343+0000","last_unstale":"2024-03-23T15:43:19.855712+0000","last_undegraded":"2024-03-23T15:43:19.855712+0000","last_fullsized":"2024-03-23T15:43:19.855712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:14:03.063397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828358+0000","last_change":"2024-03-23T15:41:18.644624+0000","last_active":"2024-03-23T15:43:19.828358+0000","last_peered":"2024-03-23T15:43:19.828358+0000","last_clean":"2024-03-23T15:43:19.828358+0000","last_became_active":"2024-03-23T15:41:18.644275+0000","last_became_peered":"2024-03-23T15:41:18.644275+0000","last_unstale":"2024-03-23T15:43:19.828358+0000","last_undegraded":"2024-03-23T15:43:19.828358+0000","last_fullsized":"2024-03-23T15:43:19.828358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:25.901935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'45","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.999901+0000","last_change":"2024-03-23T15:41:17.650068+0000","last_active":"2024-03-23T15:43:19.999901+0000","last_peered":"2024-03-23T15:43:19.999901+0000","last_clean":"2024-03-23T15:43:19.999901+0000","last_became_active":"2024-03-23T15:41:17.649331+0000","last_became_peered":"2024-03-23T15:41:17.649331+0000","last_unstale":"2024-03-23T15:43:19.999901+0000","last_undegraded":"2024-03-23T15:43:19.999901+0000","last_fullsized":"2024-03-23T15:43:19.999901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:47.578645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903378+0000","last_change":"2024-03-23T15:41:18.656413+0000","last_active":"2024-03-23T15:43:19.903378+0000","last_peered":"2024-03-23T15:43:19.903378+0000","last_clean":"2024-03-23T15:43:19.903378+0000","last_became_active":"2024-03-23T15:41:18.656043+0000","last_became_peered":"2024-03-23T15:41:18.656043+0000","last_unstale":"2024-03-23T15:43:19.903378+0000","last_undegraded":"2024-03-23T15:43:19.903378+0000","last_fullsized":"2024-03-23T15:43:19.903378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:49:25.453420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830801+0000","last_change":"2024-03-23T15:41:17.654492+0000","last_active":"2024-03-23T15:43:19.830801+0000","last_peered":"2024-03-23T15:43:19.830801+0000","last_clean":"2024-03-23T15:43:19.830801+0000","last_became_active":"2024-03-23T15:41:17.654132+0000","last_became_peered":"2024-03-23T15:41:17.654132+0000","last_unstale":"2024-03-23T15:43:19.830801+0000","last_undegraded":"2024-03-23T15:43:19.830801+0000","last_fullsized":"2024-03-23T15:43:19.830801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:39:58.210703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855201+0000","last_change":"2024-03-23T15:41:18.652659+0000","last_active":"2024-03-23T15:43:19.855201+0000","last_peered":"2024-03-23T15:43:19.855201+0000","last_clean":"2024-03-23T15:43:19.855201+0000","last_became_active":"2024-03-23T15:41:18.652246+0000","last_became_peered":"2024-03-23T15:41:18.652246+0000","last_unstale":"2024-03-23T15:43:19.855201+0000","last_undegraded":"2024-03-23T15:43:19.855201+0000","last_fullsized":"2024-03-23T15:43:19.855201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:50.868694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854944+0000","last_change":"2024-03-23T15:42:16.632166+0000","last_active":"2024-03-23T15:43:19.854944+0000","last_peered":"2024-03-23T15:43:19.854944+0000","last_clean":"2024-03-23T15:43:19.854944+0000","last_became_active":"2024-03-23T15:42:16.626587+0000","last_became_peered":"2024-03-23T15:42:16.626587+0000","last_unstale":"2024-03-23T15:43:19.854944+0000","last_undegraded":"2024-03-23T15:43:19.854944+0000","last_fullsized":"2024-03-23T15:43:19.854944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.514597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855068+0000","last_change":"2024-03-23T15:41:18.654049+0000","last_active":"2024-03-23T15:43:19.855068+0000","last_peered":"2024-03-23T15:43:19.855068+0000","last_clean":"2024-03-23T15:43:19.855068+0000","last_became_active":"2024-03-23T15:41:18.653765+0000","last_became_peered":"2024-03-23T15:41:18.653765+0000","last_unstale":"2024-03-23T15:43:19.855068+0000","last_undegraded":"2024-03-23T15:43:19.855068+0000","last_fullsized":"2024-03-23T15:43:19.855068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:12:11.467744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902818+0000","last_change":"2024-03-23T15:41:17.656679+0000","last_active":"2024-03-23T15:43:19.902818+0000","last_peered":"2024-03-23T15:43:19.902818+0000","last_clean":"2024-03-23T15:43:19.902818+0000","last_became_active":"2024-03-23T15:41:17.656368+0000","last_became_peered":"2024-03-23T15:41:17.656368+0000","last_unstale":"2024-03-23T15:43:19.902818+0000","last_undegraded":"2024-03-23T15:43:19.902818+0000","last_fullsized":"2024-03-23T15:43:19.902818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:14.176288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794941+0000","last_change":"2024-03-23T15:41:18.653637+0000","last_active":"2024-03-23T15:43:19.794941+0000","last_peered":"2024-03-23T15:43:19.794941+0000","last_clean":"2024-03-23T15:43:19.794941+0000","last_became_active":"2024-03-23T15:41:18.652864+0000","last_became_peered":"2024-03-23T15:41:18.652864+0000","last_unstale":"2024-03-23T15:43:19.794941+0000","last_undegraded":"2024-03-23T15:43:19.794941+0000","last_fullsized":"2024-03-23T15:43:19.794941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:32.104901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855151+0000","last_change":"2024-03-23T15:41:17.664417+0000","last_active":"2024-03-23T15:43:19.855151+0000","last_peered":"2024-03-23T15:43:19.855151+0000","last_clean":"2024-03-23T15:43:19.855151+0000","last_became_active":"2024-03-23T15:41:17.663454+0000","last_became_peered":"2024-03-23T15:41:17.663454+0000","last_unstale":"2024-03-23T15:43:19.855151+0000","last_undegraded":"2024-03-23T15:43:19.855151+0000","last_fullsized":"2024-03-23T15:43:19.855151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:14.947767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903278+0000","last_change":"2024-03-23T15:41:18.647857+0000","last_active":"2024-03-23T15:43:19.903278+0000","last_peered":"2024-03-23T15:43:19.903278+0000","last_clean":"2024-03-23T15:43:19.903278+0000","last_became_active":"2024-03-23T15:41:18.646924+0000","last_became_peered":"2024-03-23T15:41:18.646924+0000","last_unstale":"2024-03-23T15:43:19.903278+0000","last_undegraded":"2024-03-23T15:43:19.903278+0000","last_fullsized":"2024-03-23T15:43:19.903278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:19.494901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855517+0000","last_change":"2024-03-23T15:41:17.661995+0000","last_active":"2024-03-23T15:43:19.855517+0000","last_peered":"2024-03-23T15:43:19.855517+0000","last_clean":"2024-03-23T15:43:19.855517+0000","last_became_active":"2024-03-23T15:41:17.661632+0000","last_became_peered":"2024-03-23T15:41:17.661632+0000","last_unstale":"2024-03-23T15:43:19.855517+0000","last_undegraded":"2024-03-23T15:43:19.855517+0000","last_fullsized":"2024-03-23T15:43:19.855517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:12.540038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855504+0000","last_change":"2024-03-23T15:41:18.651258+0000","last_active":"2024-03-23T15:43:19.855504+0000","last_peered":"2024-03-23T15:43:19.855504+0000","last_clean":"2024-03-23T15:43:19.855504+0000","last_became_active":"2024-03-23T15:41:18.650910+0000","last_became_peered":"2024-03-23T15:41:18.650910+0000","last_unstale":"2024-03-23T15:43:19.855504+0000","last_undegraded":"2024-03-23T15:43:19.855504+0000","last_fullsized":"2024-03-23T15:43:19.855504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:06:50.139644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855304+0000","last_change":"2024-03-23T15:41:17.654232+0000","last_active":"2024-03-23T15:43:19.855304+0000","last_peered":"2024-03-23T15:43:19.855304+0000","last_clean":"2024-03-23T15:43:19.855304+0000","last_became_active":"2024-03-23T15:41:17.653832+0000","last_became_peered":"2024-03-23T15:41:17.653832+0000","last_unstale":"2024-03-23T15:43:19.855304+0000","last_undegraded":"2024-03-23T15:43:19.855304+0000","last_fullsized":"2024-03-23T15:43:19.855304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:05.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855755+0000","last_change":"2024-03-23T15:41:18.664444+0000","last_active":"2024-03-23T15:43:19.855755+0000","last_peered":"2024-03-23T15:43:19.855755+0000","last_clean":"2024-03-23T15:43:19.855755+0000","last_became_active":"2024-03-23T15:41:18.664127+0000","last_became_peered":"2024-03-23T15:41:18.664127+0000","last_unstale":"2024-03-23T15:43:19.855755+0000","last_undegraded":"2024-03-23T15:43:19.855755+0000","last_fullsized":"2024-03-23T15:43:19.855755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:25.942230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.995159+0000","last_change":"2024-03-23T15:41:17.664834+0000","last_active":"2024-03-23T15:43:19.995159+0000","last_peered":"2024-03-23T15:43:19.995159+0000","last_clean":"2024-03-23T15:43:19.995159+0000","last_became_active":"2024-03-23T15:41:17.664286+0000","last_became_peered":"2024-03-23T15:41:17.664286+0000","last_unstale":"2024-03-23T15:43:19.995159+0000","last_undegraded":"2024-03-23T15:43:19.995159+0000","last_fullsized":"2024-03-23T15:43:19.995159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:50.595420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855495+0000","last_change":"2024-03-23T15:41:18.646341+0000","last_active":"2024-03-23T15:43:19.855495+0000","last_peered":"2024-03-23T15:43:19.855495+0000","last_clean":"2024-03-23T15:43:19.855495+0000","last_became_active":"2024-03-23T15:41:18.645995+0000","last_became_peered":"2024-03-23T15:41:18.645995+0000","last_unstale":"2024-03-23T15:43:19.855495+0000","last_undegraded":"2024-03-23T15:43:19.855495+0000","last_fullsized":"2024-03-23T15:43:19.855495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:56.408957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828254+0000","last_change":"2024-03-23T15:42:16.694643+0000","last_active":"2024-03-23T15:43:19.828254+0000","last_peered":"2024-03-23T15:43:19.828254+0000","last_clean":"2024-03-23T15:43:19.828254+0000","last_became_active":"2024-03-23T15:42:16.690063+0000","last_became_peered":"2024-03-23T15:42:16.690063+0000","last_unstale":"2024-03-23T15:43:19.828254+0000","last_undegraded":"2024-03-23T15:43:19.828254+0000","last_fullsized":"2024-03-23T15:43:19.828254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:54.873573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855942+0000","last_change":"2024-03-23T15:41:18.655250+0000","last_active":"2024-03-23T15:43:19.855942+0000","last_peered":"2024-03-23T15:43:19.855942+0000","last_clean":"2024-03-23T15:43:19.855942+0000","last_became_active":"2024-03-23T15:41:18.654888+0000","last_became_peered":"2024-03-23T15:41:18.654888+0000","last_unstale":"2024-03-23T15:43:19.855942+0000","last_undegraded":"2024-03-23T15:43:19.855942+0000","last_fullsized":"2024-03-23T15:43:19.855942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:36.730600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795028+0000","last_change":"2024-03-23T15:41:18.654309+0000","last_active":"2024-03-23T15:43:19.795028+0000","last_peered":"2024-03-23T15:43:19.795028+0000","last_clean":"2024-03-23T15:43:19.795028+0000","last_became_active":"2024-03-23T15:41:18.653319+0000","last_became_peered":"2024-03-23T15:41:18.653319+0000","last_unstale":"2024-03-23T15:43:19.795028+0000","last_undegraded":"2024-03-23T15:43:19.795028+0000","last_fullsized":"2024-03-23T15:43:19.795028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:02:05.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855072+0000","last_change":"2024-03-23T15:41:18.655179+0000","last_active":"2024-03-23T15:43:19.855072+0000","last_peered":"2024-03-23T15:43:19.855072+0000","last_clean":"2024-03-23T15:43:19.855072+0000","last_became_active":"2024-03-23T15:41:18.654830+0000","last_became_peered":"2024-03-23T15:41:18.654830+0000","last_unstale":"2024-03-23T15:43:19.855072+0000","last_undegraded":"2024-03-23T15:43:19.855072+0000","last_fullsized":"2024-03-23T15:43:19.855072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:29.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854909+0000","last_change":"2024-03-23T15:41:18.650734+0000","last_active":"2024-03-23T15:43:19.854909+0000","last_peered":"2024-03-23T15:43:19.854909+0000","last_clean":"2024-03-23T15:43:19.854909+0000","last_became_active":"2024-03-23T15:41:18.649993+0000","last_became_peered":"2024-03-23T15:41:18.649993+0000","last_unstale":"2024-03-23T15:43:19.854909+0000","last_undegraded":"2024-03-23T15:43:19.854909+0000","last_fullsized":"2024-03-23T15:43:19.854909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:06.463525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854958+0000","last_change":"2024-03-23T15:41:18.655741+0000","last_active":"2024-03-23T15:43:19.854958+0000","last_peered":"2024-03-23T15:43:19.854958+0000","last_clean":"2024-03-23T15:43:19.854958+0000","last_became_active":"2024-03-23T15:41:18.653387+0000","last_became_peered":"2024-03-23T15:41:18.653387+0000","last_unstale":"2024-03-23T15:43:19.854958+0000","last_undegraded":"2024-03-23T15:43:19.854958+0000","last_fullsized":"2024-03-23T15:43:19.854958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:20:26.758549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828382+0000","last_change":"2024-03-23T15:41:18.654739+0000","last_active":"2024-03-23T15:43:19.828382+0000","last_peered":"2024-03-23T15:43:19.828382+0000","last_clean":"2024-03-23T15:43:19.828382+0000","last_became_active":"2024-03-23T15:41:18.654151+0000","last_became_peered":"2024-03-23T15:41:18.654151+0000","last_unstale":"2024-03-23T15:43:19.828382+0000","last_undegraded":"2024-03-23T15:43:19.828382+0000","last_fullsized":"2024-03-23T15:43:19.828382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:01.358959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795157+0000","last_change":"2024-03-23T15:41:18.654081+0000","last_active":"2024-03-23T15:43:19.795157+0000","last_peered":"2024-03-23T15:43:19.795157+0000","last_clean":"2024-03-23T15:43:19.795157+0000","last_became_active":"2024-03-23T15:41:18.653696+0000","last_became_peered":"2024-03-23T15:41:18.653696+0000","last_unstale":"2024-03-23T15:43:19.795157+0000","last_undegraded":"2024-03-23T15:43:19.795157+0000","last_fullsized":"2024-03-23T15:43:19.795157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:23:19.741645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855549+0000","last_change":"2024-03-23T15:41:18.652356+0000","last_active":"2024-03-23T15:43:19.855549+0000","last_peered":"2024-03-23T15:43:19.855549+0000","last_clean":"2024-03-23T15:43:19.855549+0000","last_became_active":"2024-03-23T15:41:18.651976+0000","last_became_peered":"2024-03-23T15:41:18.651976+0000","last_unstale":"2024-03-23T15:43:19.855549+0000","last_undegraded":"2024-03-23T15:43:19.855549+0000","last_fullsized":"2024-03-23T15:43:19.855549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:38.972186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903249+0000","last_change":"2024-03-23T15:41:18.653775+0000","last_active":"2024-03-23T15:43:19.903249+0000","last_peered":"2024-03-23T15:43:19.903249+0000","last_clean":"2024-03-23T15:43:19.903249+0000","last_became_active":"2024-03-23T15:41:18.653404+0000","last_became_peered":"2024-03-23T15:41:18.653404+0000","last_unstale":"2024-03-23T15:43:19.903249+0000","last_undegraded":"2024-03-23T15:43:19.903249+0000","last_fullsized":"2024-03-23T15:43:19.903249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:28:57.534731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854866+0000","last_change":"2024-03-23T15:42:16.843115+0000","last_active":"2024-03-23T15:43:19.854866+0000","last_peered":"2024-03-23T15:43:19.854866+0000","last_clean":"2024-03-23T15:43:19.854866+0000","last_became_active":"2024-03-23T15:42:16.842743+0000","last_became_peered":"2024-03-23T15:42:16.842743+0000","last_unstale":"2024-03-23T15:43:19.854866+0000","last_undegraded":"2024-03-23T15:43:19.854866+0000","last_fullsized":"2024-03-23T15:43:19.854866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:36:11.377398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902672+0000","last_change":"2024-03-23T15:41:18.652487+0000","last_active":"2024-03-23T15:43:19.902672+0000","last_peered":"2024-03-23T15:43:19.902672+0000","last_clean":"2024-03-23T15:43:19.902672+0000","last_became_active":"2024-03-23T15:41:18.652151+0000","last_became_peered":"2024-03-23T15:41:18.652151+0000","last_unstale":"2024-03-23T15:43:19.902672+0000","last_undegraded":"2024-03-23T15:43:19.902672+0000","last_fullsized":"2024-03-23T15:43:19.902672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:39.133585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855151+0000","last_change":"2024-03-23T15:41:18.657265+0000","last_active":"2024-03-23T15:43:19.855151+0000","last_peered":"2024-03-23T15:43:19.855151+0000","last_clean":"2024-03-23T15:43:19.855151+0000","last_became_active":"2024-03-23T15:41:18.656970+0000","last_became_peered":"2024-03-23T15:41:18.656970+0000","last_unstale":"2024-03-23T15:43:19.855151+0000","last_undegraded":"2024-03-23T15:43:19.855151+0000","last_fullsized":"2024-03-23T15:43:19.855151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:22.808034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828277+0000","last_change":"2024-03-23T15:41:18.655254+0000","last_active":"2024-03-23T15:43:19.828277+0000","last_peered":"2024-03-23T15:43:19.828277+0000","last_clean":"2024-03-23T15:43:19.828277+0000","last_became_active":"2024-03-23T15:41:18.654879+0000","last_became_peered":"2024-03-23T15:41:18.654879+0000","last_unstale":"2024-03-23T15:43:19.828277+0000","last_undegraded":"2024-03-23T15:43:19.828277+0000","last_fullsized":"2024-03-23T15:43:19.828277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:36:18.923723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795101+0000","last_change":"2024-03-23T15:41:18.658652+0000","last_active":"2024-03-23T15:43:19.795101+0000","last_peered":"2024-03-23T15:43:19.795101+0000","last_clean":"2024-03-23T15:43:19.795101+0000","last_became_active":"2024-03-23T15:41:18.658261+0000","last_became_peered":"2024-03-23T15:41:18.658261+0000","last_unstale":"2024-03-23T15:43:19.795101+0000","last_undegraded":"2024-03-23T15:43:19.795101+0000","last_fullsized":"2024-03-23T15:43:19.795101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:45.145101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106812,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597498,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":830,"num_read_kb":518,"num_write":285,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1792494,"data_compressed":81513,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":465380,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1122304,"data_stored":1250064,"data_compressed":52357,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":858832,"data_compressed":27874,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":858752,"data_compressed":27874,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":1058780,"data_compressed":44332,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":958464,"data_stored":465005,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":532165,"data_compressed":10597,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":466276,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":528283,"data_compressed":10597,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":990132,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":465833,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1189673,"data_compressed":50488,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67367,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70031,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134390,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200936,"data_compressed":8025,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T15:43:24.166 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T15:43:24.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 -- 172.21.15.76:0/1034850371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 msgr2=0x7fd164100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 --2- 172.21.15.76:0/1034850371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164100840 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd160009930 tx=0x7fd16002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 -- 172.21.15.76:0/1034850371 shutdown_connections 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 --2- 172.21.15.76:0/1034850371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd164107c30 0x7fd16410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 --2- 172.21.15.76:0/1034850371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd164100e10 0x7fd164107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 --2- 172.21.15.76:0/1034850371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164100840 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 -- 172.21.15.76:0/1034850371 >> 172.21.15.76:0/1034850371 conn(0x7fd164075930 msgr2=0x7fd164075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 -- 172.21.15.76:0/1034850371 shutdown_connections 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 -- 172.21.15.76:0/1034850371 wait complete. 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 Processor -- start 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.316+0000 7fd16bf25640 1 -- start start 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd16bf25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd16bf25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd164107c30 0x7fd16419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd16bf25640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd16419e820 0x7fd16419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd16bf25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd16419f210 con 0x7fd164100460 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd16bf25640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd16419f380 con 0x7fd164107c30 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd16bf25640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd16419f4f0 con 0x7fd16419e820 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49564/0 (socket says 172.21.15.76:49564) 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 -- 172.21.15.76:0/3584203553 learned_addr learned my addr 172.21.15.76:0/3584203553 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd16a4c2640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd16419e820 0x7fd16419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd1694c0640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd164107c30 0x7fd16419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 -- 172.21.15.76:0/3584203553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd16419e820 msgr2=0x7fd16419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd16419e820 0x7fd16419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 -- 172.21.15.76:0/3584203553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd164107c30 msgr2=0x7fd16419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd164107c30 0x7fd16419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 -- 172.21.15.76:0/3584203553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd160009560 con 0x7fd164100460 2024-03-23T15:43:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.317+0000 7fd169cc1640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164199f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd160009230 tx=0x7fd16002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:24.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.319+0000 7fd15affd640 1 -- 172.21.15.76:0/3584203553 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160004350 con 0x7fd164100460 2024-03-23T15:43:24.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.319+0000 7fd15affd640 1 -- 172.21.15.76:0/3584203553 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1600044b0 con 0x7fd164100460 2024-03-23T15:43:24.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.319+0000 7fd15affd640 1 -- 172.21.15.76:0/3584203553 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160003d00 con 0x7fd164100460 2024-03-23T15:43:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.319+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1641a6ad0 con 0x7fd164100460 2024-03-23T15:43:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.319+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1641a6f50 con 0x7fd164100460 2024-03-23T15:43:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.320+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd164067c70 con 0x7fd164100460 2024-03-23T15:43:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.321+0000 7fd15affd640 1 -- 172.21.15.76:0/3584203553 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7fd160004df0 con 0x7fd164100460 2024-03-23T15:43:24.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.322+0000 7fd15affd640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd1500a2350 0x7fd1500a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.322+0000 7fd1694c0640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd1500a2350 0x7fd1500a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.322+0000 7fd15affd640 1 -- 172.21.15.76:0/3584203553 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd1600f0220 con 0x7fd164100460 2024-03-23T15:43:24.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.322+0000 7fd1694c0640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd1500a2350 0x7fd1500a4810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd1640fecf0 tx=0x7fd154005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:24.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.326+0000 7fd15affd640 1 -- 172.21.15.76:0/3584203553 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd1600b8d70 con 0x7fd164100460 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Unable to set osd_memory_target on smithi192 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/4142234672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:24.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi076 to 176.2M 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Unable to set osd_memory_target on smithi076 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi183 to 174.1M 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Unable to set osd_memory_target on smithi183 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Updating smithi076:/etc/ceph/ceph.conf 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3213391654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.387 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:24 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2788240890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:24.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.508+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd164100e10 con 0x7fd1500a2350 2024-03-23T15:43:24.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.515+0000 7fd15affd640 1 -- 172.21.15.76:0/3584203553 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258021 (secure 0 0 0) 0x7fd164100e10 con 0x7fd1500a2350 2024-03-23T15:43:24.521 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:24.522 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-23T15:43:24.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd1500a2350 msgr2=0x7fd1500a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd1500a2350 0x7fd1500a4810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd1640fecf0 tx=0x7fd154005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 msgr2=0x7fd164199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164199f60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd160009230 tx=0x7fd16002c650 comp rx=0 tx=0).stop 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 shutdown_connections 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd1500a2350 0x7fd1500a4810 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd16419e820 0x7fd16419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd164107c30 0x7fd16419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 --2- 172.21.15.76:0/3584203553 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd164100460 0x7fd164199f60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 >> 172.21.15.76:0/3584203553 conn(0x7fd164075930 msgr2=0x7fd1640ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 shutdown_connections 2024-03-23T15:43:24.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.518+0000 7fd16bf25640 1 -- 172.21.15.76:0/3584203553 wait complete. 2024-03-23T15:43:24.532 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-03-23T15:43:23.805884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188100,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":966,"num_read_kb":637,"num_write":468,"num_write_kb":4375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369360,"kb_used_data":11840,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572464,"statfs":{"total":1151940427776,"available":1151562203136,"internally_reserved":0,"allocated":12124160,"data_stored":9129175,"data_compressed":293061,"data_compressed_allocated":393216,"data_compressed_original":3637248,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.975010"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795300+0000","last_change":"2024-03-23T15:41:18.651731+0000","last_active":"2024-03-23T15:43:19.795300+0000","last_peered":"2024-03-23T15:43:19.795300+0000","last_clean":"2024-03-23T15:43:19.795300+0000","last_became_active":"2024-03-23T15:41:18.651312+0000","last_became_peered":"2024-03-23T15:41:18.651312+0000","last_unstale":"2024-03-23T15:43:19.795300+0000","last_undegraded":"2024-03-23T15:43:19.795300+0000","last_fullsized":"2024-03-23T15:43:19.795300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:39:20.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855216+0000","last_change":"2024-03-23T15:41:18.655736+0000","last_active":"2024-03-23T15:43:19.855216+0000","last_peered":"2024-03-23T15:43:19.855216+0000","last_clean":"2024-03-23T15:43:19.855216+0000","last_became_active":"2024-03-23T15:41:18.655389+0000","last_became_peered":"2024-03-23T15:41:18.655389+0000","last_unstale":"2024-03-23T15:43:19.855216+0000","last_undegraded":"2024-03-23T15:43:19.855216+0000","last_fullsized":"2024-03-23T15:43:19.855216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:35:56.260056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855070+0000","last_change":"2024-03-23T15:41:18.650709+0000","last_active":"2024-03-23T15:43:19.855070+0000","last_peered":"2024-03-23T15:43:19.855070+0000","last_clean":"2024-03-23T15:43:19.855070+0000","last_became_active":"2024-03-23T15:41:18.650316+0000","last_became_peered":"2024-03-23T15:41:18.650316+0000","last_unstale":"2024-03-23T15:43:19.855070+0000","last_undegraded":"2024-03-23T15:43:19.855070+0000","last_fullsized":"2024-03-23T15:43:19.855070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:50:57.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830666+0000","last_change":"2024-03-23T15:41:18.652216+0000","last_active":"2024-03-23T15:43:19.830666+0000","last_peered":"2024-03-23T15:43:19.830666+0000","last_clean":"2024-03-23T15:43:19.830666+0000","last_became_active":"2024-03-23T15:41:18.650525+0000","last_became_peered":"2024-03-23T15:41:18.650525+0000","last_unstale":"2024-03-23T15:43:19.830666+0000","last_undegraded":"2024-03-23T15:43:19.830666+0000","last_fullsized":"2024-03-23T15:43:19.830666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:16.458257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854905+0000","last_change":"2024-03-23T15:41:18.649146+0000","last_active":"2024-03-23T15:43:19.854905+0000","last_peered":"2024-03-23T15:43:19.854905+0000","last_clean":"2024-03-23T15:43:19.854905+0000","last_became_active":"2024-03-23T15:41:18.648728+0000","last_became_peered":"2024-03-23T15:41:18.648728+0000","last_unstale":"2024-03-23T15:43:19.854905+0000","last_undegraded":"2024-03-23T15:43:19.854905+0000","last_fullsized":"2024-03-23T15:43:19.854905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:56.251250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794938+0000","last_change":"2024-03-23T15:41:18.652957+0000","last_active":"2024-03-23T15:43:19.794938+0000","last_peered":"2024-03-23T15:43:19.794938+0000","last_clean":"2024-03-23T15:43:19.794938+0000","last_became_active":"2024-03-23T15:41:18.652578+0000","last_became_peered":"2024-03-23T15:41:18.652578+0000","last_unstale":"2024-03-23T15:43:19.794938+0000","last_undegraded":"2024-03-23T15:43:19.794938+0000","last_fullsized":"2024-03-23T15:43:19.794938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:46:38.961554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855251+0000","last_change":"2024-03-23T15:41:18.646134+0000","last_active":"2024-03-23T15:43:19.855251+0000","last_peered":"2024-03-23T15:43:19.855251+0000","last_clean":"2024-03-23T15:43:19.855251+0000","last_became_active":"2024-03-23T15:41:18.645680+0000","last_became_peered":"2024-03-23T15:41:18.645680+0000","last_unstale":"2024-03-23T15:43:19.855251+0000","last_undegraded":"2024-03-23T15:43:19.855251+0000","last_fullsized":"2024-03-23T15:43:19.855251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:34:31.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902586+0000","last_change":"2024-03-23T15:41:18.646143+0000","last_active":"2024-03-23T15:43:19.902586+0000","last_peered":"2024-03-23T15:43:19.902586+0000","last_clean":"2024-03-23T15:43:19.902586+0000","last_became_active":"2024-03-23T15:41:18.645771+0000","last_became_peered":"2024-03-23T15:41:18.645771+0000","last_unstale":"2024-03-23T15:43:19.902586+0000","last_undegraded":"2024-03-23T15:43:19.902586+0000","last_fullsized":"2024-03-23T15:43:19.902586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:39.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855670+0000","last_change":"2024-03-23T15:41:18.665083+0000","last_active":"2024-03-23T15:43:19.855670+0000","last_peered":"2024-03-23T15:43:19.855670+0000","last_clean":"2024-03-23T15:43:19.855670+0000","last_became_active":"2024-03-23T15:41:18.663143+0000","last_became_peered":"2024-03-23T15:41:18.663143+0000","last_unstale":"2024-03-23T15:43:19.855670+0000","last_undegraded":"2024-03-23T15:43:19.855670+0000","last_fullsized":"2024-03-23T15:43:19.855670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:52:23.421645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795361+0000","last_change":"2024-03-23T15:41:18.650014+0000","last_active":"2024-03-23T15:43:19.795361+0000","last_peered":"2024-03-23T15:43:19.795361+0000","last_clean":"2024-03-23T15:43:19.795361+0000","last_became_active":"2024-03-23T15:41:18.649673+0000","last_became_peered":"2024-03-23T15:41:18.649673+0000","last_unstale":"2024-03-23T15:43:19.795361+0000","last_undegraded":"2024-03-23T15:43:19.795361+0000","last_fullsized":"2024-03-23T15:43:19.795361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:30.387993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855606+0000","last_change":"2024-03-23T15:41:18.665622+0000","last_active":"2024-03-23T15:43:19.855606+0000","last_peered":"2024-03-23T15:43:19.855606+0000","last_clean":"2024-03-23T15:43:19.855606+0000","last_became_active":"2024-03-23T15:41:18.665274+0000","last_became_peered":"2024-03-23T15:41:18.665274+0000","last_unstale":"2024-03-23T15:43:19.855606+0000","last_undegraded":"2024-03-23T15:43:19.855606+0000","last_fullsized":"2024-03-23T15:43:19.855606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:12:18.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830547+0000","last_change":"2024-03-23T15:41:18.650888+0000","last_active":"2024-03-23T15:43:19.830547+0000","last_peered":"2024-03-23T15:43:19.830547+0000","last_clean":"2024-03-23T15:43:19.830547+0000","last_became_active":"2024-03-23T15:41:18.650363+0000","last_became_peered":"2024-03-23T15:41:18.650363+0000","last_unstale":"2024-03-23T15:43:19.830547+0000","last_undegraded":"2024-03-23T15:43:19.830547+0000","last_fullsized":"2024-03-23T15:43:19.830547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:41.289697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828315+0000","last_change":"2024-03-23T15:41:18.634183+0000","last_active":"2024-03-23T15:43:19.828315+0000","last_peered":"2024-03-23T15:43:19.828315+0000","last_clean":"2024-03-23T15:43:19.828315+0000","last_became_active":"2024-03-23T15:41:18.633849+0000","last_became_peered":"2024-03-23T15:41:18.633849+0000","last_unstale":"2024-03-23T15:43:19.828315+0000","last_undegraded":"2024-03-23T15:43:19.828315+0000","last_fullsized":"2024-03-23T15:43:19.828315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:13.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854833+0000","last_change":"2024-03-23T15:41:18.645269+0000","last_active":"2024-03-23T15:43:19.854833+0000","last_peered":"2024-03-23T15:43:19.854833+0000","last_clean":"2024-03-23T15:43:19.854833+0000","last_became_active":"2024-03-23T15:41:18.644940+0000","last_became_peered":"2024-03-23T15:41:18.644940+0000","last_unstale":"2024-03-23T15:43:19.854833+0000","last_undegraded":"2024-03-23T15:43:19.854833+0000","last_fullsized":"2024-03-23T15:43:19.854833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:26.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854878+0000","last_change":"2024-03-23T15:41:18.652789+0000","last_active":"2024-03-23T15:43:19.854878+0000","last_peered":"2024-03-23T15:43:19.854878+0000","last_clean":"2024-03-23T15:43:19.854878+0000","last_became_active":"2024-03-23T15:41:18.651520+0000","last_became_peered":"2024-03-23T15:41:18.651520+0000","last_unstale":"2024-03-23T15:43:19.854878+0000","last_undegraded":"2024-03-23T15:43:19.854878+0000","last_fullsized":"2024-03-23T15:43:19.854878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:00.388217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854859+0000","last_change":"2024-03-23T15:41:18.660398+0000","last_active":"2024-03-23T15:43:19.854859+0000","last_peered":"2024-03-23T15:43:19.854859+0000","last_clean":"2024-03-23T15:43:19.854859+0000","last_became_active":"2024-03-23T15:41:18.660084+0000","last_became_peered":"2024-03-23T15:41:18.660084+0000","last_unstale":"2024-03-23T15:43:19.854859+0000","last_undegraded":"2024-03-23T15:43:19.854859+0000","last_fullsized":"2024-03-23T15:43:19.854859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:04:33.115173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795310+0000","last_change":"2024-03-23T15:41:18.663192+0000","last_active":"2024-03-23T15:43:19.795310+0000","last_peered":"2024-03-23T15:43:19.795310+0000","last_clean":"2024-03-23T15:43:19.795310+0000","last_became_active":"2024-03-23T15:41:18.662844+0000","last_became_peered":"2024-03-23T15:41:18.662844+0000","last_unstale":"2024-03-23T15:43:19.795310+0000","last_undegraded":"2024-03-23T15:43:19.795310+0000","last_fullsized":"2024-03-23T15:43:19.795310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:24.260706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793526+0000","last_change":"2024-03-23T15:41:18.646405+0000","last_active":"2024-03-23T15:43:19.793526+0000","last_peered":"2024-03-23T15:43:19.793526+0000","last_clean":"2024-03-23T15:43:19.793526+0000","last_became_active":"2024-03-23T15:41:18.645516+0000","last_became_peered":"2024-03-23T15:41:18.645516+0000","last_unstale":"2024-03-23T15:43:19.793526+0000","last_undegraded":"2024-03-23T15:43:19.793526+0000","last_fullsized":"2024-03-23T15:43:19.793526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:29:18.588190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855413+0000","last_change":"2024-03-23T15:42:16.843159+0000","last_active":"2024-03-23T15:43:19.855413+0000","last_peered":"2024-03-23T15:43:19.855413+0000","last_clean":"2024-03-23T15:43:19.855413+0000","last_became_active":"2024-03-23T15:42:16.842729+0000","last_became_peered":"2024-03-23T15:42:16.842729+0000","last_unstale":"2024-03-23T15:43:19.855413+0000","last_undegraded":"2024-03-23T15:43:19.855413+0000","last_fullsized":"2024-03-23T15:43:19.855413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:57:18.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793618+0000","last_change":"2024-03-23T15:41:17.655443+0000","last_active":"2024-03-23T15:43:19.793618+0000","last_peered":"2024-03-23T15:43:19.793618+0000","last_clean":"2024-03-23T15:43:19.793618+0000","last_became_active":"2024-03-23T15:41:17.654834+0000","last_became_peered":"2024-03-23T15:41:17.654834+0000","last_unstale":"2024-03-23T15:43:19.793618+0000","last_undegraded":"2024-03-23T15:43:19.793618+0000","last_fullsized":"2024-03-23T15:43:19.793618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:49:43.215644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903720+0000","last_change":"2024-03-23T15:41:18.650241+0000","last_active":"2024-03-23T15:43:19.903720+0000","last_peered":"2024-03-23T15:43:19.903720+0000","last_clean":"2024-03-23T15:43:19.903720+0000","last_became_active":"2024-03-23T15:41:18.649914+0000","last_became_peered":"2024-03-23T15:41:18.649914+0000","last_unstale":"2024-03-23T15:43:19.903720+0000","last_undegraded":"2024-03-23T15:43:19.903720+0000","last_fullsized":"2024-03-23T15:43:19.903720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:35:18.892649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903809+0000","last_change":"2024-03-23T15:41:17.663308+0000","last_active":"2024-03-23T15:43:19.903809+0000","last_peered":"2024-03-23T15:43:19.903809+0000","last_clean":"2024-03-23T15:43:19.903809+0000","last_became_active":"2024-03-23T15:41:17.662958+0000","last_became_peered":"2024-03-23T15:41:17.662958+0000","last_unstale":"2024-03-23T15:43:19.903809+0000","last_undegraded":"2024-03-23T15:43:19.903809+0000","last_fullsized":"2024-03-23T15:43:19.903809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:43:37.512893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855316+0000","last_change":"2024-03-23T15:41:18.651225+0000","last_active":"2024-03-23T15:43:19.855316+0000","last_peered":"2024-03-23T15:43:19.855316+0000","last_clean":"2024-03-23T15:43:19.855316+0000","last_became_active":"2024-03-23T15:41:18.650884+0000","last_became_peered":"2024-03-23T15:41:18.650884+0000","last_unstale":"2024-03-23T15:43:19.855316+0000","last_undegraded":"2024-03-23T15:43:19.855316+0000","last_fullsized":"2024-03-23T15:43:19.855316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:00.445203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855793+0000","last_change":"2024-03-23T15:41:17.662753+0000","last_active":"2024-03-23T15:43:19.855793+0000","last_peered":"2024-03-23T15:43:19.855793+0000","last_clean":"2024-03-23T15:43:19.855793+0000","last_became_active":"2024-03-23T15:41:17.662378+0000","last_became_peered":"2024-03-23T15:41:17.662378+0000","last_unstale":"2024-03-23T15:43:19.855793+0000","last_undegraded":"2024-03-23T15:43:19.855793+0000","last_fullsized":"2024-03-23T15:43:19.855793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:58:55.674739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854935+0000","last_change":"2024-03-23T15:41:18.659171+0000","last_active":"2024-03-23T15:43:19.854935+0000","last_peered":"2024-03-23T15:43:19.854935+0000","last_clean":"2024-03-23T15:43:19.854935+0000","last_became_active":"2024-03-23T15:41:18.658836+0000","last_became_peered":"2024-03-23T15:41:18.658836+0000","last_unstale":"2024-03-23T15:43:19.854935+0000","last_undegraded":"2024-03-23T15:43:19.854935+0000","last_fullsized":"2024-03-23T15:43:19.854935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:15.056528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794998+0000","last_change":"2024-03-23T15:42:16.631598+0000","last_active":"2024-03-23T15:43:19.794998+0000","last_peered":"2024-03-23T15:43:19.794998+0000","last_clean":"2024-03-23T15:43:19.794998+0000","last_became_active":"2024-03-23T15:42:16.626457+0000","last_became_peered":"2024-03-23T15:42:16.626457+0000","last_unstale":"2024-03-23T15:43:19.794998+0000","last_undegraded":"2024-03-23T15:43:19.794998+0000","last_fullsized":"2024-03-23T15:43:19.794998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:29.776204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829447+0000","last_change":"2024-03-23T15:41:18.648500+0000","last_active":"2024-03-23T15:43:19.829447+0000","last_peered":"2024-03-23T15:43:19.829447+0000","last_clean":"2024-03-23T15:43:19.829447+0000","last_became_active":"2024-03-23T15:41:18.648139+0000","last_became_peered":"2024-03-23T15:41:18.648139+0000","last_unstale":"2024-03-23T15:43:19.829447+0000","last_undegraded":"2024-03-23T15:43:19.829447+0000","last_fullsized":"2024-03-23T15:43:19.829447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:38.071057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:20.032863+0000","last_change":"2024-03-23T15:41:17.658471+0000","last_active":"2024-03-23T15:43:20.032863+0000","last_peered":"2024-03-23T15:43:20.032863+0000","last_clean":"2024-03-23T15:43:20.032863+0000","last_became_active":"2024-03-23T15:41:17.658142+0000","last_became_peered":"2024-03-23T15:41:17.658142+0000","last_unstale":"2024-03-23T15:43:20.032863+0000","last_undegraded":"2024-03-23T15:43:20.032863+0000","last_fullsized":"2024-03-23T15:43:20.032863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:23.224790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855346+0000","last_change":"2024-03-23T15:41:18.646046+0000","last_active":"2024-03-23T15:43:19.855346+0000","last_peered":"2024-03-23T15:43:19.855346+0000","last_clean":"2024-03-23T15:43:19.855346+0000","last_became_active":"2024-03-23T15:41:18.645727+0000","last_became_peered":"2024-03-23T15:41:18.645727+0000","last_unstale":"2024-03-23T15:43:19.855346+0000","last_undegraded":"2024-03-23T15:43:19.855346+0000","last_fullsized":"2024-03-23T15:43:19.855346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:55.552079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794899+0000","last_change":"2024-03-23T15:41:17.657285+0000","last_active":"2024-03-23T15:43:19.794899+0000","last_peered":"2024-03-23T15:43:19.794899+0000","last_clean":"2024-03-23T15:43:19.794899+0000","last_became_active":"2024-03-23T15:41:17.656942+0000","last_became_peered":"2024-03-23T15:41:17.656942+0000","last_unstale":"2024-03-23T15:43:19.794899+0000","last_undegraded":"2024-03-23T15:43:19.794899+0000","last_fullsized":"2024-03-23T15:43:19.794899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:18.107467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829486+0000","last_change":"2024-03-23T15:41:18.655175+0000","last_active":"2024-03-23T15:43:19.829486+0000","last_peered":"2024-03-23T15:43:19.829486+0000","last_clean":"2024-03-23T15:43:19.829486+0000","last_became_active":"2024-03-23T15:41:18.654846+0000","last_became_peered":"2024-03-23T15:41:18.654846+0000","last_unstale":"2024-03-23T15:43:19.829486+0000","last_undegraded":"2024-03-23T15:43:19.829486+0000","last_fullsized":"2024-03-23T15:43:19.829486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:43:10.008538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828140+0000","last_change":"2024-03-23T15:41:17.657391+0000","last_active":"2024-03-23T15:43:19.828140+0000","last_peered":"2024-03-23T15:43:19.828140+0000","last_clean":"2024-03-23T15:43:19.828140+0000","last_became_active":"2024-03-23T15:41:17.657081+0000","last_became_peered":"2024-03-23T15:41:17.657081+0000","last_unstale":"2024-03-23T15:43:19.828140+0000","last_undegraded":"2024-03-23T15:43:19.828140+0000","last_fullsized":"2024-03-23T15:43:19.828140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:08.744477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902586+0000","last_change":"2024-03-23T15:41:18.656268+0000","last_active":"2024-03-23T15:43:19.902586+0000","last_peered":"2024-03-23T15:43:19.902586+0000","last_clean":"2024-03-23T15:43:19.902586+0000","last_became_active":"2024-03-23T15:41:18.655910+0000","last_became_peered":"2024-03-23T15:41:18.655910+0000","last_unstale":"2024-03-23T15:43:19.902586+0000","last_undegraded":"2024-03-23T15:43:19.902586+0000","last_fullsized":"2024-03-23T15:43:19.902586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:38:57.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829296+0000","last_change":"2024-03-23T15:41:17.656555+0000","last_active":"2024-03-23T15:43:19.829296+0000","last_peered":"2024-03-23T15:43:19.829296+0000","last_clean":"2024-03-23T15:43:19.829296+0000","last_became_active":"2024-03-23T15:41:17.655732+0000","last_became_peered":"2024-03-23T15:41:17.655732+0000","last_unstale":"2024-03-23T15:43:19.829296+0000","last_undegraded":"2024-03-23T15:43:19.829296+0000","last_fullsized":"2024-03-23T15:43:19.829296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:53.839841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793401+0000","last_change":"2024-03-23T15:41:18.647334+0000","last_active":"2024-03-23T15:43:19.793401+0000","last_peered":"2024-03-23T15:43:19.793401+0000","last_clean":"2024-03-23T15:43:19.793401+0000","last_became_active":"2024-03-23T15:41:18.646679+0000","last_became_peered":"2024-03-23T15:41:18.646679+0000","last_unstale":"2024-03-23T15:43:19.793401+0000","last_undegraded":"2024-03-23T15:43:19.793401+0000","last_fullsized":"2024-03-23T15:43:19.793401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.710759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.827515+0000","last_change":"2024-03-23T15:41:17.655400+0000","last_active":"2024-03-23T15:43:19.827515+0000","last_peered":"2024-03-23T15:43:19.827515+0000","last_clean":"2024-03-23T15:43:19.827515+0000","last_became_active":"2024-03-23T15:41:17.655034+0000","last_became_peered":"2024-03-23T15:41:17.655034+0000","last_unstale":"2024-03-23T15:43:19.827515+0000","last_undegraded":"2024-03-23T15:43:19.827515+0000","last_fullsized":"2024-03-23T15:43:19.827515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:30.973048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855488+0000","last_change":"2024-03-23T15:41:18.660248+0000","last_active":"2024-03-23T15:43:19.855488+0000","last_peered":"2024-03-23T15:43:19.855488+0000","last_clean":"2024-03-23T15:43:19.855488+0000","last_became_active":"2024-03-23T15:41:18.659728+0000","last_became_peered":"2024-03-23T15:41:18.659728+0000","last_unstale":"2024-03-23T15:43:19.855488+0000","last_undegraded":"2024-03-23T15:43:19.855488+0000","last_fullsized":"2024-03-23T15:43:19.855488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:03.681499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855657+0000","last_change":"2024-03-23T15:41:17.659784+0000","last_active":"2024-03-23T15:43:19.855657+0000","last_peered":"2024-03-23T15:43:19.855657+0000","last_clean":"2024-03-23T15:43:19.855657+0000","last_became_active":"2024-03-23T15:41:17.659422+0000","last_became_peered":"2024-03-23T15:41:17.659422+0000","last_unstale":"2024-03-23T15:43:19.855657+0000","last_undegraded":"2024-03-23T15:43:19.855657+0000","last_fullsized":"2024-03-23T15:43:19.855657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:48.607487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855290+0000","last_change":"2024-03-23T15:41:18.651777+0000","last_active":"2024-03-23T15:43:19.855290+0000","last_peered":"2024-03-23T15:43:19.855290+0000","last_clean":"2024-03-23T15:43:19.855290+0000","last_became_active":"2024-03-23T15:41:18.651457+0000","last_became_peered":"2024-03-23T15:41:18.651457+0000","last_unstale":"2024-03-23T15:43:19.855290+0000","last_undegraded":"2024-03-23T15:43:19.855290+0000","last_fullsized":"2024-03-23T15:43:19.855290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:25:28.831493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829549+0000","last_change":"2024-03-23T15:41:17.656357+0000","last_active":"2024-03-23T15:43:19.829549+0000","last_peered":"2024-03-23T15:43:19.829549+0000","last_clean":"2024-03-23T15:43:19.829549+0000","last_became_active":"2024-03-23T15:41:17.655977+0000","last_became_peered":"2024-03-23T15:41:17.655977+0000","last_unstale":"2024-03-23T15:43:19.829549+0000","last_undegraded":"2024-03-23T15:43:19.829549+0000","last_fullsized":"2024-03-23T15:43:19.829549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:07:45.248954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855083+0000","last_change":"2024-03-23T15:42:16.626757+0000","last_active":"2024-03-23T15:43:19.855083+0000","last_peered":"2024-03-23T15:43:19.855083+0000","last_clean":"2024-03-23T15:43:19.855083+0000","last_became_active":"2024-03-23T15:42:16.626409+0000","last_became_peered":"2024-03-23T15:42:16.626409+0000","last_unstale":"2024-03-23T15:43:19.855083+0000","last_undegraded":"2024-03-23T15:43:19.855083+0000","last_fullsized":"2024-03-23T15:43:19.855083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:46.969160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":301,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828033+0000","last_change":"2024-03-23T15:41:17.659838+0000","last_active":"2024-03-23T15:43:19.828033+0000","last_peered":"2024-03-23T15:43:19.828033+0000","last_clean":"2024-03-23T15:43:19.828033+0000","last_became_active":"2024-03-23T15:41:17.659326+0000","last_became_peered":"2024-03-23T15:41:17.659326+0000","last_unstale":"2024-03-23T15:43:19.828033+0000","last_undegraded":"2024-03-23T15:43:19.828033+0000","last_fullsized":"2024-03-23T15:43:19.828033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:07.651340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":183,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855649+0000","last_change":"2024-03-23T15:41:18.656629+0000","last_active":"2024-03-23T15:43:19.855649+0000","last_peered":"2024-03-23T15:43:19.855649+0000","last_clean":"2024-03-23T15:43:19.855649+0000","last_became_active":"2024-03-23T15:41:18.656196+0000","last_became_peered":"2024-03-23T15:41:18.656196+0000","last_unstale":"2024-03-23T15:43:19.855649+0000","last_undegraded":"2024-03-23T15:43:19.855649+0000","last_fullsized":"2024-03-23T15:43:19.855649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:30.013411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855049+0000","last_change":"2024-03-23T15:41:17.663999+0000","last_active":"2024-03-23T15:43:19.855049+0000","last_peered":"2024-03-23T15:43:19.855049+0000","last_clean":"2024-03-23T15:43:19.855049+0000","last_became_active":"2024-03-23T15:41:17.663039+0000","last_became_peered":"2024-03-23T15:41:17.663039+0000","last_unstale":"2024-03-23T15:43:19.855049+0000","last_undegraded":"2024-03-23T15:43:19.855049+0000","last_fullsized":"2024-03-23T15:43:19.855049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:00.271821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828098+0000","last_change":"2024-03-23T15:41:18.646822+0000","last_active":"2024-03-23T15:43:19.828098+0000","last_peered":"2024-03-23T15:43:19.828098+0000","last_clean":"2024-03-23T15:43:19.828098+0000","last_became_active":"2024-03-23T15:41:18.646476+0000","last_became_peered":"2024-03-23T15:41:18.646476+0000","last_unstale":"2024-03-23T15:43:19.828098+0000","last_undegraded":"2024-03-23T15:43:19.828098+0000","last_fullsized":"2024-03-23T15:43:19.828098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:18.352065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829326+0000","last_change":"2024-03-23T15:41:17.663507+0000","last_active":"2024-03-23T15:43:19.829326+0000","last_peered":"2024-03-23T15:43:19.829326+0000","last_clean":"2024-03-23T15:43:19.829326+0000","last_became_active":"2024-03-23T15:41:17.663122+0000","last_became_peered":"2024-03-23T15:41:17.663122+0000","last_unstale":"2024-03-23T15:43:19.829326+0000","last_undegraded":"2024-03-23T15:43:19.829326+0000","last_fullsized":"2024-03-23T15:43:19.829326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:32.202576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855780+0000","last_change":"2024-03-23T15:41:18.650110+0000","last_active":"2024-03-23T15:43:19.855780+0000","last_peered":"2024-03-23T15:43:19.855780+0000","last_clean":"2024-03-23T15:43:19.855780+0000","last_became_active":"2024-03-23T15:41:18.649737+0000","last_became_peered":"2024-03-23T15:41:18.649737+0000","last_unstale":"2024-03-23T15:43:19.855780+0000","last_undegraded":"2024-03-23T15:43:19.855780+0000","last_fullsized":"2024-03-23T15:43:19.855780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:31.610961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830442+0000","last_change":"2024-03-23T15:41:17.653930+0000","last_active":"2024-03-23T15:43:19.830442+0000","last_peered":"2024-03-23T15:43:19.830442+0000","last_clean":"2024-03-23T15:43:19.830442+0000","last_became_active":"2024-03-23T15:41:17.652677+0000","last_became_peered":"2024-03-23T15:41:17.652677+0000","last_unstale":"2024-03-23T15:43:19.830442+0000","last_undegraded":"2024-03-23T15:43:19.830442+0000","last_fullsized":"2024-03-23T15:43:19.830442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:47.167052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855179+0000","last_change":"2024-03-23T15:42:16.843231+0000","last_active":"2024-03-23T15:43:19.855179+0000","last_peered":"2024-03-23T15:43:19.855179+0000","last_clean":"2024-03-23T15:43:19.855179+0000","last_became_active":"2024-03-23T15:42:16.842785+0000","last_became_peered":"2024-03-23T15:42:16.842785+0000","last_unstale":"2024-03-23T15:43:19.855179+0000","last_undegraded":"2024-03-23T15:43:19.855179+0000","last_fullsized":"2024-03-23T15:43:19.855179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:24:07.377828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854899+0000","last_change":"2024-03-23T15:41:17.662091+0000","last_active":"2024-03-23T15:43:19.854899+0000","last_peered":"2024-03-23T15:43:19.854899+0000","last_clean":"2024-03-23T15:43:19.854899+0000","last_became_active":"2024-03-23T15:41:17.661747+0000","last_became_peered":"2024-03-23T15:41:17.661747+0000","last_unstale":"2024-03-23T15:43:19.854899+0000","last_undegraded":"2024-03-23T15:43:19.854899+0000","last_fullsized":"2024-03-23T15:43:19.854899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:08.661262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829293+0000","last_change":"2024-03-23T15:42:16.627776+0000","last_active":"2024-03-23T15:43:19.829293+0000","last_peered":"2024-03-23T15:43:19.829293+0000","last_clean":"2024-03-23T15:43:19.829293+0000","last_became_active":"2024-03-23T15:42:16.627331+0000","last_became_peered":"2024-03-23T15:42:16.627331+0000","last_unstale":"2024-03-23T15:43:19.829293+0000","last_undegraded":"2024-03-23T15:43:19.829293+0000","last_fullsized":"2024-03-23T15:43:19.829293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:02:57.288827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855572+0000","last_change":"2024-03-23T15:41:17.660898+0000","last_active":"2024-03-23T15:43:19.855572+0000","last_peered":"2024-03-23T15:43:19.855572+0000","last_clean":"2024-03-23T15:43:19.855572+0000","last_became_active":"2024-03-23T15:41:17.660515+0000","last_became_peered":"2024-03-23T15:41:17.660515+0000","last_unstale":"2024-03-23T15:43:19.855572+0000","last_undegraded":"2024-03-23T15:43:19.855572+0000","last_fullsized":"2024-03-23T15:43:19.855572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:58.911321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795271+0000","last_change":"2024-03-23T15:41:18.653582+0000","last_active":"2024-03-23T15:43:19.795271+0000","last_peered":"2024-03-23T15:43:19.795271+0000","last_clean":"2024-03-23T15:43:19.795271+0000","last_became_active":"2024-03-23T15:41:18.653184+0000","last_became_peered":"2024-03-23T15:41:18.653184+0000","last_unstale":"2024-03-23T15:43:19.795271+0000","last_undegraded":"2024-03-23T15:43:19.795271+0000","last_fullsized":"2024-03-23T15:43:19.795271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:59:16.726981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793476+0000","last_change":"2024-03-23T15:41:17.651886+0000","last_active":"2024-03-23T15:43:19.793476+0000","last_peered":"2024-03-23T15:43:19.793476+0000","last_clean":"2024-03-23T15:43:19.793476+0000","last_became_active":"2024-03-23T15:41:17.651472+0000","last_became_peered":"2024-03-23T15:41:17.651472+0000","last_unstale":"2024-03-23T15:43:19.793476+0000","last_undegraded":"2024-03-23T15:43:19.793476+0000","last_fullsized":"2024-03-23T15:43:19.793476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:37.098333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795196+0000","last_change":"2024-03-23T15:41:18.644205+0000","last_active":"2024-03-23T15:43:19.795196+0000","last_peered":"2024-03-23T15:43:19.795196+0000","last_clean":"2024-03-23T15:43:19.795196+0000","last_became_active":"2024-03-23T15:41:18.643850+0000","last_became_peered":"2024-03-23T15:41:18.643850+0000","last_unstale":"2024-03-23T15:43:19.795196+0000","last_undegraded":"2024-03-23T15:43:19.795196+0000","last_fullsized":"2024-03-23T15:43:19.795196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:48:10.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855322+0000","last_change":"2024-03-23T15:41:17.661051+0000","last_active":"2024-03-23T15:43:19.855322+0000","last_peered":"2024-03-23T15:43:19.855322+0000","last_clean":"2024-03-23T15:43:19.855322+0000","last_became_active":"2024-03-23T15:41:17.659673+0000","last_became_peered":"2024-03-23T15:41:17.659673+0000","last_unstale":"2024-03-23T15:43:19.855322+0000","last_undegraded":"2024-03-23T15:43:19.855322+0000","last_fullsized":"2024-03-23T15:43:19.855322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.028104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795266+0000","last_change":"2024-03-23T15:42:16.843248+0000","last_active":"2024-03-23T15:43:19.795266+0000","last_peered":"2024-03-23T15:43:19.795266+0000","last_clean":"2024-03-23T15:43:19.795266+0000","last_became_active":"2024-03-23T15:42:16.842800+0000","last_became_peered":"2024-03-23T15:42:16.842800+0000","last_unstale":"2024-03-23T15:43:19.795266+0000","last_undegraded":"2024-03-23T15:43:19.795266+0000","last_fullsized":"2024-03-23T15:43:19.795266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829351+0000","last_change":"2024-03-23T15:41:18.653560+0000","last_active":"2024-03-23T15:43:19.829351+0000","last_peered":"2024-03-23T15:43:19.829351+0000","last_clean":"2024-03-23T15:43:19.829351+0000","last_became_active":"2024-03-23T15:41:18.653249+0000","last_became_peered":"2024-03-23T15:41:18.653249+0000","last_unstale":"2024-03-23T15:43:19.829351+0000","last_undegraded":"2024-03-23T15:43:19.829351+0000","last_fullsized":"2024-03-23T15:43:19.829351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:22.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828395+0000","last_change":"2024-03-23T15:41:17.656318+0000","last_active":"2024-03-23T15:43:19.828395+0000","last_peered":"2024-03-23T15:43:19.828395+0000","last_clean":"2024-03-23T15:43:19.828395+0000","last_became_active":"2024-03-23T15:41:17.655958+0000","last_became_peered":"2024-03-23T15:41:17.655958+0000","last_unstale":"2024-03-23T15:43:19.828395+0000","last_undegraded":"2024-03-23T15:43:19.828395+0000","last_fullsized":"2024-03-23T15:43:19.828395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:12.333504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903433+0000","last_change":"2024-03-23T15:41:18.657616+0000","last_active":"2024-03-23T15:43:19.903433+0000","last_peered":"2024-03-23T15:43:19.903433+0000","last_clean":"2024-03-23T15:43:19.903433+0000","last_became_active":"2024-03-23T15:41:18.657271+0000","last_became_peered":"2024-03-23T15:41:18.657271+0000","last_unstale":"2024-03-23T15:43:19.903433+0000","last_undegraded":"2024-03-23T15:43:19.903433+0000","last_fullsized":"2024-03-23T15:43:19.903433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:02.241087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830384+0000","last_change":"2024-03-23T15:41:17.654358+0000","last_active":"2024-03-23T15:43:19.830384+0000","last_peered":"2024-03-23T15:43:19.830384+0000","last_clean":"2024-03-23T15:43:19.830384+0000","last_became_active":"2024-03-23T15:41:17.653558+0000","last_became_peered":"2024-03-23T15:41:17.653558+0000","last_unstale":"2024-03-23T15:43:19.830384+0000","last_undegraded":"2024-03-23T15:43:19.830384+0000","last_fullsized":"2024-03-23T15:43:19.830384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:32.377883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855332+0000","last_change":"2024-03-23T15:40:05.871801+0000","last_active":"2024-03-23T15:43:19.855332+0000","last_peered":"2024-03-23T15:43:19.855332+0000","last_clean":"2024-03-23T15:43:19.855332+0000","last_became_active":"2024-03-23T15:40:05.871417+0000","last_became_peered":"2024-03-23T15:40:05.871417+0000","last_unstale":"2024-03-23T15:43:19.855332+0000","last_undegraded":"2024-03-23T15:43:19.855332+0000","last_fullsized":"2024-03-23T15:43:19.855332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_clean_scrub_stamp":"2024-03-23T15:36:39.745753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:55.235176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.827993+0000","last_change":"2024-03-23T15:41:18.652251+0000","last_active":"2024-03-23T15:43:19.827993+0000","last_peered":"2024-03-23T15:43:19.827993+0000","last_clean":"2024-03-23T15:43:19.827993+0000","last_became_active":"2024-03-23T15:41:18.651876+0000","last_became_peered":"2024-03-23T15:41:18.651876+0000","last_unstale":"2024-03-23T15:43:19.827993+0000","last_undegraded":"2024-03-23T15:43:19.827993+0000","last_fullsized":"2024-03-23T15:43:19.827993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:45.235243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855564+0000","last_change":"2024-03-23T15:42:16.692783+0000","last_active":"2024-03-23T15:43:19.855564+0000","last_peered":"2024-03-23T15:43:19.855564+0000","last_clean":"2024-03-23T15:43:19.855564+0000","last_became_active":"2024-03-23T15:42:16.690169+0000","last_became_peered":"2024-03-23T15:42:16.690169+0000","last_unstale":"2024-03-23T15:43:19.855564+0000","last_undegraded":"2024-03-23T15:43:19.855564+0000","last_fullsized":"2024-03-23T15:43:19.855564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:20.679040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855712+0000","last_change":"2024-03-23T15:41:17.660703+0000","last_active":"2024-03-23T15:43:19.855712+0000","last_peered":"2024-03-23T15:43:19.855712+0000","last_clean":"2024-03-23T15:43:19.855712+0000","last_became_active":"2024-03-23T15:41:17.660343+0000","last_became_peered":"2024-03-23T15:41:17.660343+0000","last_unstale":"2024-03-23T15:43:19.855712+0000","last_undegraded":"2024-03-23T15:43:19.855712+0000","last_fullsized":"2024-03-23T15:43:19.855712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:14:03.063397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828358+0000","last_change":"2024-03-23T15:41:18.644624+0000","last_active":"2024-03-23T15:43:19.828358+0000","last_peered":"2024-03-23T15:43:19.828358+0000","last_clean":"2024-03-23T15:43:19.828358+0000","last_became_active":"2024-03-23T15:41:18.644275+0000","last_became_peered":"2024-03-23T15:41:18.644275+0000","last_unstale":"2024-03-23T15:43:19.828358+0000","last_undegraded":"2024-03-23T15:43:19.828358+0000","last_fullsized":"2024-03-23T15:43:19.828358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:25.901935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'45","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.999901+0000","last_change":"2024-03-23T15:41:17.650068+0000","last_active":"2024-03-23T15:43:19.999901+0000","last_peered":"2024-03-23T15:43:19.999901+0000","last_clean":"2024-03-23T15:43:19.999901+0000","last_became_active":"2024-03-23T15:41:17.649331+0000","last_became_peered":"2024-03-23T15:41:17.649331+0000","last_unstale":"2024-03-23T15:43:19.999901+0000","last_undegraded":"2024-03-23T15:43:19.999901+0000","last_fullsized":"2024-03-23T15:43:19.999901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:47.578645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903378+0000","last_change":"2024-03-23T15:41:18.656413+0000","last_active":"2024-03-23T15:43:19.903378+0000","last_peered":"2024-03-23T15:43:19.903378+0000","last_clean":"2024-03-23T15:43:19.903378+0000","last_became_active":"2024-03-23T15:41:18.656043+0000","last_became_peered":"2024-03-23T15:41:18.656043+0000","last_unstale":"2024-03-23T15:43:19.903378+0000","last_undegraded":"2024-03-23T15:43:19.903378+0000","last_fullsized":"2024-03-23T15:43:19.903378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:49:25.453420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830801+0000","last_change":"2024-03-23T15:41:17.654492+0000","last_active":"2024-03-23T15:43:19.830801+0000","last_peered":"2024-03-23T15:43:19.830801+0000","last_clean":"2024-03-23T15:43:19.830801+0000","last_became_active":"2024-03-23T15:41:17.654132+0000","last_became_peered":"2024-03-23T15:41:17.654132+0000","last_unstale":"2024-03-23T15:43:19.830801+0000","last_undegraded":"2024-03-23T15:43:19.830801+0000","last_fullsized":"2024-03-23T15:43:19.830801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:39:58.210703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855201+0000","last_change":"2024-03-23T15:41:18.652659+0000","last_active":"2024-03-23T15:43:19.855201+0000","last_peered":"2024-03-23T15:43:19.855201+0000","last_clean":"2024-03-23T15:43:19.855201+0000","last_became_active":"2024-03-23T15:41:18.652246+0000","last_became_peered":"2024-03-23T15:41:18.652246+0000","last_unstale":"2024-03-23T15:43:19.855201+0000","last_undegraded":"2024-03-23T15:43:19.855201+0000","last_fullsized":"2024-03-23T15:43:19.855201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:50.868694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854944+0000","last_change":"2024-03-23T15:42:16.632166+0000","last_active":"2024-03-23T15:43:19.854944+0000","last_peered":"2024-03-23T15:43:19.854944+0000","last_clean":"2024-03-23T15:43:19.854944+0000","last_became_active":"2024-03-23T15:42:16.626587+0000","last_became_peered":"2024-03-23T15:42:16.626587+0000","last_unstale":"2024-03-23T15:43:19.854944+0000","last_undegraded":"2024-03-23T15:43:19.854944+0000","last_fullsized":"2024-03-23T15:43:19.854944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.514597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855068+0000","last_change":"2024-03-23T15:41:18.654049+0000","last_active":"2024-03-23T15:43:19.855068+0000","last_peered":"2024-03-23T15:43:19.855068+0000","last_clean":"2024-03-23T15:43:19.855068+0000","last_became_active":"2024-03-23T15:41:18.653765+0000","last_became_peered":"2024-03-23T15:41:18.653765+0000","last_unstale":"2024-03-23T15:43:19.855068+0000","last_undegraded":"2024-03-23T15:43:19.855068+0000","last_fullsized":"2024-03-23T15:43:19.855068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:12:11.467744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902818+0000","last_change":"2024-03-23T15:41:17.656679+0000","last_active":"2024-03-23T15:43:19.902818+0000","last_peered":"2024-03-23T15:43:19.902818+0000","last_clean":"2024-03-23T15:43:19.902818+0000","last_became_active":"2024-03-23T15:41:17.656368+0000","last_became_peered":"2024-03-23T15:41:17.656368+0000","last_unstale":"2024-03-23T15:43:19.902818+0000","last_undegraded":"2024-03-23T15:43:19.902818+0000","last_fullsized":"2024-03-23T15:43:19.902818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:14.176288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794941+0000","last_change":"2024-03-23T15:41:18.653637+0000","last_active":"2024-03-23T15:43:19.794941+0000","last_peered":"2024-03-23T15:43:19.794941+0000","last_clean":"2024-03-23T15:43:19.794941+0000","last_became_active":"2024-03-23T15:41:18.652864+0000","last_became_peered":"2024-03-23T15:41:18.652864+0000","last_unstale":"2024-03-23T15:43:19.794941+0000","last_undegraded":"2024-03-23T15:43:19.794941+0000","last_fullsized":"2024-03-23T15:43:19.794941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:32.104901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855151+0000","last_change":"2024-03-23T15:41:17.664417+0000","last_active":"2024-03-23T15:43:19.855151+0000","last_peered":"2024-03-23T15:43:19.855151+0000","last_clean":"2024-03-23T15:43:19.855151+0000","last_became_active":"2024-03-23T15:41:17.663454+0000","last_became_peered":"2024-03-23T15:41:17.663454+0000","last_unstale":"2024-03-23T15:43:19.855151+0000","last_undegraded":"2024-03-23T15:43:19.855151+0000","last_fullsized":"2024-03-23T15:43:19.855151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:14.947767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903278+0000","last_change":"2024-03-23T15:41:18.647857+0000","last_active":"2024-03-23T15:43:19.903278+0000","last_peered":"2024-03-23T15:43:19.903278+0000","last_clean":"2024-03-23T15:43:19.903278+0000","last_became_active":"2024-03-23T15:41:18.646924+0000","last_became_peered":"2024-03-23T15:41:18.646924+0000","last_unstale":"2024-03-23T15:43:19.903278+0000","last_undegraded":"2024-03-23T15:43:19.903278+0000","last_fullsized":"2024-03-23T15:43:19.903278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:19.494901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855517+0000","last_change":"2024-03-23T15:41:17.661995+0000","last_active":"2024-03-23T15:43:19.855517+0000","last_peered":"2024-03-23T15:43:19.855517+0000","last_clean":"2024-03-23T15:43:19.855517+0000","last_became_active":"2024-03-23T15:41:17.661632+0000","last_became_peered":"2024-03-23T15:41:17.661632+0000","last_unstale":"2024-03-23T15:43:19.855517+0000","last_undegraded":"2024-03-23T15:43:19.855517+0000","last_fullsized":"2024-03-23T15:43:19.855517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:12.540038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855504+0000","last_change":"2024-03-23T15:41:18.651258+0000","last_active":"2024-03-23T15:43:19.855504+0000","last_peered":"2024-03-23T15:43:19.855504+0000","last_clean":"2024-03-23T15:43:19.855504+0000","last_became_active":"2024-03-23T15:41:18.650910+0000","last_became_peered":"2024-03-23T15:41:18.650910+0000","last_unstale":"2024-03-23T15:43:19.855504+0000","last_undegraded":"2024-03-23T15:43:19.855504+0000","last_fullsized":"2024-03-23T15:43:19.855504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:06:50.139644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855304+0000","last_change":"2024-03-23T15:41:17.654232+0000","last_active":"2024-03-23T15:43:19.855304+0000","last_peered":"2024-03-23T15:43:19.855304+0000","last_clean":"2024-03-23T15:43:19.855304+0000","last_became_active":"2024-03-23T15:41:17.653832+0000","last_became_peered":"2024-03-23T15:41:17.653832+0000","last_unstale":"2024-03-23T15:43:19.855304+0000","last_undegraded":"2024-03-23T15:43:19.855304+0000","last_fullsized":"2024-03-23T15:43:19.855304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:05.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855755+0000","last_change":"2024-03-23T15:41:18.664444+0000","last_active":"2024-03-23T15:43:19.855755+0000","last_peered":"2024-03-23T15:43:19.855755+0000","last_clean":"2024-03-23T15:43:19.855755+0000","last_became_active":"2024-03-23T15:41:18.664127+0000","last_became_peered":"2024-03-23T15:41:18.664127+0000","last_unstale":"2024-03-23T15:43:19.855755+0000","last_undegraded":"2024-03-23T15:43:19.855755+0000","last_fullsized":"2024-03-23T15:43:19.855755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:25.942230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.995159+0000","last_change":"2024-03-23T15:41:17.664834+0000","last_active":"2024-03-23T15:43:19.995159+0000","last_peered":"2024-03-23T15:43:19.995159+0000","last_clean":"2024-03-23T15:43:19.995159+0000","last_became_active":"2024-03-23T15:41:17.664286+0000","last_became_peered":"2024-03-23T15:41:17.664286+0000","last_unstale":"2024-03-23T15:43:19.995159+0000","last_undegraded":"2024-03-23T15:43:19.995159+0000","last_fullsized":"2024-03-23T15:43:19.995159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:50.595420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855495+0000","last_change":"2024-03-23T15:41:18.646341+0000","last_active":"2024-03-23T15:43:19.855495+0000","last_peered":"2024-03-23T15:43:19.855495+0000","last_clean":"2024-03-23T15:43:19.855495+0000","last_became_active":"2024-03-23T15:41:18.645995+0000","last_became_peered":"2024-03-23T15:41:18.645995+0000","last_unstale":"2024-03-23T15:43:19.855495+0000","last_undegraded":"2024-03-23T15:43:19.855495+0000","last_fullsized":"2024-03-23T15:43:19.855495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:56.408957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828254+0000","last_change":"2024-03-23T15:42:16.694643+0000","last_active":"2024-03-23T15:43:19.828254+0000","last_peered":"2024-03-23T15:43:19.828254+0000","last_clean":"2024-03-23T15:43:19.828254+0000","last_became_active":"2024-03-23T15:42:16.690063+0000","last_became_peered":"2024-03-23T15:42:16.690063+0000","last_unstale":"2024-03-23T15:43:19.828254+0000","last_undegraded":"2024-03-23T15:43:19.828254+0000","last_fullsized":"2024-03-23T15:43:19.828254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:54.873573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855942+0000","last_change":"2024-03-23T15:41:18.655250+0000","last_active":"2024-03-23T15:43:19.855942+0000","last_peered":"2024-03-23T15:43:19.855942+0000","last_clean":"2024-03-23T15:43:19.855942+0000","last_became_active":"2024-03-23T15:41:18.654888+0000","last_became_peered":"2024-03-23T15:41:18.654888+0000","last_unstale":"2024-03-23T15:43:19.855942+0000","last_undegraded":"2024-03-23T15:43:19.855942+0000","last_fullsized":"2024-03-23T15:43:19.855942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:36.730600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795028+0000","last_change":"2024-03-23T15:41:18.654309+0000","last_active":"2024-03-23T15:43:19.795028+0000","last_peered":"2024-03-23T15:43:19.795028+0000","last_clean":"2024-03-23T15:43:19.795028+0000","last_became_active":"2024-03-23T15:41:18.653319+0000","last_became_peered":"2024-03-23T15:41:18.653319+0000","last_unstale":"2024-03-23T15:43:19.795028+0000","last_undegraded":"2024-03-23T15:43:19.795028+0000","last_fullsized":"2024-03-23T15:43:19.795028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:02:05.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855072+0000","last_change":"2024-03-23T15:41:18.655179+0000","last_active":"2024-03-23T15:43:19.855072+0000","last_peered":"2024-03-23T15:43:19.855072+0000","last_clean":"2024-03-23T15:43:19.855072+0000","last_became_active":"2024-03-23T15:41:18.654830+0000","last_became_peered":"2024-03-23T15:41:18.654830+0000","last_unstale":"2024-03-23T15:43:19.855072+0000","last_undegraded":"2024-03-23T15:43:19.855072+0000","last_fullsized":"2024-03-23T15:43:19.855072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:29.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854909+0000","last_change":"2024-03-23T15:41:18.650734+0000","last_active":"2024-03-23T15:43:19.854909+0000","last_peered":"2024-03-23T15:43:19.854909+0000","last_clean":"2024-03-23T15:43:19.854909+0000","last_became_active":"2024-03-23T15:41:18.649993+0000","last_became_peered":"2024-03-23T15:41:18.649993+0000","last_unstale":"2024-03-23T15:43:19.854909+0000","last_undegraded":"2024-03-23T15:43:19.854909+0000","last_fullsized":"2024-03-23T15:43:19.854909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:06.463525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854958+0000","last_change":"2024-03-23T15:41:18.655741+0000","last_active":"2024-03-23T15:43:19.854958+0000","last_peered":"2024-03-23T15:43:19.854958+0000","last_clean":"2024-03-23T15:43:19.854958+0000","last_became_active":"2024-03-23T15:41:18.653387+0000","last_became_peered":"2024-03-23T15:41:18.653387+0000","last_unstale":"2024-03-23T15:43:19.854958+0000","last_undegraded":"2024-03-23T15:43:19.854958+0000","last_fullsized":"2024-03-23T15:43:19.854958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:20:26.758549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828382+0000","last_change":"2024-03-23T15:41:18.654739+0000","last_active":"2024-03-23T15:43:19.828382+0000","last_peered":"2024-03-23T15:43:19.828382+0000","last_clean":"2024-03-23T15:43:19.828382+0000","last_became_active":"2024-03-23T15:41:18.654151+0000","last_became_peered":"2024-03-23T15:41:18.654151+0000","last_unstale":"2024-03-23T15:43:19.828382+0000","last_undegraded":"2024-03-23T15:43:19.828382+0000","last_fullsized":"2024-03-23T15:43:19.828382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:01.358959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795157+0000","last_change":"2024-03-23T15:41:18.654081+0000","last_active":"2024-03-23T15:43:19.795157+0000","last_peered":"2024-03-23T15:43:19.795157+0000","last_clean":"2024-03-23T15:43:19.795157+0000","last_became_active":"2024-03-23T15:41:18.653696+0000","last_became_peered":"2024-03-23T15:41:18.653696+0000","last_unstale":"2024-03-23T15:43:19.795157+0000","last_undegraded":"2024-03-23T15:43:19.795157+0000","last_fullsized":"2024-03-23T15:43:19.795157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:23:19.741645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855549+0000","last_change":"2024-03-23T15:41:18.652356+0000","last_active":"2024-03-23T15:43:19.855549+0000","last_peered":"2024-03-23T15:43:19.855549+0000","last_clean":"2024-03-23T15:43:19.855549+0000","last_became_active":"2024-03-23T15:41:18.651976+0000","last_became_peered":"2024-03-23T15:41:18.651976+0000","last_unstale":"2024-03-23T15:43:19.855549+0000","last_undegraded":"2024-03-23T15:43:19.855549+0000","last_fullsized":"2024-03-23T15:43:19.855549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:38.972186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903249+0000","last_change":"2024-03-23T15:41:18.653775+0000","last_active":"2024-03-23T15:43:19.903249+0000","last_peered":"2024-03-23T15:43:19.903249+0000","last_clean":"2024-03-23T15:43:19.903249+0000","last_became_active":"2024-03-23T15:41:18.653404+0000","last_became_peered":"2024-03-23T15:41:18.653404+0000","last_unstale":"2024-03-23T15:43:19.903249+0000","last_undegraded":"2024-03-23T15:43:19.903249+0000","last_fullsized":"2024-03-23T15:43:19.903249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:28:57.534731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854866+0000","last_change":"2024-03-23T15:42:16.843115+0000","last_active":"2024-03-23T15:43:19.854866+0000","last_peered":"2024-03-23T15:43:19.854866+0000","last_clean":"2024-03-23T15:43:19.854866+0000","last_became_active":"2024-03-23T15:42:16.842743+0000","last_became_peered":"2024-03-23T15:42:16.842743+0000","last_unstale":"2024-03-23T15:43:19.854866+0000","last_undegraded":"2024-03-23T15:43:19.854866+0000","last_fullsized":"2024-03-23T15:43:19.854866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:36:11.377398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902672+0000","last_change":"2024-03-23T15:41:18.652487+0000","last_active":"2024-03-23T15:43:19.902672+0000","last_peered":"2024-03-23T15:43:19.902672+0000","last_clean":"2024-03-23T15:43:19.902672+0000","last_became_active":"2024-03-23T15:41:18.652151+0000","last_became_peered":"2024-03-23T15:41:18.652151+0000","last_unstale":"2024-03-23T15:43:19.902672+0000","last_undegraded":"2024-03-23T15:43:19.902672+0000","last_fullsized":"2024-03-23T15:43:19.902672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:39.133585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855151+0000","last_change":"2024-03-23T15:41:18.657265+0000","last_active":"2024-03-23T15:43:19.855151+0000","last_peered":"2024-03-23T15:43:19.855151+0000","last_clean":"2024-03-23T15:43:19.855151+0000","last_became_active":"2024-03-23T15:41:18.656970+0000","last_became_peered":"2024-03-23T15:41:18.656970+0000","last_unstale":"2024-03-23T15:43:19.855151+0000","last_undegraded":"2024-03-23T15:43:19.855151+0000","last_fullsized":"2024-03-23T15:43:19.855151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:22.808034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828277+0000","last_change":"2024-03-23T15:41:18.655254+0000","last_active":"2024-03-23T15:43:19.828277+0000","last_peered":"2024-03-23T15:43:19.828277+0000","last_clean":"2024-03-23T15:43:19.828277+0000","last_became_active":"2024-03-23T15:41:18.654879+0000","last_became_peered":"2024-03-23T15:41:18.654879+0000","last_unstale":"2024-03-23T15:43:19.828277+0000","last_undegraded":"2024-03-23T15:43:19.828277+0000","last_fullsized":"2024-03-23T15:43:19.828277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:36:18.923723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795101+0000","last_change":"2024-03-23T15:41:18.658652+0000","last_active":"2024-03-23T15:43:19.795101+0000","last_peered":"2024-03-23T15:43:19.795101+0000","last_clean":"2024-03-23T15:43:19.795101+0000","last_became_active":"2024-03-23T15:41:18.658261+0000","last_became_peered":"2024-03-23T15:41:18.658261+0000","last_unstale":"2024-03-23T15:43:19.795101+0000","last_undegraded":"2024-03-23T15:43:19.795101+0000","last_fullsized":"2024-03-23T15:43:19.795101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:45.145101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106812,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597498,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":830,"num_read_kb":518,"num_write":285,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1792494,"data_compressed":81513,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":465380,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1122304,"data_stored":1250064,"data_compressed":52357,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":858832,"data_compressed":27874,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":858752,"data_compressed":27874,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":1058780,"data_compressed":44332,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":958464,"data_stored":465005,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":532165,"data_compressed":10597,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":466276,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":528283,"data_compressed":10597,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":990132,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":465833,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1189673,"data_compressed":50488,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67367,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70031,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134390,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200936,"data_compressed":8025,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T15:43:24.536 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-23T15:43:24.537 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Unable to set osd_memory_target on smithi192 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/4142234672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:24.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi076 to 176.2M 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Unable to set osd_memory_target on smithi076 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:43:24.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi183 to 174.1M 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Unable to set osd_memory_target on smithi183 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Updating smithi076:/etc/ceph/ceph.conf 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Updating smithi192:/etc/ceph/ceph.conf 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3213391654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T15:43:24.572 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:24 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2788240890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 -- 172.21.15.76:0/3660108947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 msgr2=0x7fd008106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 --2- 172.21.15.76:0/3660108947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd008106a60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd000009ca0 tx=0x7fd00002c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 -- 172.21.15.76:0/3660108947 shutdown_connections 2024-03-23T15:43:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 --2- 172.21.15.76:0/3660108947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd008100f80 0x7fd00810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 --2- 172.21.15.76:0/3660108947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0081005c0 0x7fd008100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 --2- 172.21.15.76:0/3660108947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd008106a60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 -- 172.21.15.76:0/3660108947 >> 172.21.15.76:0/3660108947 conn(0x7fd0080776e0 msgr2=0x7fd008077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 -- 172.21.15.76:0/3660108947 shutdown_connections 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.672+0000 7fd00f847640 1 -- 172.21.15.76:0/3660108947 wait complete. 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 Processor -- start 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 -- start start 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0081005c0 0x7fd008198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd008100f80 0x7fd008199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd0081a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd008199890 con 0x7fd008106680 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd008199a00 con 0x7fd008100f80 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00f847640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081a4180 con 0x7fd0081005c0 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd0081a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd0081a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49830/0 (socket says 172.21.15.76:49830) 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 -- 172.21.15.76:0/310177193 learned_addr learned my addr 172.21.15.76:0/310177193 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:24.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00d5e3640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0081005c0 0x7fd008198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00cde2640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd008100f80 0x7fd008199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 -- 172.21.15.76:0/310177193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0081005c0 msgr2=0x7fd008198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0081005c0 0x7fd008198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 -- 172.21.15.76:0/310177193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd008100f80 msgr2=0x7fd008199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd008100f80 0x7fd008199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 -- 172.21.15.76:0/310177193 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd000009900 con 0x7fd008106680 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.673+0000 7fd00dde4640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd0081a3bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcff800d950 tx=0x7fcff800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.674+0000 7fd0067fc640 1 -- 172.21.15.76:0/310177193 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8009a30 con 0x7fd008106680 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.674+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0081a4460 con 0x7fd008106680 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.674+0000 7fd0067fc640 1 -- 172.21.15.76:0/310177193 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcff8009b90 con 0x7fd008106680 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.674+0000 7fd0067fc640 1 -- 172.21.15.76:0/310177193 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8003d60 con 0x7fd008106680 2024-03-23T15:43:24.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.674+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0081a4970 con 0x7fd008106680 2024-03-23T15:43:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.675+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd008101ce0 con 0x7fd008106680 2024-03-23T15:43:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.676+0000 7fd0067fc640 1 -- 172.21.15.76:0/310177193 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7fcff8009d00 con 0x7fd008106680 2024-03-23T15:43:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.676+0000 7fd0067fc640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fcff00a2350 0x7fcff00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.676+0000 7fd00d5e3640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fcff00a2350 0x7fcff00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.676+0000 7fd0067fc640 1 -- 172.21.15.76:0/310177193 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fcff80ce090 con 0x7fd008106680 2024-03-23T15:43:24.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.677+0000 7fd00d5e3640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fcff00a2350 0x7fcff00a4810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd000004af0 tx=0x7fd000005c50 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:24.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.680+0000 7fd0067fc640 1 -- 172.21.15.76:0/310177193 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fcff8096ad0 con 0x7fd008106680 2024-03-23T15:43:24.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.851+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd008108ae0 con 0x7fcff00a2350 2024-03-23T15:43:24.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.858+0000 7fd0067fc640 1 -- 172.21.15.76:0/310177193 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258021 (secure 0 0 0) 0x7fd008108ae0 con 0x7fcff00a2350 2024-03-23T15:43:24.864 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:24.865 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fcff00a2350 msgr2=0x7fcff00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fcff00a2350 0x7fcff00a4810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd000004af0 tx=0x7fd000005c50 comp rx=0 tx=0).stop 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 msgr2=0x7fd0081a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd0081a3bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcff800d950 tx=0x7fcff800de10 comp rx=0 tx=0).stop 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 shutdown_connections 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fcff00a2350 0x7fcff00a4810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd008106680 0x7fd0081a3bc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd008100f80 0x7fd008199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 --2- 172.21.15.76:0/310177193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0081005c0 0x7fd008198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 >> 172.21.15.76:0/310177193 conn(0x7fd0080776e0 msgr2=0x7fd0080ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 shutdown_connections 2024-03-23T15:43:24.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:24.861+0000 7fd00f847640 1 -- 172.21.15.76:0/310177193 wait complete. 2024-03-23T15:43:24.875 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-03-23T15:43:23.805884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188100,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":966,"num_read_kb":637,"num_write":468,"num_write_kb":4375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":863,"ondisk_log_size":863,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369360,"kb_used_data":11840,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572464,"statfs":{"total":1151940427776,"available":1151562203136,"internally_reserved":0,"allocated":12124160,"data_stored":9129175,"data_compressed":293061,"data_compressed_allocated":393216,"data_compressed_original":3637248,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.975010"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795300+0000","last_change":"2024-03-23T15:41:18.651731+0000","last_active":"2024-03-23T15:43:19.795300+0000","last_peered":"2024-03-23T15:43:19.795300+0000","last_clean":"2024-03-23T15:43:19.795300+0000","last_became_active":"2024-03-23T15:41:18.651312+0000","last_became_peered":"2024-03-23T15:41:18.651312+0000","last_unstale":"2024-03-23T15:43:19.795300+0000","last_undegraded":"2024-03-23T15:43:19.795300+0000","last_fullsized":"2024-03-23T15:43:19.795300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:39:20.652463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855216+0000","last_change":"2024-03-23T15:41:18.655736+0000","last_active":"2024-03-23T15:43:19.855216+0000","last_peered":"2024-03-23T15:43:19.855216+0000","last_clean":"2024-03-23T15:43:19.855216+0000","last_became_active":"2024-03-23T15:41:18.655389+0000","last_became_peered":"2024-03-23T15:41:18.655389+0000","last_unstale":"2024-03-23T15:43:19.855216+0000","last_undegraded":"2024-03-23T15:43:19.855216+0000","last_fullsized":"2024-03-23T15:43:19.855216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:35:56.260056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855070+0000","last_change":"2024-03-23T15:41:18.650709+0000","last_active":"2024-03-23T15:43:19.855070+0000","last_peered":"2024-03-23T15:43:19.855070+0000","last_clean":"2024-03-23T15:43:19.855070+0000","last_became_active":"2024-03-23T15:41:18.650316+0000","last_became_peered":"2024-03-23T15:41:18.650316+0000","last_unstale":"2024-03-23T15:43:19.855070+0000","last_undegraded":"2024-03-23T15:43:19.855070+0000","last_fullsized":"2024-03-23T15:43:19.855070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:50:57.811521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830666+0000","last_change":"2024-03-23T15:41:18.652216+0000","last_active":"2024-03-23T15:43:19.830666+0000","last_peered":"2024-03-23T15:43:19.830666+0000","last_clean":"2024-03-23T15:43:19.830666+0000","last_became_active":"2024-03-23T15:41:18.650525+0000","last_became_peered":"2024-03-23T15:41:18.650525+0000","last_unstale":"2024-03-23T15:43:19.830666+0000","last_undegraded":"2024-03-23T15:43:19.830666+0000","last_fullsized":"2024-03-23T15:43:19.830666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:16.458257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854905+0000","last_change":"2024-03-23T15:41:18.649146+0000","last_active":"2024-03-23T15:43:19.854905+0000","last_peered":"2024-03-23T15:43:19.854905+0000","last_clean":"2024-03-23T15:43:19.854905+0000","last_became_active":"2024-03-23T15:41:18.648728+0000","last_became_peered":"2024-03-23T15:41:18.648728+0000","last_unstale":"2024-03-23T15:43:19.854905+0000","last_undegraded":"2024-03-23T15:43:19.854905+0000","last_fullsized":"2024-03-23T15:43:19.854905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:09:56.251250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794938+0000","last_change":"2024-03-23T15:41:18.652957+0000","last_active":"2024-03-23T15:43:19.794938+0000","last_peered":"2024-03-23T15:43:19.794938+0000","last_clean":"2024-03-23T15:43:19.794938+0000","last_became_active":"2024-03-23T15:41:18.652578+0000","last_became_peered":"2024-03-23T15:41:18.652578+0000","last_unstale":"2024-03-23T15:43:19.794938+0000","last_undegraded":"2024-03-23T15:43:19.794938+0000","last_fullsized":"2024-03-23T15:43:19.794938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:46:38.961554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855251+0000","last_change":"2024-03-23T15:41:18.646134+0000","last_active":"2024-03-23T15:43:19.855251+0000","last_peered":"2024-03-23T15:43:19.855251+0000","last_clean":"2024-03-23T15:43:19.855251+0000","last_became_active":"2024-03-23T15:41:18.645680+0000","last_became_peered":"2024-03-23T15:41:18.645680+0000","last_unstale":"2024-03-23T15:43:19.855251+0000","last_undegraded":"2024-03-23T15:43:19.855251+0000","last_fullsized":"2024-03-23T15:43:19.855251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:34:31.850115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902586+0000","last_change":"2024-03-23T15:41:18.646143+0000","last_active":"2024-03-23T15:43:19.902586+0000","last_peered":"2024-03-23T15:43:19.902586+0000","last_clean":"2024-03-23T15:43:19.902586+0000","last_became_active":"2024-03-23T15:41:18.645771+0000","last_became_peered":"2024-03-23T15:41:18.645771+0000","last_unstale":"2024-03-23T15:43:19.902586+0000","last_undegraded":"2024-03-23T15:43:19.902586+0000","last_fullsized":"2024-03-23T15:43:19.902586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:55:39.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855670+0000","last_change":"2024-03-23T15:41:18.665083+0000","last_active":"2024-03-23T15:43:19.855670+0000","last_peered":"2024-03-23T15:43:19.855670+0000","last_clean":"2024-03-23T15:43:19.855670+0000","last_became_active":"2024-03-23T15:41:18.663143+0000","last_became_peered":"2024-03-23T15:41:18.663143+0000","last_unstale":"2024-03-23T15:43:19.855670+0000","last_undegraded":"2024-03-23T15:43:19.855670+0000","last_fullsized":"2024-03-23T15:43:19.855670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:52:23.421645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795361+0000","last_change":"2024-03-23T15:41:18.650014+0000","last_active":"2024-03-23T15:43:19.795361+0000","last_peered":"2024-03-23T15:43:19.795361+0000","last_clean":"2024-03-23T15:43:19.795361+0000","last_became_active":"2024-03-23T15:41:18.649673+0000","last_became_peered":"2024-03-23T15:41:18.649673+0000","last_unstale":"2024-03-23T15:43:19.795361+0000","last_undegraded":"2024-03-23T15:43:19.795361+0000","last_fullsized":"2024-03-23T15:43:19.795361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:30.387993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855606+0000","last_change":"2024-03-23T15:41:18.665622+0000","last_active":"2024-03-23T15:43:19.855606+0000","last_peered":"2024-03-23T15:43:19.855606+0000","last_clean":"2024-03-23T15:43:19.855606+0000","last_became_active":"2024-03-23T15:41:18.665274+0000","last_became_peered":"2024-03-23T15:41:18.665274+0000","last_unstale":"2024-03-23T15:43:19.855606+0000","last_undegraded":"2024-03-23T15:43:19.855606+0000","last_fullsized":"2024-03-23T15:43:19.855606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:12:18.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830547+0000","last_change":"2024-03-23T15:41:18.650888+0000","last_active":"2024-03-23T15:43:19.830547+0000","last_peered":"2024-03-23T15:43:19.830547+0000","last_clean":"2024-03-23T15:43:19.830547+0000","last_became_active":"2024-03-23T15:41:18.650363+0000","last_became_peered":"2024-03-23T15:41:18.650363+0000","last_unstale":"2024-03-23T15:43:19.830547+0000","last_undegraded":"2024-03-23T15:43:19.830547+0000","last_fullsized":"2024-03-23T15:43:19.830547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:42:41.289697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828315+0000","last_change":"2024-03-23T15:41:18.634183+0000","last_active":"2024-03-23T15:43:19.828315+0000","last_peered":"2024-03-23T15:43:19.828315+0000","last_clean":"2024-03-23T15:43:19.828315+0000","last_became_active":"2024-03-23T15:41:18.633849+0000","last_became_peered":"2024-03-23T15:41:18.633849+0000","last_unstale":"2024-03-23T15:43:19.828315+0000","last_undegraded":"2024-03-23T15:43:19.828315+0000","last_fullsized":"2024-03-23T15:43:19.828315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:49:13.707720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854833+0000","last_change":"2024-03-23T15:41:18.645269+0000","last_active":"2024-03-23T15:43:19.854833+0000","last_peered":"2024-03-23T15:43:19.854833+0000","last_clean":"2024-03-23T15:43:19.854833+0000","last_became_active":"2024-03-23T15:41:18.644940+0000","last_became_peered":"2024-03-23T15:41:18.644940+0000","last_unstale":"2024-03-23T15:43:19.854833+0000","last_undegraded":"2024-03-23T15:43:19.854833+0000","last_fullsized":"2024-03-23T15:43:19.854833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:22:26.401890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854878+0000","last_change":"2024-03-23T15:41:18.652789+0000","last_active":"2024-03-23T15:43:19.854878+0000","last_peered":"2024-03-23T15:43:19.854878+0000","last_clean":"2024-03-23T15:43:19.854878+0000","last_became_active":"2024-03-23T15:41:18.651520+0000","last_became_peered":"2024-03-23T15:41:18.651520+0000","last_unstale":"2024-03-23T15:43:19.854878+0000","last_undegraded":"2024-03-23T15:43:19.854878+0000","last_fullsized":"2024-03-23T15:43:19.854878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:00.388217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854859+0000","last_change":"2024-03-23T15:41:18.660398+0000","last_active":"2024-03-23T15:43:19.854859+0000","last_peered":"2024-03-23T15:43:19.854859+0000","last_clean":"2024-03-23T15:43:19.854859+0000","last_became_active":"2024-03-23T15:41:18.660084+0000","last_became_peered":"2024-03-23T15:41:18.660084+0000","last_unstale":"2024-03-23T15:43:19.854859+0000","last_undegraded":"2024-03-23T15:43:19.854859+0000","last_fullsized":"2024-03-23T15:43:19.854859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:04:33.115173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795310+0000","last_change":"2024-03-23T15:41:18.663192+0000","last_active":"2024-03-23T15:43:19.795310+0000","last_peered":"2024-03-23T15:43:19.795310+0000","last_clean":"2024-03-23T15:43:19.795310+0000","last_became_active":"2024-03-23T15:41:18.662844+0000","last_became_peered":"2024-03-23T15:41:18.662844+0000","last_unstale":"2024-03-23T15:43:19.795310+0000","last_undegraded":"2024-03-23T15:43:19.795310+0000","last_fullsized":"2024-03-23T15:43:19.795310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:24.260706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793526+0000","last_change":"2024-03-23T15:41:18.646405+0000","last_active":"2024-03-23T15:43:19.793526+0000","last_peered":"2024-03-23T15:43:19.793526+0000","last_clean":"2024-03-23T15:43:19.793526+0000","last_became_active":"2024-03-23T15:41:18.645516+0000","last_became_peered":"2024-03-23T15:41:18.645516+0000","last_unstale":"2024-03-23T15:43:19.793526+0000","last_undegraded":"2024-03-23T15:43:19.793526+0000","last_fullsized":"2024-03-23T15:43:19.793526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:29:18.588190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855413+0000","last_change":"2024-03-23T15:42:16.843159+0000","last_active":"2024-03-23T15:43:19.855413+0000","last_peered":"2024-03-23T15:43:19.855413+0000","last_clean":"2024-03-23T15:43:19.855413+0000","last_became_active":"2024-03-23T15:42:16.842729+0000","last_became_peered":"2024-03-23T15:42:16.842729+0000","last_unstale":"2024-03-23T15:43:19.855413+0000","last_undegraded":"2024-03-23T15:43:19.855413+0000","last_fullsized":"2024-03-23T15:43:19.855413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:57:18.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793618+0000","last_change":"2024-03-23T15:41:17.655443+0000","last_active":"2024-03-23T15:43:19.793618+0000","last_peered":"2024-03-23T15:43:19.793618+0000","last_clean":"2024-03-23T15:43:19.793618+0000","last_became_active":"2024-03-23T15:41:17.654834+0000","last_became_peered":"2024-03-23T15:41:17.654834+0000","last_unstale":"2024-03-23T15:43:19.793618+0000","last_undegraded":"2024-03-23T15:43:19.793618+0000","last_fullsized":"2024-03-23T15:43:19.793618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:49:43.215644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903720+0000","last_change":"2024-03-23T15:41:18.650241+0000","last_active":"2024-03-23T15:43:19.903720+0000","last_peered":"2024-03-23T15:43:19.903720+0000","last_clean":"2024-03-23T15:43:19.903720+0000","last_became_active":"2024-03-23T15:41:18.649914+0000","last_became_peered":"2024-03-23T15:41:18.649914+0000","last_unstale":"2024-03-23T15:43:19.903720+0000","last_undegraded":"2024-03-23T15:43:19.903720+0000","last_fullsized":"2024-03-23T15:43:19.903720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:35:18.892649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903809+0000","last_change":"2024-03-23T15:41:17.663308+0000","last_active":"2024-03-23T15:43:19.903809+0000","last_peered":"2024-03-23T15:43:19.903809+0000","last_clean":"2024-03-23T15:43:19.903809+0000","last_became_active":"2024-03-23T15:41:17.662958+0000","last_became_peered":"2024-03-23T15:41:17.662958+0000","last_unstale":"2024-03-23T15:43:19.903809+0000","last_undegraded":"2024-03-23T15:43:19.903809+0000","last_fullsized":"2024-03-23T15:43:19.903809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:43:37.512893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855316+0000","last_change":"2024-03-23T15:41:18.651225+0000","last_active":"2024-03-23T15:43:19.855316+0000","last_peered":"2024-03-23T15:43:19.855316+0000","last_clean":"2024-03-23T15:43:19.855316+0000","last_became_active":"2024-03-23T15:41:18.650884+0000","last_became_peered":"2024-03-23T15:41:18.650884+0000","last_unstale":"2024-03-23T15:43:19.855316+0000","last_undegraded":"2024-03-23T15:43:19.855316+0000","last_fullsized":"2024-03-23T15:43:19.855316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:58:00.445203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855793+0000","last_change":"2024-03-23T15:41:17.662753+0000","last_active":"2024-03-23T15:43:19.855793+0000","last_peered":"2024-03-23T15:43:19.855793+0000","last_clean":"2024-03-23T15:43:19.855793+0000","last_became_active":"2024-03-23T15:41:17.662378+0000","last_became_peered":"2024-03-23T15:41:17.662378+0000","last_unstale":"2024-03-23T15:43:19.855793+0000","last_undegraded":"2024-03-23T15:43:19.855793+0000","last_fullsized":"2024-03-23T15:43:19.855793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:58:55.674739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854935+0000","last_change":"2024-03-23T15:41:18.659171+0000","last_active":"2024-03-23T15:43:19.854935+0000","last_peered":"2024-03-23T15:43:19.854935+0000","last_clean":"2024-03-23T15:43:19.854935+0000","last_became_active":"2024-03-23T15:41:18.658836+0000","last_became_peered":"2024-03-23T15:41:18.658836+0000","last_unstale":"2024-03-23T15:43:19.854935+0000","last_undegraded":"2024-03-23T15:43:19.854935+0000","last_fullsized":"2024-03-23T15:43:19.854935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:15.056528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794998+0000","last_change":"2024-03-23T15:42:16.631598+0000","last_active":"2024-03-23T15:43:19.794998+0000","last_peered":"2024-03-23T15:43:19.794998+0000","last_clean":"2024-03-23T15:43:19.794998+0000","last_became_active":"2024-03-23T15:42:16.626457+0000","last_became_peered":"2024-03-23T15:42:16.626457+0000","last_unstale":"2024-03-23T15:43:19.794998+0000","last_undegraded":"2024-03-23T15:43:19.794998+0000","last_fullsized":"2024-03-23T15:43:19.794998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:16:29.776204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829447+0000","last_change":"2024-03-23T15:41:18.648500+0000","last_active":"2024-03-23T15:43:19.829447+0000","last_peered":"2024-03-23T15:43:19.829447+0000","last_clean":"2024-03-23T15:43:19.829447+0000","last_became_active":"2024-03-23T15:41:18.648139+0000","last_became_peered":"2024-03-23T15:41:18.648139+0000","last_unstale":"2024-03-23T15:43:19.829447+0000","last_undegraded":"2024-03-23T15:43:19.829447+0000","last_fullsized":"2024-03-23T15:43:19.829447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:38.071057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":152,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:20.032863+0000","last_change":"2024-03-23T15:41:17.658471+0000","last_active":"2024-03-23T15:43:20.032863+0000","last_peered":"2024-03-23T15:43:20.032863+0000","last_clean":"2024-03-23T15:43:20.032863+0000","last_became_active":"2024-03-23T15:41:17.658142+0000","last_became_peered":"2024-03-23T15:41:17.658142+0000","last_unstale":"2024-03-23T15:43:20.032863+0000","last_undegraded":"2024-03-23T15:43:20.032863+0000","last_fullsized":"2024-03-23T15:43:20.032863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:47:23.224790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855346+0000","last_change":"2024-03-23T15:41:18.646046+0000","last_active":"2024-03-23T15:43:19.855346+0000","last_peered":"2024-03-23T15:43:19.855346+0000","last_clean":"2024-03-23T15:43:19.855346+0000","last_became_active":"2024-03-23T15:41:18.645727+0000","last_became_peered":"2024-03-23T15:41:18.645727+0000","last_unstale":"2024-03-23T15:43:19.855346+0000","last_undegraded":"2024-03-23T15:43:19.855346+0000","last_fullsized":"2024-03-23T15:43:19.855346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:56:55.552079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794899+0000","last_change":"2024-03-23T15:41:17.657285+0000","last_active":"2024-03-23T15:43:19.794899+0000","last_peered":"2024-03-23T15:43:19.794899+0000","last_clean":"2024-03-23T15:43:19.794899+0000","last_became_active":"2024-03-23T15:41:17.656942+0000","last_became_peered":"2024-03-23T15:41:17.656942+0000","last_unstale":"2024-03-23T15:43:19.794899+0000","last_undegraded":"2024-03-23T15:43:19.794899+0000","last_fullsized":"2024-03-23T15:43:19.794899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:18.107467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829486+0000","last_change":"2024-03-23T15:41:18.655175+0000","last_active":"2024-03-23T15:43:19.829486+0000","last_peered":"2024-03-23T15:43:19.829486+0000","last_clean":"2024-03-23T15:43:19.829486+0000","last_became_active":"2024-03-23T15:41:18.654846+0000","last_became_peered":"2024-03-23T15:41:18.654846+0000","last_unstale":"2024-03-23T15:43:19.829486+0000","last_undegraded":"2024-03-23T15:43:19.829486+0000","last_fullsized":"2024-03-23T15:43:19.829486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:43:10.008538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828140+0000","last_change":"2024-03-23T15:41:17.657391+0000","last_active":"2024-03-23T15:43:19.828140+0000","last_peered":"2024-03-23T15:43:19.828140+0000","last_clean":"2024-03-23T15:43:19.828140+0000","last_became_active":"2024-03-23T15:41:17.657081+0000","last_became_peered":"2024-03-23T15:41:17.657081+0000","last_unstale":"2024-03-23T15:43:19.828140+0000","last_undegraded":"2024-03-23T15:43:19.828140+0000","last_fullsized":"2024-03-23T15:43:19.828140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:31:08.744477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902586+0000","last_change":"2024-03-23T15:41:18.656268+0000","last_active":"2024-03-23T15:43:19.902586+0000","last_peered":"2024-03-23T15:43:19.902586+0000","last_clean":"2024-03-23T15:43:19.902586+0000","last_became_active":"2024-03-23T15:41:18.655910+0000","last_became_peered":"2024-03-23T15:41:18.655910+0000","last_unstale":"2024-03-23T15:43:19.902586+0000","last_undegraded":"2024-03-23T15:43:19.902586+0000","last_fullsized":"2024-03-23T15:43:19.902586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:38:57.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829296+0000","last_change":"2024-03-23T15:41:17.656555+0000","last_active":"2024-03-23T15:43:19.829296+0000","last_peered":"2024-03-23T15:43:19.829296+0000","last_clean":"2024-03-23T15:43:19.829296+0000","last_became_active":"2024-03-23T15:41:17.655732+0000","last_became_peered":"2024-03-23T15:41:17.655732+0000","last_unstale":"2024-03-23T15:43:19.829296+0000","last_undegraded":"2024-03-23T15:43:19.829296+0000","last_fullsized":"2024-03-23T15:43:19.829296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:53.839841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793401+0000","last_change":"2024-03-23T15:41:18.647334+0000","last_active":"2024-03-23T15:43:19.793401+0000","last_peered":"2024-03-23T15:43:19.793401+0000","last_clean":"2024-03-23T15:43:19.793401+0000","last_became_active":"2024-03-23T15:41:18.646679+0000","last_became_peered":"2024-03-23T15:41:18.646679+0000","last_unstale":"2024-03-23T15:43:19.793401+0000","last_undegraded":"2024-03-23T15:43:19.793401+0000","last_fullsized":"2024-03-23T15:43:19.793401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:26:39.710759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.827515+0000","last_change":"2024-03-23T15:41:17.655400+0000","last_active":"2024-03-23T15:43:19.827515+0000","last_peered":"2024-03-23T15:43:19.827515+0000","last_clean":"2024-03-23T15:43:19.827515+0000","last_became_active":"2024-03-23T15:41:17.655034+0000","last_became_peered":"2024-03-23T15:41:17.655034+0000","last_unstale":"2024-03-23T15:43:19.827515+0000","last_undegraded":"2024-03-23T15:43:19.827515+0000","last_fullsized":"2024-03-23T15:43:19.827515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:06:30.973048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855488+0000","last_change":"2024-03-23T15:41:18.660248+0000","last_active":"2024-03-23T15:43:19.855488+0000","last_peered":"2024-03-23T15:43:19.855488+0000","last_clean":"2024-03-23T15:43:19.855488+0000","last_became_active":"2024-03-23T15:41:18.659728+0000","last_became_peered":"2024-03-23T15:41:18.659728+0000","last_unstale":"2024-03-23T15:43:19.855488+0000","last_undegraded":"2024-03-23T15:43:19.855488+0000","last_fullsized":"2024-03-23T15:43:19.855488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:40:03.681499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855657+0000","last_change":"2024-03-23T15:41:17.659784+0000","last_active":"2024-03-23T15:43:19.855657+0000","last_peered":"2024-03-23T15:43:19.855657+0000","last_clean":"2024-03-23T15:43:19.855657+0000","last_became_active":"2024-03-23T15:41:17.659422+0000","last_became_peered":"2024-03-23T15:41:17.659422+0000","last_unstale":"2024-03-23T15:43:19.855657+0000","last_undegraded":"2024-03-23T15:43:19.855657+0000","last_fullsized":"2024-03-23T15:43:19.855657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:32:48.607487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855290+0000","last_change":"2024-03-23T15:41:18.651777+0000","last_active":"2024-03-23T15:43:19.855290+0000","last_peered":"2024-03-23T15:43:19.855290+0000","last_clean":"2024-03-23T15:43:19.855290+0000","last_became_active":"2024-03-23T15:41:18.651457+0000","last_became_peered":"2024-03-23T15:41:18.651457+0000","last_unstale":"2024-03-23T15:43:19.855290+0000","last_undegraded":"2024-03-23T15:43:19.855290+0000","last_fullsized":"2024-03-23T15:43:19.855290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:25:28.831493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829549+0000","last_change":"2024-03-23T15:41:17.656357+0000","last_active":"2024-03-23T15:43:19.829549+0000","last_peered":"2024-03-23T15:43:19.829549+0000","last_clean":"2024-03-23T15:43:19.829549+0000","last_became_active":"2024-03-23T15:41:17.655977+0000","last_became_peered":"2024-03-23T15:41:17.655977+0000","last_unstale":"2024-03-23T15:43:19.829549+0000","last_undegraded":"2024-03-23T15:43:19.829549+0000","last_fullsized":"2024-03-23T15:43:19.829549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:07:45.248954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855083+0000","last_change":"2024-03-23T15:42:16.626757+0000","last_active":"2024-03-23T15:43:19.855083+0000","last_peered":"2024-03-23T15:43:19.855083+0000","last_clean":"2024-03-23T15:43:19.855083+0000","last_became_active":"2024-03-23T15:42:16.626409+0000","last_became_peered":"2024-03-23T15:42:16.626409+0000","last_unstale":"2024-03-23T15:43:19.855083+0000","last_undegraded":"2024-03-23T15:43:19.855083+0000","last_fullsized":"2024-03-23T15:43:19.855083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:17:46.969160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":301,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828033+0000","last_change":"2024-03-23T15:41:17.659838+0000","last_active":"2024-03-23T15:43:19.828033+0000","last_peered":"2024-03-23T15:43:19.828033+0000","last_clean":"2024-03-23T15:43:19.828033+0000","last_became_active":"2024-03-23T15:41:17.659326+0000","last_became_peered":"2024-03-23T15:41:17.659326+0000","last_unstale":"2024-03-23T15:43:19.828033+0000","last_undegraded":"2024-03-23T15:43:19.828033+0000","last_fullsized":"2024-03-23T15:43:19.828033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:07.651340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":246,"num_read_kb":183,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855649+0000","last_change":"2024-03-23T15:41:18.656629+0000","last_active":"2024-03-23T15:43:19.855649+0000","last_peered":"2024-03-23T15:43:19.855649+0000","last_clean":"2024-03-23T15:43:19.855649+0000","last_became_active":"2024-03-23T15:41:18.656196+0000","last_became_peered":"2024-03-23T15:41:18.656196+0000","last_unstale":"2024-03-23T15:43:19.855649+0000","last_undegraded":"2024-03-23T15:43:19.855649+0000","last_fullsized":"2024-03-23T15:43:19.855649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:22:30.013411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855049+0000","last_change":"2024-03-23T15:41:17.663999+0000","last_active":"2024-03-23T15:43:19.855049+0000","last_peered":"2024-03-23T15:43:19.855049+0000","last_clean":"2024-03-23T15:43:19.855049+0000","last_became_active":"2024-03-23T15:41:17.663039+0000","last_became_peered":"2024-03-23T15:41:17.663039+0000","last_unstale":"2024-03-23T15:43:19.855049+0000","last_undegraded":"2024-03-23T15:43:19.855049+0000","last_fullsized":"2024-03-23T15:43:19.855049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:16:00.271821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828098+0000","last_change":"2024-03-23T15:41:18.646822+0000","last_active":"2024-03-23T15:43:19.828098+0000","last_peered":"2024-03-23T15:43:19.828098+0000","last_clean":"2024-03-23T15:43:19.828098+0000","last_became_active":"2024-03-23T15:41:18.646476+0000","last_became_peered":"2024-03-23T15:41:18.646476+0000","last_unstale":"2024-03-23T15:43:19.828098+0000","last_undegraded":"2024-03-23T15:43:19.828098+0000","last_fullsized":"2024-03-23T15:43:19.828098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:20:18.352065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829326+0000","last_change":"2024-03-23T15:41:17.663507+0000","last_active":"2024-03-23T15:43:19.829326+0000","last_peered":"2024-03-23T15:43:19.829326+0000","last_clean":"2024-03-23T15:43:19.829326+0000","last_became_active":"2024-03-23T15:41:17.663122+0000","last_became_peered":"2024-03-23T15:41:17.663122+0000","last_unstale":"2024-03-23T15:43:19.829326+0000","last_undegraded":"2024-03-23T15:43:19.829326+0000","last_fullsized":"2024-03-23T15:43:19.829326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:20:32.202576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855780+0000","last_change":"2024-03-23T15:41:18.650110+0000","last_active":"2024-03-23T15:43:19.855780+0000","last_peered":"2024-03-23T15:43:19.855780+0000","last_clean":"2024-03-23T15:43:19.855780+0000","last_became_active":"2024-03-23T15:41:18.649737+0000","last_became_peered":"2024-03-23T15:41:18.649737+0000","last_unstale":"2024-03-23T15:43:19.855780+0000","last_undegraded":"2024-03-23T15:43:19.855780+0000","last_fullsized":"2024-03-23T15:43:19.855780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:06:31.610961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830442+0000","last_change":"2024-03-23T15:41:17.653930+0000","last_active":"2024-03-23T15:43:19.830442+0000","last_peered":"2024-03-23T15:43:19.830442+0000","last_clean":"2024-03-23T15:43:19.830442+0000","last_became_active":"2024-03-23T15:41:17.652677+0000","last_became_peered":"2024-03-23T15:41:17.652677+0000","last_unstale":"2024-03-23T15:43:19.830442+0000","last_undegraded":"2024-03-23T15:43:19.830442+0000","last_fullsized":"2024-03-23T15:43:19.830442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:51:47.167052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855179+0000","last_change":"2024-03-23T15:42:16.843231+0000","last_active":"2024-03-23T15:43:19.855179+0000","last_peered":"2024-03-23T15:43:19.855179+0000","last_clean":"2024-03-23T15:43:19.855179+0000","last_became_active":"2024-03-23T15:42:16.842785+0000","last_became_peered":"2024-03-23T15:42:16.842785+0000","last_unstale":"2024-03-23T15:43:19.855179+0000","last_undegraded":"2024-03-23T15:43:19.855179+0000","last_fullsized":"2024-03-23T15:43:19.855179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:24:07.377828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854899+0000","last_change":"2024-03-23T15:41:17.662091+0000","last_active":"2024-03-23T15:43:19.854899+0000","last_peered":"2024-03-23T15:43:19.854899+0000","last_clean":"2024-03-23T15:43:19.854899+0000","last_became_active":"2024-03-23T15:41:17.661747+0000","last_became_peered":"2024-03-23T15:41:17.661747+0000","last_unstale":"2024-03-23T15:43:19.854899+0000","last_undegraded":"2024-03-23T15:43:19.854899+0000","last_fullsized":"2024-03-23T15:43:19.854899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:08.661262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829293+0000","last_change":"2024-03-23T15:42:16.627776+0000","last_active":"2024-03-23T15:43:19.829293+0000","last_peered":"2024-03-23T15:43:19.829293+0000","last_clean":"2024-03-23T15:43:19.829293+0000","last_became_active":"2024-03-23T15:42:16.627331+0000","last_became_peered":"2024-03-23T15:42:16.627331+0000","last_unstale":"2024-03-23T15:43:19.829293+0000","last_undegraded":"2024-03-23T15:43:19.829293+0000","last_fullsized":"2024-03-23T15:43:19.829293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:02:57.288827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855572+0000","last_change":"2024-03-23T15:41:17.660898+0000","last_active":"2024-03-23T15:43:19.855572+0000","last_peered":"2024-03-23T15:43:19.855572+0000","last_clean":"2024-03-23T15:43:19.855572+0000","last_became_active":"2024-03-23T15:41:17.660515+0000","last_became_peered":"2024-03-23T15:41:17.660515+0000","last_unstale":"2024-03-23T15:43:19.855572+0000","last_undegraded":"2024-03-23T15:43:19.855572+0000","last_fullsized":"2024-03-23T15:43:19.855572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:06:58.911321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795271+0000","last_change":"2024-03-23T15:41:18.653582+0000","last_active":"2024-03-23T15:43:19.795271+0000","last_peered":"2024-03-23T15:43:19.795271+0000","last_clean":"2024-03-23T15:43:19.795271+0000","last_became_active":"2024-03-23T15:41:18.653184+0000","last_became_peered":"2024-03-23T15:41:18.653184+0000","last_unstale":"2024-03-23T15:43:19.795271+0000","last_undegraded":"2024-03-23T15:43:19.795271+0000","last_fullsized":"2024-03-23T15:43:19.795271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:59:16.726981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.793476+0000","last_change":"2024-03-23T15:41:17.651886+0000","last_active":"2024-03-23T15:43:19.793476+0000","last_peered":"2024-03-23T15:43:19.793476+0000","last_clean":"2024-03-23T15:43:19.793476+0000","last_became_active":"2024-03-23T15:41:17.651472+0000","last_became_peered":"2024-03-23T15:41:17.651472+0000","last_unstale":"2024-03-23T15:43:19.793476+0000","last_undegraded":"2024-03-23T15:43:19.793476+0000","last_fullsized":"2024-03-23T15:43:19.793476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:56:37.098333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795196+0000","last_change":"2024-03-23T15:41:18.644205+0000","last_active":"2024-03-23T15:43:19.795196+0000","last_peered":"2024-03-23T15:43:19.795196+0000","last_clean":"2024-03-23T15:43:19.795196+0000","last_became_active":"2024-03-23T15:41:18.643850+0000","last_became_peered":"2024-03-23T15:41:18.643850+0000","last_unstale":"2024-03-23T15:43:19.795196+0000","last_undegraded":"2024-03-23T15:43:19.795196+0000","last_fullsized":"2024-03-23T15:43:19.795196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:48:10.652113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855322+0000","last_change":"2024-03-23T15:41:17.661051+0000","last_active":"2024-03-23T15:43:19.855322+0000","last_peered":"2024-03-23T15:43:19.855322+0000","last_clean":"2024-03-23T15:43:19.855322+0000","last_became_active":"2024-03-23T15:41:17.659673+0000","last_became_peered":"2024-03-23T15:41:17.659673+0000","last_unstale":"2024-03-23T15:43:19.855322+0000","last_undegraded":"2024-03-23T15:43:19.855322+0000","last_fullsized":"2024-03-23T15:43:19.855322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:04:43.028104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795266+0000","last_change":"2024-03-23T15:42:16.843248+0000","last_active":"2024-03-23T15:43:19.795266+0000","last_peered":"2024-03-23T15:43:19.795266+0000","last_clean":"2024-03-23T15:43:19.795266+0000","last_became_active":"2024-03-23T15:42:16.842800+0000","last_became_peered":"2024-03-23T15:42:16.842800+0000","last_unstale":"2024-03-23T15:43:19.795266+0000","last_undegraded":"2024-03-23T15:43:19.795266+0000","last_fullsized":"2024-03-23T15:43:19.795266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.353368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.829351+0000","last_change":"2024-03-23T15:41:18.653560+0000","last_active":"2024-03-23T15:43:19.829351+0000","last_peered":"2024-03-23T15:43:19.829351+0000","last_clean":"2024-03-23T15:43:19.829351+0000","last_became_active":"2024-03-23T15:41:18.653249+0000","last_became_peered":"2024-03-23T15:41:18.653249+0000","last_unstale":"2024-03-23T15:43:19.829351+0000","last_undegraded":"2024-03-23T15:43:19.829351+0000","last_fullsized":"2024-03-23T15:43:19.829351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:34:22.514453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":144,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828395+0000","last_change":"2024-03-23T15:41:17.656318+0000","last_active":"2024-03-23T15:43:19.828395+0000","last_peered":"2024-03-23T15:43:19.828395+0000","last_clean":"2024-03-23T15:43:19.828395+0000","last_became_active":"2024-03-23T15:41:17.655958+0000","last_became_peered":"2024-03-23T15:41:17.655958+0000","last_unstale":"2024-03-23T15:43:19.828395+0000","last_undegraded":"2024-03-23T15:43:19.828395+0000","last_fullsized":"2024-03-23T15:43:19.828395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:16:12.333504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903433+0000","last_change":"2024-03-23T15:41:18.657616+0000","last_active":"2024-03-23T15:43:19.903433+0000","last_peered":"2024-03-23T15:43:19.903433+0000","last_clean":"2024-03-23T15:43:19.903433+0000","last_became_active":"2024-03-23T15:41:18.657271+0000","last_became_peered":"2024-03-23T15:41:18.657271+0000","last_unstale":"2024-03-23T15:43:19.903433+0000","last_undegraded":"2024-03-23T15:43:19.903433+0000","last_fullsized":"2024-03-23T15:43:19.903433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:02.241087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830384+0000","last_change":"2024-03-23T15:41:17.654358+0000","last_active":"2024-03-23T15:43:19.830384+0000","last_peered":"2024-03-23T15:43:19.830384+0000","last_clean":"2024-03-23T15:43:19.830384+0000","last_became_active":"2024-03-23T15:41:17.653558+0000","last_became_peered":"2024-03-23T15:41:17.653558+0000","last_unstale":"2024-03-23T15:43:19.830384+0000","last_undegraded":"2024-03-23T15:43:19.830384+0000","last_fullsized":"2024-03-23T15:43:19.830384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:25:32.377883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855332+0000","last_change":"2024-03-23T15:40:05.871801+0000","last_active":"2024-03-23T15:43:19.855332+0000","last_peered":"2024-03-23T15:43:19.855332+0000","last_clean":"2024-03-23T15:43:19.855332+0000","last_became_active":"2024-03-23T15:40:05.871417+0000","last_became_peered":"2024-03-23T15:40:05.871417+0000","last_unstale":"2024-03-23T15:43:19.855332+0000","last_undegraded":"2024-03-23T15:43:19.855332+0000","last_fullsized":"2024-03-23T15:43:19.855332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:36:39.745753+0000","last_clean_scrub_stamp":"2024-03-23T15:36:39.745753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:45:55.235176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.827993+0000","last_change":"2024-03-23T15:41:18.652251+0000","last_active":"2024-03-23T15:43:19.827993+0000","last_peered":"2024-03-23T15:43:19.827993+0000","last_clean":"2024-03-23T15:43:19.827993+0000","last_became_active":"2024-03-23T15:41:18.651876+0000","last_became_peered":"2024-03-23T15:41:18.651876+0000","last_unstale":"2024-03-23T15:43:19.827993+0000","last_undegraded":"2024-03-23T15:43:19.827993+0000","last_fullsized":"2024-03-23T15:43:19.827993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:24:45.235243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855564+0000","last_change":"2024-03-23T15:42:16.692783+0000","last_active":"2024-03-23T15:43:19.855564+0000","last_peered":"2024-03-23T15:43:19.855564+0000","last_clean":"2024-03-23T15:43:19.855564+0000","last_became_active":"2024-03-23T15:42:16.690169+0000","last_became_peered":"2024-03-23T15:42:16.690169+0000","last_unstale":"2024-03-23T15:43:19.855564+0000","last_undegraded":"2024-03-23T15:43:19.855564+0000","last_fullsized":"2024-03-23T15:43:19.855564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:20.679040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855712+0000","last_change":"2024-03-23T15:41:17.660703+0000","last_active":"2024-03-23T15:43:19.855712+0000","last_peered":"2024-03-23T15:43:19.855712+0000","last_clean":"2024-03-23T15:43:19.855712+0000","last_became_active":"2024-03-23T15:41:17.660343+0000","last_became_peered":"2024-03-23T15:41:17.660343+0000","last_unstale":"2024-03-23T15:43:19.855712+0000","last_undegraded":"2024-03-23T15:43:19.855712+0000","last_fullsized":"2024-03-23T15:43:19.855712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:14:03.063397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828358+0000","last_change":"2024-03-23T15:41:18.644624+0000","last_active":"2024-03-23T15:43:19.828358+0000","last_peered":"2024-03-23T15:43:19.828358+0000","last_clean":"2024-03-23T15:43:19.828358+0000","last_became_active":"2024-03-23T15:41:18.644275+0000","last_became_peered":"2024-03-23T15:41:18.644275+0000","last_unstale":"2024-03-23T15:43:19.828358+0000","last_undegraded":"2024-03-23T15:43:19.828358+0000","last_fullsized":"2024-03-23T15:43:19.828358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:57:25.901935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'45","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.999901+0000","last_change":"2024-03-23T15:41:17.650068+0000","last_active":"2024-03-23T15:43:19.999901+0000","last_peered":"2024-03-23T15:43:19.999901+0000","last_clean":"2024-03-23T15:43:19.999901+0000","last_became_active":"2024-03-23T15:41:17.649331+0000","last_became_peered":"2024-03-23T15:41:17.649331+0000","last_unstale":"2024-03-23T15:43:19.999901+0000","last_undegraded":"2024-03-23T15:43:19.999901+0000","last_fullsized":"2024-03-23T15:43:19.999901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:01:47.578645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903378+0000","last_change":"2024-03-23T15:41:18.656413+0000","last_active":"2024-03-23T15:43:19.903378+0000","last_peered":"2024-03-23T15:43:19.903378+0000","last_clean":"2024-03-23T15:43:19.903378+0000","last_became_active":"2024-03-23T15:41:18.656043+0000","last_became_peered":"2024-03-23T15:41:18.656043+0000","last_unstale":"2024-03-23T15:43:19.903378+0000","last_undegraded":"2024-03-23T15:43:19.903378+0000","last_fullsized":"2024-03-23T15:43:19.903378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:49:25.453420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.830801+0000","last_change":"2024-03-23T15:41:17.654492+0000","last_active":"2024-03-23T15:43:19.830801+0000","last_peered":"2024-03-23T15:43:19.830801+0000","last_clean":"2024-03-23T15:43:19.830801+0000","last_became_active":"2024-03-23T15:41:17.654132+0000","last_became_peered":"2024-03-23T15:41:17.654132+0000","last_unstale":"2024-03-23T15:43:19.830801+0000","last_undegraded":"2024-03-23T15:43:19.830801+0000","last_fullsized":"2024-03-23T15:43:19.830801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:39:58.210703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855201+0000","last_change":"2024-03-23T15:41:18.652659+0000","last_active":"2024-03-23T15:43:19.855201+0000","last_peered":"2024-03-23T15:43:19.855201+0000","last_clean":"2024-03-23T15:43:19.855201+0000","last_became_active":"2024-03-23T15:41:18.652246+0000","last_became_peered":"2024-03-23T15:41:18.652246+0000","last_unstale":"2024-03-23T15:43:19.855201+0000","last_undegraded":"2024-03-23T15:43:19.855201+0000","last_fullsized":"2024-03-23T15:43:19.855201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:50.868694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854944+0000","last_change":"2024-03-23T15:42:16.632166+0000","last_active":"2024-03-23T15:43:19.854944+0000","last_peered":"2024-03-23T15:43:19.854944+0000","last_clean":"2024-03-23T15:43:19.854944+0000","last_became_active":"2024-03-23T15:42:16.626587+0000","last_became_peered":"2024-03-23T15:42:16.626587+0000","last_unstale":"2024-03-23T15:43:19.854944+0000","last_undegraded":"2024-03-23T15:43:19.854944+0000","last_fullsized":"2024-03-23T15:43:19.854944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:22:21.514597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855068+0000","last_change":"2024-03-23T15:41:18.654049+0000","last_active":"2024-03-23T15:43:19.855068+0000","last_peered":"2024-03-23T15:43:19.855068+0000","last_clean":"2024-03-23T15:43:19.855068+0000","last_became_active":"2024-03-23T15:41:18.653765+0000","last_became_peered":"2024-03-23T15:41:18.653765+0000","last_unstale":"2024-03-23T15:43:19.855068+0000","last_undegraded":"2024-03-23T15:43:19.855068+0000","last_fullsized":"2024-03-23T15:43:19.855068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:12:11.467744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902818+0000","last_change":"2024-03-23T15:41:17.656679+0000","last_active":"2024-03-23T15:43:19.902818+0000","last_peered":"2024-03-23T15:43:19.902818+0000","last_clean":"2024-03-23T15:43:19.902818+0000","last_became_active":"2024-03-23T15:41:17.656368+0000","last_became_peered":"2024-03-23T15:41:17.656368+0000","last_unstale":"2024-03-23T15:43:19.902818+0000","last_undegraded":"2024-03-23T15:43:19.902818+0000","last_fullsized":"2024-03-23T15:43:19.902818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:35:14.176288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.794941+0000","last_change":"2024-03-23T15:41:18.653637+0000","last_active":"2024-03-23T15:43:19.794941+0000","last_peered":"2024-03-23T15:43:19.794941+0000","last_clean":"2024-03-23T15:43:19.794941+0000","last_became_active":"2024-03-23T15:41:18.652864+0000","last_became_peered":"2024-03-23T15:41:18.652864+0000","last_unstale":"2024-03-23T15:43:19.794941+0000","last_undegraded":"2024-03-23T15:43:19.794941+0000","last_fullsized":"2024-03-23T15:43:19.794941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:35:32.104901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855151+0000","last_change":"2024-03-23T15:41:17.664417+0000","last_active":"2024-03-23T15:43:19.855151+0000","last_peered":"2024-03-23T15:43:19.855151+0000","last_clean":"2024-03-23T15:43:19.855151+0000","last_became_active":"2024-03-23T15:41:17.663454+0000","last_became_peered":"2024-03-23T15:41:17.663454+0000","last_unstale":"2024-03-23T15:43:19.855151+0000","last_undegraded":"2024-03-23T15:43:19.855151+0000","last_fullsized":"2024-03-23T15:43:19.855151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T16:43:14.947767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903278+0000","last_change":"2024-03-23T15:41:18.647857+0000","last_active":"2024-03-23T15:43:19.903278+0000","last_peered":"2024-03-23T15:43:19.903278+0000","last_clean":"2024-03-23T15:43:19.903278+0000","last_became_active":"2024-03-23T15:41:18.646924+0000","last_became_peered":"2024-03-23T15:41:18.646924+0000","last_unstale":"2024-03-23T15:43:19.903278+0000","last_undegraded":"2024-03-23T15:43:19.903278+0000","last_fullsized":"2024-03-23T15:43:19.903278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:44:19.494901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855517+0000","last_change":"2024-03-23T15:41:17.661995+0000","last_active":"2024-03-23T15:43:19.855517+0000","last_peered":"2024-03-23T15:43:19.855517+0000","last_clean":"2024-03-23T15:43:19.855517+0000","last_became_active":"2024-03-23T15:41:17.661632+0000","last_became_peered":"2024-03-23T15:41:17.661632+0000","last_unstale":"2024-03-23T15:43:19.855517+0000","last_undegraded":"2024-03-23T15:43:19.855517+0000","last_fullsized":"2024-03-23T15:43:19.855517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:38:12.540038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855504+0000","last_change":"2024-03-23T15:41:18.651258+0000","last_active":"2024-03-23T15:43:19.855504+0000","last_peered":"2024-03-23T15:43:19.855504+0000","last_clean":"2024-03-23T15:43:19.855504+0000","last_became_active":"2024-03-23T15:41:18.650910+0000","last_became_peered":"2024-03-23T15:41:18.650910+0000","last_unstale":"2024-03-23T15:43:19.855504+0000","last_undegraded":"2024-03-23T15:43:19.855504+0000","last_fullsized":"2024-03-23T15:43:19.855504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:06:50.139644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855304+0000","last_change":"2024-03-23T15:41:17.654232+0000","last_active":"2024-03-23T15:43:19.855304+0000","last_peered":"2024-03-23T15:43:19.855304+0000","last_clean":"2024-03-23T15:43:19.855304+0000","last_became_active":"2024-03-23T15:41:17.653832+0000","last_became_peered":"2024-03-23T15:41:17.653832+0000","last_unstale":"2024-03-23T15:43:19.855304+0000","last_undegraded":"2024-03-23T15:43:19.855304+0000","last_fullsized":"2024-03-23T15:43:19.855304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:55:05.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855755+0000","last_change":"2024-03-23T15:41:18.664444+0000","last_active":"2024-03-23T15:43:19.855755+0000","last_peered":"2024-03-23T15:43:19.855755+0000","last_clean":"2024-03-23T15:43:19.855755+0000","last_became_active":"2024-03-23T15:41:18.664127+0000","last_became_peered":"2024-03-23T15:41:18.664127+0000","last_unstale":"2024-03-23T15:43:19.855755+0000","last_undegraded":"2024-03-23T15:43:19.855755+0000","last_fullsized":"2024-03-23T15:43:19.855755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:19:25.942230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.995159+0000","last_change":"2024-03-23T15:41:17.664834+0000","last_active":"2024-03-23T15:43:19.995159+0000","last_peered":"2024-03-23T15:43:19.995159+0000","last_clean":"2024-03-23T15:43:19.995159+0000","last_became_active":"2024-03-23T15:41:17.664286+0000","last_became_peered":"2024-03-23T15:41:17.664286+0000","last_unstale":"2024-03-23T15:43:19.995159+0000","last_undegraded":"2024-03-23T15:43:19.995159+0000","last_fullsized":"2024-03-23T15:43:19.995159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:50.595420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855495+0000","last_change":"2024-03-23T15:41:18.646341+0000","last_active":"2024-03-23T15:43:19.855495+0000","last_peered":"2024-03-23T15:43:19.855495+0000","last_clean":"2024-03-23T15:43:19.855495+0000","last_became_active":"2024-03-23T15:41:18.645995+0000","last_became_peered":"2024-03-23T15:41:18.645995+0000","last_unstale":"2024-03-23T15:43:19.855495+0000","last_undegraded":"2024-03-23T15:43:19.855495+0000","last_fullsized":"2024-03-23T15:43:19.855495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:15:56.408957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828254+0000","last_change":"2024-03-23T15:42:16.694643+0000","last_active":"2024-03-23T15:43:19.828254+0000","last_peered":"2024-03-23T15:43:19.828254+0000","last_clean":"2024-03-23T15:43:19.828254+0000","last_became_active":"2024-03-23T15:42:16.690063+0000","last_became_peered":"2024-03-23T15:42:16.690063+0000","last_unstale":"2024-03-23T15:43:19.828254+0000","last_undegraded":"2024-03-23T15:43:19.828254+0000","last_fullsized":"2024-03-23T15:43:19.828254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:16.621001+0000","last_clean_scrub_stamp":"2024-03-23T15:41:16.621001+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:59:54.873573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855942+0000","last_change":"2024-03-23T15:41:18.655250+0000","last_active":"2024-03-23T15:43:19.855942+0000","last_peered":"2024-03-23T15:43:19.855942+0000","last_clean":"2024-03-23T15:43:19.855942+0000","last_became_active":"2024-03-23T15:41:18.654888+0000","last_became_peered":"2024-03-23T15:41:18.654888+0000","last_unstale":"2024-03-23T15:43:19.855942+0000","last_undegraded":"2024-03-23T15:43:19.855942+0000","last_fullsized":"2024-03-23T15:43:19.855942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:26:36.730600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795028+0000","last_change":"2024-03-23T15:41:18.654309+0000","last_active":"2024-03-23T15:43:19.795028+0000","last_peered":"2024-03-23T15:43:19.795028+0000","last_clean":"2024-03-23T15:43:19.795028+0000","last_became_active":"2024-03-23T15:41:18.653319+0000","last_became_peered":"2024-03-23T15:41:18.653319+0000","last_unstale":"2024-03-23T15:43:19.795028+0000","last_undegraded":"2024-03-23T15:43:19.795028+0000","last_fullsized":"2024-03-23T15:43:19.795028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:02:05.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855072+0000","last_change":"2024-03-23T15:41:18.655179+0000","last_active":"2024-03-23T15:43:19.855072+0000","last_peered":"2024-03-23T15:43:19.855072+0000","last_clean":"2024-03-23T15:43:19.855072+0000","last_became_active":"2024-03-23T15:41:18.654830+0000","last_became_peered":"2024-03-23T15:41:18.654830+0000","last_unstale":"2024-03-23T15:43:19.855072+0000","last_undegraded":"2024-03-23T15:43:19.855072+0000","last_fullsized":"2024-03-23T15:43:19.855072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:29.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854909+0000","last_change":"2024-03-23T15:41:18.650734+0000","last_active":"2024-03-23T15:43:19.854909+0000","last_peered":"2024-03-23T15:43:19.854909+0000","last_clean":"2024-03-23T15:43:19.854909+0000","last_became_active":"2024-03-23T15:41:18.649993+0000","last_became_peered":"2024-03-23T15:41:18.649993+0000","last_unstale":"2024-03-23T15:43:19.854909+0000","last_undegraded":"2024-03-23T15:43:19.854909+0000","last_fullsized":"2024-03-23T15:43:19.854909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:22:06.463525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854958+0000","last_change":"2024-03-23T15:41:18.655741+0000","last_active":"2024-03-23T15:43:19.854958+0000","last_peered":"2024-03-23T15:43:19.854958+0000","last_clean":"2024-03-23T15:43:19.854958+0000","last_became_active":"2024-03-23T15:41:18.653387+0000","last_became_peered":"2024-03-23T15:41:18.653387+0000","last_unstale":"2024-03-23T15:43:19.854958+0000","last_undegraded":"2024-03-23T15:43:19.854958+0000","last_fullsized":"2024-03-23T15:43:19.854958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:20:26.758549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828382+0000","last_change":"2024-03-23T15:41:18.654739+0000","last_active":"2024-03-23T15:43:19.828382+0000","last_peered":"2024-03-23T15:43:19.828382+0000","last_clean":"2024-03-23T15:43:19.828382+0000","last_became_active":"2024-03-23T15:41:18.654151+0000","last_became_peered":"2024-03-23T15:41:18.654151+0000","last_unstale":"2024-03-23T15:43:19.828382+0000","last_undegraded":"2024-03-23T15:43:19.828382+0000","last_fullsized":"2024-03-23T15:43:19.828382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:07:01.358959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795157+0000","last_change":"2024-03-23T15:41:18.654081+0000","last_active":"2024-03-23T15:43:19.795157+0000","last_peered":"2024-03-23T15:43:19.795157+0000","last_clean":"2024-03-23T15:43:19.795157+0000","last_became_active":"2024-03-23T15:41:18.653696+0000","last_became_peered":"2024-03-23T15:41:18.653696+0000","last_unstale":"2024-03-23T15:43:19.795157+0000","last_undegraded":"2024-03-23T15:43:19.795157+0000","last_fullsized":"2024-03-23T15:43:19.795157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:23:19.741645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855549+0000","last_change":"2024-03-23T15:41:18.652356+0000","last_active":"2024-03-23T15:43:19.855549+0000","last_peered":"2024-03-23T15:43:19.855549+0000","last_clean":"2024-03-23T15:43:19.855549+0000","last_became_active":"2024-03-23T15:41:18.651976+0000","last_became_peered":"2024-03-23T15:41:18.651976+0000","last_unstale":"2024-03-23T15:43:19.855549+0000","last_undegraded":"2024-03-23T15:43:19.855549+0000","last_fullsized":"2024-03-23T15:43:19.855549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:43:38.972186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.903249+0000","last_change":"2024-03-23T15:41:18.653775+0000","last_active":"2024-03-23T15:43:19.903249+0000","last_peered":"2024-03-23T15:43:19.903249+0000","last_clean":"2024-03-23T15:43:19.903249+0000","last_became_active":"2024-03-23T15:41:18.653404+0000","last_became_peered":"2024-03-23T15:41:18.653404+0000","last_unstale":"2024-03-23T15:43:19.903249+0000","last_undegraded":"2024-03-23T15:43:19.903249+0000","last_fullsized":"2024-03-23T15:43:19.903249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T03:28:57.534731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.854866+0000","last_change":"2024-03-23T15:42:16.843115+0000","last_active":"2024-03-23T15:43:19.854866+0000","last_peered":"2024-03-23T15:43:19.854866+0000","last_clean":"2024-03-23T15:43:19.854866+0000","last_became_active":"2024-03-23T15:42:16.842743+0000","last_became_peered":"2024-03-23T15:42:16.842743+0000","last_unstale":"2024-03-23T15:43:19.854866+0000","last_undegraded":"2024-03-23T15:43:19.854866+0000","last_fullsized":"2024-03-23T15:43:19.854866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:36:11.377398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.902672+0000","last_change":"2024-03-23T15:41:18.652487+0000","last_active":"2024-03-23T15:43:19.902672+0000","last_peered":"2024-03-23T15:43:19.902672+0000","last_clean":"2024-03-23T15:43:19.902672+0000","last_became_active":"2024-03-23T15:41:18.652151+0000","last_became_peered":"2024-03-23T15:41:18.652151+0000","last_unstale":"2024-03-23T15:43:19.902672+0000","last_undegraded":"2024-03-23T15:43:19.902672+0000","last_fullsized":"2024-03-23T15:43:19.902672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:15:39.133585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.855151+0000","last_change":"2024-03-23T15:41:18.657265+0000","last_active":"2024-03-23T15:43:19.855151+0000","last_peered":"2024-03-23T15:43:19.855151+0000","last_clean":"2024-03-23T15:43:19.855151+0000","last_became_active":"2024-03-23T15:41:18.656970+0000","last_became_peered":"2024-03-23T15:41:18.656970+0000","last_unstale":"2024-03-23T15:43:19.855151+0000","last_undegraded":"2024-03-23T15:43:19.855151+0000","last_fullsized":"2024-03-23T15:43:19.855151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:19:22.808034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.828277+0000","last_change":"2024-03-23T15:41:18.655254+0000","last_active":"2024-03-23T15:43:19.828277+0000","last_peered":"2024-03-23T15:43:19.828277+0000","last_clean":"2024-03-23T15:43:19.828277+0000","last_became_active":"2024-03-23T15:41:18.654879+0000","last_became_peered":"2024-03-23T15:41:18.654879+0000","last_unstale":"2024-03-23T15:43:19.828277+0000","last_undegraded":"2024-03-23T15:43:19.828277+0000","last_fullsized":"2024-03-23T15:43:19.828277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:36:18.923723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-23T15:43:19.795101+0000","last_change":"2024-03-23T15:41:18.658652+0000","last_active":"2024-03-23T15:43:19.795101+0000","last_peered":"2024-03-23T15:43:19.795101+0000","last_clean":"2024-03-23T15:43:19.795101+0000","last_became_active":"2024-03-23T15:41:18.658261+0000","last_became_peered":"2024-03-23T15:41:18.658261+0000","last_unstale":"2024-03-23T15:43:19.795101+0000","last_undegraded":"2024-03-23T15:43:19.795101+0000","last_fullsized":"2024-03-23T15:43:19.795101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T15:41:17.632246+0000","last_clean_scrub_stamp":"2024-03-23T15:41:17.632246+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:00:45.145101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106812,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597498,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":830,"num_read_kb":518,"num_write":285,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1792494,"data_compressed":81513,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":465380,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1122304,"data_stored":1250064,"data_compressed":52357,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":858832,"data_compressed":27874,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":858752,"data_compressed":27874,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":1058780,"data_compressed":44332,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":958464,"data_stored":465005,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":532165,"data_compressed":10597,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":466276,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":528283,"data_compressed":10597,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":990132,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":465833,"data_compressed":8728,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1189673,"data_compressed":50488,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35604,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67367,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70031,"data_compressed":1869,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134390,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200936,"data_compressed":8025,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T15:43:24.879 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 -- 172.21.15.76:0/990215360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04381066b0 msgr2=0x7f0438106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 --2- 172.21.15.76:0/990215360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04381066b0 0x7f0438106a90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0428009900 tx=0x7f042802bcd0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 -- 172.21.15.76:0/990215360 shutdown_connections 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 --2- 172.21.15.76:0/990215360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0438100d40 0x7f043810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 --2- 172.21.15.76:0/990215360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438106fd0 0x7f0438100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 --2- 172.21.15.76:0/990215360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04381066b0 0x7f0438106a90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 -- 172.21.15.76:0/990215360 >> 172.21.15.76:0/990215360 conn(0x7f0438077710 msgr2=0x7f0438077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 -- 172.21.15.76:0/990215360 shutdown_connections 2024-03-23T15:43:25.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.014+0000 7f043d15e640 1 -- 172.21.15.76:0/990215360 wait complete. 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 Processor -- start 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 -- start start 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0438100d40 0x7f0438199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04381066b0 0x7f043819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438106fd0 0x7f043819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f043819ee10 con 0x7f0438100d40 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f043819ef80 con 0x7f0438106fd0 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.015+0000 7f043d15e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f043819f140 con 0x7f04381066b0 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0438100d40 0x7f0438199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0438100d40 0x7f0438199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49848/0 (socket says 172.21.15.76:49848) 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 -- 172.21.15.76:0/2914659753 learned_addr learned my addr 172.21.15.76:0/2914659753 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f043759e640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438106fd0 0x7f043819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f043659c640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04381066b0 0x7f043819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 -- 172.21.15.76:0/2914659753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04381066b0 msgr2=0x7f043819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04381066b0 0x7f043819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 -- 172.21.15.76:0/2914659753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438106fd0 msgr2=0x7f043819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438106fd0 0x7f043819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 -- 172.21.15.76:0/2914659753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0428009560 con 0x7f0438100d40 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f0436d9d640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0438100d40 0x7f0438199f50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f042802c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f041ffff640 1 -- 172.21.15.76:0/2914659753 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042802db20 con 0x7f0438100d40 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f041ffff640 1 -- 172.21.15.76:0/2914659753 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f042802dc80 con 0x7f0438100d40 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043819f3c0 con 0x7f0438100d40 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f041ffff640 1 -- 172.21.15.76:0/2914659753 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428003b20 con 0x7f0438100d40 2024-03-23T15:43:25.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.016+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381a6ca0 con 0x7f0438100d40 2024-03-23T15:43:25.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.017+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0438101d50 con 0x7f0438100d40 2024-03-23T15:43:25.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.018+0000 7f041ffff640 1 -- 172.21.15.76:0/2914659753 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f0428003c80 con 0x7f0438100d40 2024-03-23T15:43:25.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.019+0000 7f041ffff640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f04200a2350 0x7f04200a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.019+0000 7f043659c640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f04200a2350 0x7f04200a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.019+0000 7f041ffff640 1 -- 172.21.15.76:0/2914659753 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f04280f0730 con 0x7f0438100d40 2024-03-23T15:43:25.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.019+0000 7f043659c640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f04200a2350 0x7f04200a4810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04380ffde0 tx=0x7f042c005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.023+0000 7f041ffff640 1 -- 172.21.15.76:0/2914659753 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f04280b9280 con 0x7f0438100d40 2024-03-23T15:43:25.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.196+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0438068500 con 0x7f0438100d40 2024-03-23T15:43:25.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.197+0000 7f041ffff640 1 -- 172.21.15.76:0/2914659753 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23107 (secure 0 0 0) 0x7f04280bc940 con 0x7f0438100d40 2024-03-23T15:43:25.201 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:25.201 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":88,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:43:19.788338+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/4023925236":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6801/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6800/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/1911361196":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/4249446235":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/3432469273":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f04200a2350 msgr2=0x7f04200a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f04200a2350 0x7f04200a4810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04380ffde0 tx=0x7f042c005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0438100d40 msgr2=0x7f0438199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0438100d40 0x7f0438199f50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0428000c00 tx=0x7f042802c690 comp rx=0 tx=0).stop 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 shutdown_connections 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f04200a2350 0x7f04200a4810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438106fd0 0x7f043819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04381066b0 0x7f043819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 --2- 172.21.15.76:0/2914659753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0438100d40 0x7f0438199f50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 >> 172.21.15.76:0/2914659753 conn(0x7f0438077710 msgr2=0x7f04380688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:25.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 shutdown_connections 2024-03-23T15:43:25.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.200+0000 7f043d15e640 1 -- 172.21.15.76:0/2914659753 wait complete. 2024-03-23T15:43:25.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:43:25.214 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.316+0000 7f1981637640 1 -- 172.21.15.76:0/3377866682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 msgr2=0x7f197c102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.316+0000 7f1981637640 1 --2- 172.21.15.76:0/3377866682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c102d10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c02c180 comp rx=0 tx=0).stop 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.316+0000 7f1981637640 1 -- 172.21.15.76:0/3377866682 shutdown_connections 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.316+0000 7f1981637640 1 --2- 172.21.15.76:0/3377866682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c10bc70 0x7f197c10e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.316+0000 7f1981637640 1 --2- 172.21.15.76:0/3377866682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f197c103250 0x7f197c10b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.316+0000 7f1981637640 1 --2- 172.21.15.76:0/3377866682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c102d10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.316+0000 7f1981637640 1 -- 172.21.15.76:0/3377866682 >> 172.21.15.76:0/3377866682 conn(0x7f197c0777c0 msgr2=0x7f197c077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 -- 172.21.15.76:0/3377866682 shutdown_connections 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 -- 172.21.15.76:0/3377866682 wait complete. 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 Processor -- start 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 -- start start 2024-03-23T15:43:25.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c103250 0x7f197c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f197c10bc70 0x7f197c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f197c198180 con 0x7f197c102930 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1982f0 con 0x7f197c10bc70 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.317+0000 7f1981637640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f197c1984b0 con 0x7f197c103250 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c19d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49862/0 (socket says 172.21.15.76:49862) 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 -- 172.21.15.76:0/2781507991 learned_addr learned my addr 172.21.15.76:0/2781507991 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197a7fc640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c103250 0x7f197c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197b7fe640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f197c10bc70 0x7f197c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 -- 172.21.15.76:0/2781507991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c103250 msgr2=0x7f197c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c103250 0x7f197c19de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 -- 172.21.15.76:0/2781507991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f197c10bc70 msgr2=0x7f197c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f197c10bc70 0x7f197c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 -- 172.21.15.76:0/2781507991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c009900 con 0x7f197c102930 2024-03-23T15:43:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f197affd640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c19d930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f196c005e30 tx=0x7f196c034740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f1973fff640 1 -- 172.21.15.76:0/2781507991 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c003d80 con 0x7f197c102930 2024-03-23T15:43:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c198730 con 0x7f197c102930 2024-03-23T15:43:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f1973fff640 1 -- 172.21.15.76:0/2781507991 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f196c032c70 con 0x7f197c102930 2024-03-23T15:43:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c1a49d0 con 0x7f197c102930 2024-03-23T15:43:25.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.318+0000 7f1973fff640 1 -- 172.21.15.76:0/2781507991 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c004740 con 0x7f197c102930 2024-03-23T15:43:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.319+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f197c068500 con 0x7f197c102930 2024-03-23T15:43:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.320+0000 7f1973fff640 1 -- 172.21.15.76:0/2781507991 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f196c00a040 con 0x7f197c102930 2024-03-23T15:43:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.321+0000 7f1973fff640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f19680a2350 0x7f19680a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.321+0000 7f197a7fc640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f19680a2350 0x7f19680a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.321+0000 7f1973fff640 1 -- 172.21.15.76:0/2781507991 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f196c0f0d40 con 0x7f197c102930 2024-03-23T15:43:25.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.321+0000 7f197a7fc640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f19680a2350 0x7f19680a4810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f197c0fff90 tx=0x7f1964005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.325+0000 7f1973fff640 1 -- 172.21.15.76:0/2781507991 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f196c0b9890 con 0x7f197c102930 2024-03-23T15:43:25.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.502+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f197c106e40 con 0x7f197c102930 2024-03-23T15:43:25.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.502+0000 7f1973fff640 1 -- 172.21.15.76:0/2781507991 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f196c0bcf50 con 0x7f197c102930 2024-03-23T15:43:25.505 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:43:25.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f19680a2350 msgr2=0x7f19680a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f19680a2350 0x7f19680a4810 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f197c0fff90 tx=0x7f1964005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 msgr2=0x7f197c19d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c19d930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f196c005e30 tx=0x7f196c034740 comp rx=0 tx=0).stop 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 shutdown_connections 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f19680a2350 0x7f19680a4810 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f197c10bc70 0x7f197c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f197c103250 0x7f197c19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 --2- 172.21.15.76:0/2781507991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f197c102930 0x7f197c19d930 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 >> 172.21.15.76:0/2781507991 conn(0x7f197c0777c0 msgr2=0x7f197c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 shutdown_connections 2024-03-23T15:43:25.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.504+0000 7f1981637640 1 -- 172.21.15.76:0/2781507991 wait complete. 2024-03-23T15:43:25.517 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.517 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.518 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.519 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:25.519 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:43:25.519 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:43:25.519 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.519 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:25.519 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:25 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2914659753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:25.520 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:43:25.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:25.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:25 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2914659753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.622+0000 7fdd7942a640 1 -- 172.21.15.76:0/3327673279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd74068910 msgr2=0x7fdd74068d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.622+0000 7fdd7942a640 1 --2- 172.21.15.76:0/3327673279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd74068910 0x7fdd74068d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009cd0 tx=0x7fdd6802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 -- 172.21.15.76:0/3327673279 shutdown_connections 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 --2- 172.21.15.76:0/3327673279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd740692d0 0x7fdd7410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 --2- 172.21.15.76:0/3327673279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd74068910 0x7fdd74068d90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 --2- 172.21.15.76:0/3327673279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd74107760 0x7fdd74107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 -- 172.21.15.76:0/3327673279 >> 172.21.15.76:0/3327673279 conn(0x7fdd74077710 msgr2=0x7fdd74077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 -- 172.21.15.76:0/3327673279 shutdown_connections 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 -- 172.21.15.76:0/3327673279 wait complete. 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 Processor -- start 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 -- start start 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.623+0000 7fdd7942a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd74068910 0x7fdd7419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd7942a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd740692d0 0x7fdd7419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd7942a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd74107760 0x7fdd74197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd7942a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741982c0 con 0x7fdd740692d0 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd7942a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd74198430 con 0x7fdd74068910 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd7942a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741985f0 con 0x7fdd74107760 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd740692d0 0x7fdd7419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd740692d0 0x7fdd7419dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49868/0 (socket says 172.21.15.76:49868) 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 -- 172.21.15.76:0/4019620664 learned_addr learned my addr 172.21.15.76:0/4019620664 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd72ffd640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd74068910 0x7fdd7419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd737fe640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd74107760 0x7fdd74197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 -- 172.21.15.76:0/4019620664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd74107760 msgr2=0x7fdd74197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd74107760 0x7fdd74197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 -- 172.21.15.76:0/4019620664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd74068910 msgr2=0x7fdd7419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd74068910 0x7fdd7419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 -- 172.21.15.76:0/4019620664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c009490 con 0x7fdd740692d0 2024-03-23T15:43:25.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd727fc640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd740692d0 0x7fdd7419dfb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdd68004af0 tx=0x7fdd680325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd5bfff640 1 -- 172.21.15.76:0/4019620664 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd680328a0 con 0x7fdd740692d0 2024-03-23T15:43:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd68009900 con 0x7fdd740692d0 2024-03-23T15:43:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd5bfff640 1 -- 172.21.15.76:0/4019620664 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd68003e50 con 0x7fdd740692d0 2024-03-23T15:43:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd5bfff640 1 -- 172.21.15.76:0/4019620664 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68004770 con 0x7fdd740692d0 2024-03-23T15:43:25.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.624+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd741989d0 con 0x7fdd740692d0 2024-03-23T15:43:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.625+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd74068500 con 0x7fdd740692d0 2024-03-23T15:43:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.626+0000 7fdd5bfff640 1 -- 172.21.15.76:0/4019620664 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7fdd6803b040 con 0x7fdd740692d0 2024-03-23T15:43:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.627+0000 7fdd5bfff640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fdd5c0a2350 0x7fdd5c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.627+0000 7fdd72ffd640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fdd5c0a2350 0x7fdd5c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.627+0000 7fdd5bfff640 1 -- 172.21.15.76:0/4019620664 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fdd680f0b90 con 0x7fdd740692d0 2024-03-23T15:43:25.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.627+0000 7fdd72ffd640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fdd5c0a2350 0x7fdd5c0a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009d60 tx=0x7fdd6c005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-03-23T15:43:25.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: Updating smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: Updating smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: Updating smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:43:25.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:43:25.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:43:25.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:43:25.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:25.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:25 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2914659753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:25.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.631+0000 7fdd5bfff640 1 -- 172.21.15.76:0/4019620664 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fdd680b96e0 con 0x7fdd740692d0 2024-03-23T15:43:25.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.807+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd74102de0 con 0x7fdd740692d0 2024-03-23T15:43:25.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.808+0000 7fdd5bfff640 1 -- 172.21.15.76:0/4019620664 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fdd680bcda0 con 0x7fdd740692d0 2024-03-23T15:43:25.811 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:43:25.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fdd5c0a2350 msgr2=0x7fdd5c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fdd5c0a2350 0x7fdd5c0a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009d60 tx=0x7fdd6c005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:25.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd740692d0 msgr2=0x7fdd7419dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd740692d0 0x7fdd7419dfb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdd68004af0 tx=0x7fdd680325e0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 shutdown_connections 2024-03-23T15:43:25.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fdd5c0a2350 0x7fdd5c0a4810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd74107760 0x7fdd74197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd740692d0 0x7fdd7419dfb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 --2- 172.21.15.76:0/4019620664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd74068910 0x7fdd7419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 >> 172.21.15.76:0/4019620664 conn(0x7fdd74077710 msgr2=0x7fdd740fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:25.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 shutdown_connections 2024-03-23T15:43:25.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.810+0000 7fdd7942a640 1 -- 172.21.15.76:0/4019620664 wait complete. 2024-03-23T15:43:25.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:43:25.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 -- 172.21.15.76:0/3797131181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c1013f0 msgr2=0x7f2b3c1017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/3797131181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c1013f0 0x7f2b3c1017d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009930 tx=0x7f2b2c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 -- 172.21.15.76:0/3797131181 shutdown_connections 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/3797131181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b3c105940 0x7f2b3c10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/3797131181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c101da0 0x7f2b3c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/3797131181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c1013f0 0x7f2b3c1017d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 -- 172.21.15.76:0/3797131181 >> 172.21.15.76:0/3797131181 conn(0x7f2b3c0767a0 msgr2=0x7f2b3c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 -- 172.21.15.76:0/3797131181 shutdown_connections 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.927+0000 7f2b41bd0640 1 -- 172.21.15.76:0/3797131181 wait complete. 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 Processor -- start 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 -- start start 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b3c1013f0 0x7f2b3c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c105940 0x7f2b3c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c197b00 0x7f2b3c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c1984f0 con 0x7f2b3c197b00 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c198660 con 0x7f2b3c1013f0 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b41bd0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c198820 con 0x7f2b3c105940 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c197b00 0x7f2b3c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b3c1013f0 0x7f2b3c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c197b00 0x7f2b3c197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49898/0 (socket says 172.21.15.76:49898) 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 -- 172.21.15.76:0/840674259 learned_addr learned my addr 172.21.15.76:0/840674259 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3affd640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c105940 0x7f2b3c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 -- 172.21.15.76:0/840674259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c105940 msgr2=0x7f2b3c19de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c105940 0x7f2b3c19de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 -- 172.21.15.76:0/840674259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b3c1013f0 msgr2=0x7f2b3c19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b3c1013f0 0x7f2b3c19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.928+0000 7f2b3bfff640 1 -- 172.21.15.76:0/840674259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c009560 con 0x7f2b3c197b00 2024-03-23T15:43:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.929+0000 7f2b3b7fe640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b3c1013f0 0x7f2b3c19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.929+0000 7f2b3bfff640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c197b00 0x7f2b3c197fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400eb10 tx=0x7f2b3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.929+0000 7f2b38ff9640 1 -- 172.21.15.76:0/840674259 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34004b20 con 0x7f2b3c197b00 2024-03-23T15:43:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.929+0000 7f2b38ff9640 1 -- 172.21.15.76:0/840674259 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b34004c80 con 0x7f2b3c197b00 2024-03-23T15:43:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.929+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c1a4810 con 0x7f2b3c197b00 2024-03-23T15:43:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.929+0000 7f2b38ff9640 1 -- 172.21.15.76:0/840674259 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34010650 con 0x7f2b3c197b00 2024-03-23T15:43:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.929+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1a4cd0 con 0x7f2b3c197b00 2024-03-23T15:43:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.930+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c068950 con 0x7f2b3c197b00 2024-03-23T15:43:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.931+0000 7f2b38ff9640 1 -- 172.21.15.76:0/840674259 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f2b34005020 con 0x7f2b3c197b00 2024-03-23T15:43:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.931+0000 7f2b38ff9640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2b240a2350 0x7f2b240a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.931+0000 7f2b3b7fe640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2b240a2350 0x7f2b240a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.932+0000 7f2b38ff9640 1 -- 172.21.15.76:0/840674259 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f2b34014070 con 0x7f2b3c197b00 2024-03-23T15:43:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.932+0000 7f2b3b7fe640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2b240a2350 0x7f2b240a4810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005ab0 tx=0x7f2b2c005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:25.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:25.936+0000 7f2b38ff9640 1 -- 172.21.15.76:0/840674259 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2b340972e0 con 0x7f2b3c197b00 2024-03-23T15:43:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.110+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b3c109b30 con 0x7f2b3c197b00 2024-03-23T15:43:26.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.110+0000 7f2b38ff9640 1 -- 172.21.15.76:0/840674259 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f2b3409a9a0 con 0x7f2b3c197b00 2024-03-23T15:43:26.114 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:43:26.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2b240a2350 msgr2=0x7f2b240a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2b240a2350 0x7f2b240a4810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005ab0 tx=0x7f2b2c005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c197b00 msgr2=0x7f2b3c197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c197b00 0x7f2b3c197fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400eb10 tx=0x7f2b3400c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 shutdown_connections 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2b240a2350 0x7f2b240a4810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b3c197b00 0x7f2b3c197fb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c105940 0x7f2b3c19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 --2- 172.21.15.76:0/840674259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b3c1013f0 0x7f2b3c19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 >> 172.21.15.76:0/840674259 conn(0x7f2b3c0767a0 msgr2=0x7f2b3c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 shutdown_connections 2024-03-23T15:43:26.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.112+0000 7f2b41bd0640 1 -- 172.21.15.76:0/840674259 wait complete. 2024-03-23T15:43:26.126 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.224+0000 7f71a94c0640 1 -- 172.21.15.76:0/2164795295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a41066b0 msgr2=0x7f71a4106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.224+0000 7f71a94c0640 1 --2- 172.21.15.76:0/2164795295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a41066b0 0x7f71a4106a90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f719402c180 comp rx=0 tx=0).stop 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.224+0000 7f71a94c0640 1 -- 172.21.15.76:0/2164795295 shutdown_connections 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 --2- 172.21.15.76:0/2164795295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71a4100d40 0x7f71a410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 --2- 172.21.15.76:0/2164795295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a4106fd0 0x7f71a4100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 --2- 172.21.15.76:0/2164795295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a41066b0 0x7f71a4106a90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 -- 172.21.15.76:0/2164795295 >> 172.21.15.76:0/2164795295 conn(0x7f71a4077710 msgr2=0x7f71a4077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 -- 172.21.15.76:0/2164795295 shutdown_connections 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 -- 172.21.15.76:0/2164795295 wait complete. 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 Processor -- start 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 -- start start 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71a4100d40 0x7f71a4197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.225+0000 7f71a94c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a41066b0 0x7f71a41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a94c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a4106fd0 0x7f71a419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a94c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71a419cc50 con 0x7f71a4106fd0 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a94c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71a419cdc0 con 0x7f71a4100d40 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a94c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71a419cf80 con 0x7f71a41066b0 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a4106fd0 0x7f71a419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a4106fd0 0x7f71a419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49910/0 (socket says 172.21.15.76:49910) 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 -- 172.21.15.76:0/1832998324 learned_addr learned my addr 172.21.15.76:0/1832998324 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a2ffd640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71a4100d40 0x7f71a4197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a27fc640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a41066b0 0x7f71a41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 -- 172.21.15.76:0/1832998324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a41066b0 msgr2=0x7f71a41982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a41066b0 0x7f71a41982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 -- 172.21.15.76:0/1832998324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71a4100d40 msgr2=0x7f71a4197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71a4100d40 0x7f71a4197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 -- 172.21.15.76:0/1832998324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7194009900 con 0x7f71a4106fd0 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a37fe640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a4106fd0 0x7f71a419c650 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f719c00e970 tx=0x7f719c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f719bfff640 1 -- 172.21.15.76:0/1832998324 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c009720 con 0x7f71a4106fd0 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f719bfff640 1 -- 172.21.15.76:0/1832998324 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719c005500 con 0x7f71a4106fd0 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a419d260 con 0x7f71a4106fd0 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f719bfff640 1 -- 172.21.15.76:0/1832998324 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c010430 con 0x7f71a4106fd0 2024-03-23T15:43:26.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.226+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a41a4ae0 con 0x7f71a4106fd0 2024-03-23T15:43:26.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.227+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a4068500 con 0x7f71a4106fd0 2024-03-23T15:43:26.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.228+0000 7f719bfff640 1 -- 172.21.15.76:0/1832998324 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f719c005020 con 0x7f71a4106fd0 2024-03-23T15:43:26.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.229+0000 7f719bfff640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f71900a2350 0x7f71900a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.229+0000 7f71a2ffd640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f71900a2350 0x7f71900a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.229+0000 7f719bfff640 1 -- 172.21.15.76:0/1832998324 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f719c014070 con 0x7f71a4106fd0 2024-03-23T15:43:26.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.229+0000 7f71a2ffd640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f71900a2350 0x7f71900a4810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7194005cc0 tx=0x7f7194005c50 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.233+0000 7f719bfff640 1 -- 172.21.15.76:0/1832998324 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f719c0970e0 con 0x7f71a4106fd0 2024-03-23T15:43:26.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.445+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71a41990e0 con 0x7f71a4106fd0 2024-03-23T15:43:26.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.446+0000 7f719bfff640 1 -- 172.21.15.76:0/1832998324 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f719c09a7a0 con 0x7f71a4106fd0 2024-03-23T15:43:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:43:18.737936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24463,24449]},"id":1}]} 2024-03-23T15:43:26.449 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 22 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f71900a2350 msgr2=0x7f71900a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f71900a2350 0x7f71900a4810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7194005cc0 tx=0x7f7194005c50 comp rx=0 tx=0).stop 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a4106fd0 msgr2=0x7f71a419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a4106fd0 0x7f71a419c650 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f719c00e970 tx=0x7f719c00ed70 comp rx=0 tx=0).stop 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 shutdown_connections 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f71900a2350 0x7f71900a4810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71a4106fd0 0x7f71a419c650 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71a41066b0 0x7f71a41982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 --2- 172.21.15.76:0/1832998324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71a4100d40 0x7f71a4197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 >> 172.21.15.76:0/1832998324 conn(0x7f71a4077710 msgr2=0x7f71a4104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 shutdown_connections 2024-03-23T15:43:26.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.448+0000 7f71a94c0640 1 -- 172.21.15.76:0/1832998324 wait complete. 2024-03-23T15:43:26.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:26 smithi076 ceph-mon[32176]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:26.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:26 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2781507991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:26 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/4019620664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.461 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:26 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/840674259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:43:26.534 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:26 smithi183 ceph-mon[36296]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:26.535 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:26 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2781507991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.535 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:26 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/4019620664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.535 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:26 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/840674259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 -- 172.21.15.76:0/3305095719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 msgr2=0x7f914010d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3305095719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914010d450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9138009ca0 tx=0x7f913802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 -- 172.21.15.76:0/3305095719 shutdown_connections 2024-03-23T15:43:26.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3305095719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914010d450 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3305095719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91401005d0 0x7f9140100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3305095719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9140106690 0x7f9140106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 -- 172.21.15.76:0/3305095719 >> 172.21.15.76:0/3305095719 conn(0x7f9140076990 msgr2=0x7f9140076da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 -- 172.21.15.76:0/3305095719 shutdown_connections 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 -- 172.21.15.76:0/3305095719 wait complete. 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 Processor -- start 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.562+0000 7f9145e2a640 1 -- start start 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9145e2a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91401005d0 0x7f914019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9145e2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9145e2a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9140106690 0x7f9140197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9145e2a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f91401980f0 con 0x7f9140100f90 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9145e2a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9140198260 con 0x7f9140106690 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9145e2a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9140198420 con 0x7f91401005d0 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914019dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49932/0 (socket says 172.21.15.76:49932) 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 -- 172.21.15.76:0/3142516292 learned_addr learned my addr 172.21.15.76:0/3142516292 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f913f7fe640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91401005d0 0x7f914019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f913ffff640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9140106690 0x7f9140197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 -- 172.21.15.76:0/3142516292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91401005d0 msgr2=0x7f914019d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91401005d0 0x7f914019d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 -- 172.21.15.76:0/3142516292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9140106690 msgr2=0x7f9140197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9140106690 0x7f9140197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 -- 172.21.15.76:0/3142516292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9138009900 con 0x7f9140100f90 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.563+0000 7f9136ffd640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914019dde0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f912800e9f0 tx=0x7f912800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.564+0000 7f913d7fa640 1 -- 172.21.15.76:0/3142516292 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128009720 con 0x7f9140100f90 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.564+0000 7f913d7fa640 1 -- 172.21.15.76:0/3142516292 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f912800cd80 con 0x7f9140100f90 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.564+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9140198700 con 0x7f9140100f90 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.564+0000 7f913d7fa640 1 -- 172.21.15.76:0/3142516292 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128010690 con 0x7f9140100f90 2024-03-23T15:43:26.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.564+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a4940 con 0x7f9140100f90 2024-03-23T15:43:26.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:26 smithi192 ceph-mon[36653]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T15:43:26.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:26 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2781507991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:26 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/4019620664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:26 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/840674259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:43:26.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.565+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9140101cf0 con 0x7f9140100f90 2024-03-23T15:43:26.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.566+0000 7f913d7fa640 1 -- 172.21.15.76:0/3142516292 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f91280046c0 con 0x7f9140100f90 2024-03-23T15:43:26.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.566+0000 7f913d7fa640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f912c0a2350 0x7f912c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.566+0000 7f913f7fe640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f912c0a2350 0x7f912c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.566+0000 7f913d7fa640 1 -- 172.21.15.76:0/3142516292 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9128014070 con 0x7f9140100f90 2024-03-23T15:43:26.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.567+0000 7f913f7fe640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f912c0a2350 0x7f912c0a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f91401056e0 tx=0x7f9130005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.571+0000 7f913d7fa640 1 -- 172.21.15.76:0/3142516292 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9128097100 con 0x7f9140100f90 2024-03-23T15:43:26.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.752+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9140068950 con 0x7f9140100f90 2024-03-23T15:43:26.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.753+0000 7f913d7fa640 1 -- 172.21.15.76:0/3142516292 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23107 (secure 0 0 0) 0x7f912809a7c0 con 0x7f9140100f90 2024-03-23T15:43:26.757 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:26.757 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":88,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:43:19.788338+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/4023925236":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6801/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6800/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/1911361196":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/4249446235":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/3432469273":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f912c0a2350 msgr2=0x7f912c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f912c0a2350 0x7f912c0a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f91401056e0 tx=0x7f9130005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 msgr2=0x7f914019dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914019dde0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f912800e9f0 tx=0x7f912800edf0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 shutdown_connections 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f912c0a2350 0x7f912c0a4810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9140106690 0x7f9140197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9140100f90 0x7f914019dde0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 --2- 172.21.15.76:0/3142516292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91401005d0 0x7f914019d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 >> 172.21.15.76:0/3142516292 conn(0x7f9140076990 msgr2=0x7f91400ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 shutdown_connections 2024-03-23T15:43:26.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.755+0000 7f9145e2a640 1 -- 172.21.15.76:0/3142516292 wait complete. 2024-03-23T15:43:26.770 DEBUG:tasks.fwd_scrub:Yielding 2024-03-23T15:43:26.770 INFO:teuthology.run_tasks:Running task exec... 2024-03-23T15:43:26.779 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T15:43:26.779 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:43:26.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-23T15:43:26.780 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-23T15:43:26.780 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T15:43:26.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:26.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 -- 172.21.15.76:0/1966447408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41801077e0 msgr2=0x7f4180107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 --2- 172.21.15.76:0/1966447408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41801077e0 0x7f4180107bc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4168009930 tx=0x7f416802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 -- 172.21.15.76:0/1966447408 shutdown_connections 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 --2- 172.21.15.76:0/1966447408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 0x7f418010e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 --2- 172.21.15.76:0/1966447408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4180068d90 0x7f41800691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 --2- 172.21.15.76:0/1966447408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41801077e0 0x7f4180107bc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 -- 172.21.15.76:0/1966447408 >> 172.21.15.76:0/1966447408 conn(0x7f4180077460 msgr2=0x7f4180077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.883+0000 7f418528f640 1 -- 172.21.15.76:0/1966447408 shutdown_connections 2024-03-23T15:43:26.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 -- 172.21.15.76:0/1966447408 wait complete. 2024-03-23T15:43:26.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 Processor -- start 2024-03-23T15:43:26.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 -- 172.21.15.76:0/3349861173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44106660 msgr2=0x7f3d44106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/3349861173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44106660 0x7f3d44106a40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009900 tx=0x7f3d3802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:26.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 -- start start 2024-03-23T15:43:26.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 -- 172.21.15.76:0/3349861173 shutdown_connections 2024-03-23T15:43:26.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/3349861173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d44100f60 0x7f3d4410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/3349861173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 0x7f3d44100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/3349861173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44106660 0x7f3d44106a40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 -- 172.21.15.76:0/3349861173 >> 172.21.15.76:0/3349861173 conn(0x7f3d44077710 msgr2=0x7f3d44077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 -- 172.21.15.76:0/3349861173 shutdown_connections 2024-03-23T15:43:26.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f3d4b08f640 1 -- 172.21.15.76:0/3349861173 wait complete. 2024-03-23T15:43:26.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4180068d90 0x7f4180197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 0x7f41801983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f418019c750 0x7f418019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f418019d140 con 0x7f418019c750 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f418019d2b0 con 0x7f4180069730 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.884+0000 7f418528f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f418019d420 con 0x7f4180068d90 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f41767fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 0x7f41801983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f418019c750 0x7f418019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4180068d90 0x7f4180197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f41767fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 0x7f41801983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:34058/0 (socket says 172.21.15.76:34058) 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f41767fc640 1 -- 172.21.15.76:0/74328691 learned_addr learned my addr 172.21.15.76:0/74328691 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f418019c750 0x7f418019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:49968/0 (socket says 172.21.15.76:49968) 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 -- 172.21.15.76:0/74328691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4180068d90 msgr2=0x7f4180197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4180068d90 0x7f4180197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 -- 172.21.15.76:0/74328691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 msgr2=0x7f41801983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 0x7f41801983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 -- 172.21.15.76:0/74328691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4178009840 con 0x7f418019c750 2024-03-23T15:43:26.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f41767fc640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 0x7f41801983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:26.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 Processor -- start 2024-03-23T15:43:26.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 -- start start 2024-03-23T15:43:26.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417f7fe640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f418019c750 0x7f418019cc00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f417800ebc0 tx=0x7f417800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 0x7f3d44102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44100f60 0x7f3d44102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417cff9640 1 -- 172.21.15.76:0/74328691 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178003b70 con 0x7f418019c750 2024-03-23T15:43:26.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d44106660 0x7f3d44102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4168009560 con 0x7f418019c750 2024-03-23T15:43:26.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417cff9640 1 -- 172.21.15.76:0/74328691 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417800eec0 con 0x7f418019c750 2024-03-23T15:43:26.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3d441a60d0 con 0x7f3d44100f60 2024-03-23T15:43:26.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d441a6240 con 0x7f3d441005a0 2024-03-23T15:43:26.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4b08f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d441a6400 con 0x7f3d44106660 2024-03-23T15:43:26.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f417cff9640 1 -- 172.21.15.76:0/74328691 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178010670 con 0x7f418019c750 2024-03-23T15:43:26.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801a4d20 con 0x7f418019c750 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 0x7f3d44102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 0x7f3d44102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:34062/0 (socket says 172.21.15.76:34062) 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2695514943 learned_addr learned my addr 172.21.15.76:0/2695514943 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d4962c640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d44106660 0x7f3d44102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.885+0000 7f3d43fff640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44100f60 0x7f3d44102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2695514943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d44106660 msgr2=0x7f3d44102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d44106660 0x7f3d44102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44100f60 msgr2=0x7f3d44102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44100f60 0x7f3d44102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2695514943 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d30009490 con 0x7f3d441005a0 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 0x7f3d44102510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d38005940 tx=0x7f3d3802c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d41ffb640 1 -- 172.21.15.76:0/2695514943 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380042d0 con 0x7f3d441005a0 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d41ffb640 1 -- 172.21.15.76:0/2695514943 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d38004430 con 0x7f3d441005a0 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d38009560 con 0x7f3d441005a0 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d41ffb640 1 -- 172.21.15.76:0/2695514943 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38003c50 con 0x7f3d441005a0 2024-03-23T15:43:26.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.886+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d441a6a50 con 0x7f3d441005a0 2024-03-23T15:43:26.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.887+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418004f470 con 0x7f418019c750 2024-03-23T15:43:26.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.888+0000 7f417cff9640 1 -- 172.21.15.76:0/74328691 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f4178004b40 con 0x7f418019c750 2024-03-23T15:43:26.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.888+0000 7f417cff9640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f416c0a2470 0x7f416c0a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.888+0000 7f3d41ffb640 1 -- 172.21.15.76:0/2695514943 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f3d3803e450 con 0x7f3d441005a0 2024-03-23T15:43:26.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f417effd640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f416c0a2470 0x7f416c0a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f417cff9640 1 -- 172.21.15.76:0/74328691 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4178014070 con 0x7f418019c750 2024-03-23T15:43:26.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f417effd640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f416c0a2470 0x7f416c0a4930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4168009230 tx=0x7f4168005e80 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d41ffb640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3d340a23d0 0x7f3d340a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d4dc78740 1 Processor -- start 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d4dc78740 1 -- start start 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d43fff640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3d340a23d0 0x7f3d340a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d41ffb640 1 -- 172.21.15.76:0/2695514943 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3d380f02a0 con 0x7f3d441005a0 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f5760 0x5596451f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x5596451fe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.889+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x559645201050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559645163530 con 0x5596451f6080 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559645160a60 con 0x5596451fec60 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55964515c240 con 0x5596451f5760 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f5760 0x5596451f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4962c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x559645201050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f5760 0x5596451f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:38204/0 (socket says 172.21.15.76:38204) 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d48e2b640 1 -- 172.21.15.76:0/3185347241 learned_addr learned my addr 172.21.15.76:0/3185347241 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d43fff640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3d340a23d0 0x7f3d340a4890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d44103fa0 tx=0x7f3d30005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d43fff640 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x5596451fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4962c640 1 -- 172.21.15.76:0/3185347241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f5760 msgr2=0x5596451f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4962c640 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f5760 0x5596451f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4962c640 1 -- 172.21.15.76:0/3185347241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 msgr2=0x5596451fe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4962c640 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x5596451fe720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4962c640 1 -- 172.21.15.76:0/3185347241 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964515a8c0 con 0x5596451fec60 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f5760 0x5596451f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d4962c640 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x559645201050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009f00 tx=0x7f3d3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f51045022bb0b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d2effd640 1 -- 172.21.15.76:0/3185347241 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c004d20 con 0x5596451fec60 2024-03-23T15:43:26.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d2effd640 1 -- 172.21.15.76:0/3185347241 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3c004e80 con 0x5596451fec60 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.890+0000 7f3d2effd640 1 -- 172.21.15.76:0/3185347241 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c005280 con 0x5596451fec60 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3185347241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 msgr2=0x559645201050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x559645201050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009f00 tx=0x7f3d3c011810 comp rx=0 tx=0).stop 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3185347241 shutdown_connections 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x559645201050 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x5596451fe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3185347241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f5760 0x5596451f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3185347241 >> 172.21.15.76:0/3185347241 conn(0x5596451efe90 msgr2=0x5596451f0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3185347241 shutdown_connections 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3185347241 wait complete. 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.891+0000 7f3d4dc78740 1 -- start start 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 1 Processor -- start 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x5596451fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f6080 0x5596451fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x5596451fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559645100f80 con 0x5596451f5760 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559644e2f4b0 con 0x5596451fec60 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5596450e1ad0 con 0x5596451f6080 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4dc78740 10 client.0 _ll_register_callbacks cb 0x559645114b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x5596451fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d43fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f6080 0x5596451fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x5596451fbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50000/0 (socket says 172.21.15.76:50000) 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2205909411 learned_addr learned my addr 172.21.15.76:0/2205909411 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d4962c640 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x5596451fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2205909411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f6080 msgr2=0x5596451fc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f6080 0x5596451fc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 msgr2=0x5596451fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x5596451fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.892+0000 7f3d48e2b640 1 -- 172.21.15.76:0/2205909411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c009be0 con 0x5596451f5760 2024-03-23T15:43:26.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f417cff9640 1 -- 172.21.15.76:0/74328691 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f41780974a0 con 0x7f418019c750 2024-03-23T15:43:26.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x5596451fbcf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d380edff0 tx=0x7f3d380bd340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c07ac85c7823834 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d23fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:26.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d23fff640 1 -- 172.21.15.76:0/2205909411 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380d4070 con 0x5596451f5760 2024-03-23T15:43:26.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596451521f0 con 0x5596451f5760 2024-03-23T15:43:26.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d23fff640 1 -- 172.21.15.76:0/2205909411 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3d380c52f0 con 0x5596451f5760 2024-03-23T15:43:26.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d23fff640 1 -- 172.21.15.76:0/2205909411 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3d380bae50 con 0x5596451f5760 2024-03-23T15:43:26.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 10 client.15513 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5596451126d0 con 0x5596451f5760 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d23fff640 1 -- 172.21.15.76:0/2205909411 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d3809f020 con 0x5596451f5760 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d23fff640 1 -- 172.21.15.76:0/2205909411 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3d3803c070 con 0x5596451f5760 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 10 client.15513 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 20 client.15513 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645211030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 4 client.15513 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.893+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559644e5b860 con 0x55964520ec20 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.894+0000 7f3d43fff640 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645211030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.894+0000 7f3d23fff640 10 client.15513 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:26.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.894+0000 7f3d43fff640 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645211030 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6d9269d59c1198 server_cookie=2aaddf694f95fe6d in_seq=0 out_seq=0 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d23fff640 1 -- 172.21.15.76:0/2205909411 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559644e5b860 con 0x55964520ec20 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d23fff640 10 client.15513 handle_command_reply: tid=0 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 client.15513 shutdown 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 msgr2=0x5596451fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x5596451fbcf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d380edff0 tx=0x7f3d380bd340 comp rx=0 tx=0).stop 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 shutdown_connections 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645211030 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451fec60 0x5596451fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451f6080 0x5596451fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/2205909411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x5596451fbcf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d23fff640 0 client.15513 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 >> 172.21.15.76:0/2205909411 conn(0x5596451efe90 msgr2=0x5596451f1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 shutdown_connections 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 1 -- 172.21.15.76:0/2205909411 wait complete. 2024-03-23T15:43:26.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.895+0000 7f3d4dc78740 20 client.15513 trim_cache size 0 max 16384 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 Processor -- start 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 -- start start 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x55964520a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f6080 0x5596451fc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645216900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55964520b840 con 0x5596451f5760 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596451f0e90 con 0x5596451f6080 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5596451f4f30 con 0x5596451fec60 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x55964520a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d43fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f6080 0x5596451fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d48e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x55964520a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50002/0 (socket says 172.21.15.76:50002) 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d4962c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645216900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d48e2b640 1 -- 172.21.15.76:0/3550298283 learned_addr learned my addr 172.21.15.76:0/3550298283 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d43fff640 1 -- 172.21.15.76:0/3550298283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 msgr2=0x559645216900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d43fff640 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645216900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d43fff640 1 -- 172.21.15.76:0/3550298283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 msgr2=0x55964520a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d43fff640 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x55964520a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d43fff640 1 -- 172.21.15.76:0/3550298283 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596450f6c60 con 0x5596451f6080 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.913+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x55964520a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.914+0000 7f3d43fff640 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f6080 0x5596451fc590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5596451fd2c0 tx=0x7f3d30014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=594d49d7dd2242a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.914+0000 7f3d2e7fc640 1 -- 172.21.15.76:0/3550298283 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30021040 con 0x5596451f6080 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.914+0000 7f3d2e7fc640 1 -- 172.21.15.76:0/3550298283 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d30022040 con 0x5596451f6080 2024-03-23T15:43:26.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.914+0000 7f3d2e7fc640 1 -- 172.21.15.76:0/3550298283 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30029470 con 0x5596451f6080 2024-03-23T15:43:26.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3550298283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f6080 msgr2=0x5596451fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f6080 0x5596451fc590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5596451fd2c0 tx=0x7f3d30014000 comp rx=0 tx=0).stop 2024-03-23T15:43:26.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3550298283 shutdown_connections 2024-03-23T15:43:26.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645216900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f6080 0x5596451fc590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/3550298283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f5760 0x55964520a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3550298283 >> 172.21.15.76:0/3550298283 conn(0x5596451efe90 msgr2=0x5596451eeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3550298283 shutdown_connections 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- 172.21.15.76:0/3550298283 wait complete. 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- start start 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 Processor -- start 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f5760 0x559645206b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x559645203af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645204030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559645207220 con 0x5596451f6080 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559645207390 con 0x5596451f5760 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559645204570 con 0x5596451fec60 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d4dc78740 10 client.0 _ll_register_callbacks cb 0x7f3d380bc520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.915+0000 7f3d43fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x559645203af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x559645203af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50010/0 (socket says 172.21.15.76:50010) 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 -- 172.21.15.76:0/1461595475 learned_addr learned my addr 172.21.15.76:0/1461595475 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f5760 0x559645206b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d4962c640 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645204030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 -- 172.21.15.76:0/1461595475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 msgr2=0x559645204030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645204030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 -- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f5760 msgr2=0x559645206b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f5760 0x559645206b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 -- 172.21.15.76:0/1461595475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596451126d0 con 0x5596451f6080 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d43fff640 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x559645203af0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d3001ce80 tx=0x7f3d3001ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae9a928d91b3e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d23fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d23fff640 1 -- 172.21.15.76:0/1461595475 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30021050 con 0x5596451f6080 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596451807a0 con 0x5596451f6080 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d23fff640 1 -- 172.21.15.76:0/1461595475 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3d3001c2e0 con 0x5596451f6080 2024-03-23T15:43:26.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d23fff640 1 -- 172.21.15.76:0/1461595475 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3d3001b850 con 0x5596451f6080 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d4dc78740 10 client.15519 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x7f3d30017050 con 0x5596451f6080 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.916+0000 7f3d23fff640 1 -- 172.21.15.76:0/1461595475 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5596451807a0 con 0x5596451f6080 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d23fff640 1 -- 172.21.15.76:0/1461595475 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3d30028410 con 0x5596451f6080 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d4dc78740 10 client.15519 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d4dc78740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d4dc78740 20 client.15519 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645278190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d4dc78740 4 client.15519 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559644eac410 con 0x55964520ec20 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645278190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d48e2b640 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645278190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9381837e69bc20ea server_cookie=fb54e53d17df20c in_seq=0 out_seq=0 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.917+0000 7f3d23fff640 10 client.15519 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d23fff640 1 -- 172.21.15.76:0/1461595475 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559644eac410 con 0x55964520ec20 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d23fff640 10 client.15519 handle_command_reply: tid=0 2024-03-23T15:43:26.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 client.15519 shutdown 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 msgr2=0x559645203af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x559645203af0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d3001ce80 tx=0x7f3d3001ceb0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 shutdown_connections 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5596451fec60 0x559645204030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5596451f6080 0x559645203af0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55964520ec20 0x559645278190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d23fff640 0 client.15519 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 --2- 172.21.15.76:0/1461595475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596451f5760 0x559645206b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.918+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 >> 172.21.15.76:0/1461595475 conn(0x5596451efe90 msgr2=0x5596451f1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 shutdown_connections 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4dc78740 1 -- 172.21.15.76:0/1461595475 wait complete. 2024-03-23T15:43:26.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4dc78740 20 client.15519 trim_cache size 0 max 16384 2024-03-23T15:43:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:26.927 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-03-23T15:43:26.927 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "cde0d5da-5eb6-494e-9b96-fecff3db4c7c", 2024-03-23T15:43:26.927 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-03-23T15:43:26.927 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3d340a23d0 msgr2=0x7f3d340a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3d340a23d0 0x7f3d340a4890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d44103fa0 tx=0x7f3d30005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 msgr2=0x7f3d44102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 0x7f3d44102510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d38005940 tx=0x7f3d3802c670 comp rx=0 tx=0).stop 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 shutdown_connections 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3d340a23d0 0x7f3d340a4890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d44106660 0x7f3d44102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d44100f60 0x7f3d44102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 --2- 172.21.15.76:0/2695514943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d441005a0 0x7f3d44102510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 >> 172.21.15.76:0/2695514943 conn(0x7f3d44077710 msgr2=0x7f3d44104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 shutdown_connections 2024-03-23T15:43:26.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:26.919+0000 7f3d4b08f640 1 -- 172.21.15.76:0/2695514943 wait complete. 2024-03-23T15:43:26.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T15:43:26.933 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T15:43:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 -- 172.21.15.76:0/1223831040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 msgr2=0x7ff86c100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 --2- 172.21.15.76:0/1223831040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c100a20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009900 tx=0x7ff85c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 -- 172.21.15.76:0/1223831040 shutdown_connections 2024-03-23T15:43:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 --2- 172.21.15.76:0/1223831040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100f60 0x7ff86c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 --2- 172.21.15.76:0/1223831040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c100a20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 --2- 172.21.15.76:0/1223831040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c106660 0x7ff86c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 -- 172.21.15.76:0/1223831040 >> 172.21.15.76:0/1223831040 conn(0x7ff86c077710 msgr2=0x7ff86c077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 -- 172.21.15.76:0/1223831040 shutdown_connections 2024-03-23T15:43:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.036+0000 7ff87328f640 1 -- 172.21.15.76:0/1223831040 wait complete. 2024-03-23T15:43:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 Processor -- start 2024-03-23T15:43:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 -- start start 2024-03-23T15:43:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100f60 0x7ff86c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c106660 0x7ff86c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c197050 con 0x7ff86c1005a0 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c1971c0 con 0x7ff86c106660 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87328f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c197380 con 0x7ff86c100f60 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50028/0 (socket says 172.21.15.76:50028) 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 -- 172.21.15.76:0/815232812 learned_addr learned my addr 172.21.15.76:0/815232812 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87082a640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100f60 0x7ff86c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87182c640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c106660 0x7ff86c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 -- 172.21.15.76:0/815232812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100f60 msgr2=0x7ff86c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100f60 0x7ff86c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 -- 172.21.15.76:0/815232812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c106660 msgr2=0x7ff86c196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c106660 0x7ff86c196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.037+0000 7ff87102b640 1 -- 172.21.15.76:0/815232812 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c009560 con 0x7ff86c1005a0 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.038+0000 7ff87102b640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c19c850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff86800e9f0 tx=0x7ff86800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.038+0000 7ff8627fc640 1 -- 172.21.15.76:0/815232812 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868009700 con 0x7ff86c1005a0 2024-03-23T15:43:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.038+0000 7ff8627fc640 1 -- 172.21.15.76:0/815232812 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86800cd80 con 0x7ff86c1005a0 2024-03-23T15:43:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.038+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86c197660 con 0x7ff86c1005a0 2024-03-23T15:43:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.038+0000 7ff8627fc640 1 -- 172.21.15.76:0/815232812 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868018920 con 0x7ff86c1005a0 2024-03-23T15:43:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.038+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c1a38f0 con 0x7ff86c1005a0 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.040+0000 7ff8627fc640 1 -- 172.21.15.76:0/815232812 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7ff868018a80 con 0x7ff86c1005a0 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.040+0000 7ff875e73740 1 Processor -- start 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.040+0000 7ff875e73740 1 -- start start 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff8627fc640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff8580a2350 0x7ff8580a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff875e73740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74be90 0x564eeb74c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff875e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb7573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff875e73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb759cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff875e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564eeb670f30 con 0x564eeb74c7b0 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff875e73740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564eeb6ba810 con 0x564eeb74be90 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff875e73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564eeb6b7d40 con 0x564eeb7578e0 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87182c640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff8580a2350 0x7ff8580a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87082a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb7573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74be90 0x564eeb74c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74be90 0x564eeb74c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:34150/0 (socket says 172.21.15.76:34150) 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87082a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb7573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50030/0 (socket says 172.21.15.76:50030) 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 -- 172.21.15.76:0/2041031766 learned_addr learned my addr 172.21.15.76:0/2041031766 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff8627fc640 1 -- 172.21.15.76:0/815232812 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff868014070 con 0x7ff86c1005a0 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87182c640 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb759cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 -- 172.21.15.76:0/2041031766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 msgr2=0x564eeb759cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb759cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 -- 172.21.15.76:0/2041031766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 msgr2=0x564eeb7573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb7573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 -- 172.21.15.76:0/2041031766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eeb674ea0 con 0x564eeb74be90 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87082a640 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb7573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87182c640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff8580a2350 0x7ff8580a4810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff864005fd0 tx=0x7ff864009e60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.041+0000 7ff87102b640 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74be90 0x564eeb74c270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff868097380 tx=0x7ff868010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2584f549e57369 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff8577fe640 1 -- 172.21.15.76:0/2041031766 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86809af20 con 0x564eeb74be90 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff8577fe640 1 -- 172.21.15.76:0/2041031766 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86809efc0 con 0x564eeb74be90 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff8577fe640 1 -- 172.21.15.76:0/2041031766 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8680aeec0 con 0x564eeb74be90 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 -- 172.21.15.76:0/2041031766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74be90 msgr2=0x564eeb74c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74be90 0x564eeb74c270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff868097380 tx=0x7ff868010f50 comp rx=0 tx=0).stop 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 -- 172.21.15.76:0/2041031766 shutdown_connections 2024-03-23T15:43:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb759cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb7573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 --2- 172.21.15.76:0/2041031766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74be90 0x564eeb74c270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 -- 172.21.15.76:0/2041031766 >> 172.21.15.76:0/2041031766 conn(0x564eeb746900 msgr2=0x564eeb746ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 -- 172.21.15.76:0/2041031766 shutdown_connections 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.042+0000 7ff875e73740 1 -- 172.21.15.76:0/2041031766 wait complete. 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 -- start start 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 Processor -- start 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb7548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb754de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb7578e0 0x564eeb755320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564eeb6b3520 con 0x564eeb74c7b0 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564eeb386430 con 0x564eeb7578e0 2024-03-23T15:43:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564eeb342e00 con 0x564eeb74be90 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff875e73740 10 client.0 _ll_register_callbacks cb 0x564eeb651d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff87082a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb754de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff87082a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb754de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50044/0 (socket says 172.21.15.76:50044) 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff87082a640 1 -- 172.21.15.76:0/2956583644 learned_addr learned my addr 172.21.15.76:0/2956583644 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff87102b640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb7548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.043+0000 7ff87182c640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb7578e0 0x564eeb755320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87082a640 1 -- 172.21.15.76:0/2956583644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 msgr2=0x564eeb7548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87082a640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb7548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87082a640 1 -- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb7578e0 msgr2=0x564eeb755320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87082a640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb7578e0 0x564eeb755320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87082a640 1 -- 172.21.15.76:0/2956583644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c1a38f0 con 0x564eeb74c7b0 2024-03-23T15:43:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87182c640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb7578e0 0x564eeb755320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:27.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87102b640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb7548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:27.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff87082a640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb754de0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff85c035000 tx=0x7ff85c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29610a0e9e6fc82 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff854ff9640 1 -- 172.21.15.76:0/2956583644 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c004a90 con 0x564eeb74c7b0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff854ff9640 1 -- 172.21.15.76:0/2956583644 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff85c004260 con 0x564eeb74c7b0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564eeb3b27e0 con 0x564eeb74c7b0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff854ff9640 1 -- 172.21.15.76:0/2956583644 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff85c004420 con 0x564eeb74c7b0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff875e73740 10 client.15531 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564eeb6b1ba0 con 0x564eeb74c7b0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff854ff9640 1 -- 172.21.15.76:0/2956583644 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564eeb3b27e0 con 0x564eeb74c7b0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff854ff9640 1 -- 172.21.15.76:0/2956583644 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff85c03c040 con 0x564eeb74c7b0 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff875e73740 10 client.15531 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff875e73740 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.044+0000 7ff875e73740 20 client.15531 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.045+0000 7ff875e73740 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb767df0 0x564eeb76a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.045+0000 7ff875e73740 4 client.15531 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.045+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eeb3ef720 con 0x564eeb767df0 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.045+0000 7ff87102b640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb767df0 0x564eeb76a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.045+0000 7ff87102b640 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb767df0 0x564eeb76a210 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a05f1f8d08b1bcb server_cookie=9b9630c754629b4e in_seq=0 out_seq=0 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.045+0000 7ff854ff9640 10 client.15531 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff854ff9640 1 -- 172.21.15.76:0/2956583644 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564eeb3ef720 con 0x564eeb767df0 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff854ff9640 10 client.15531 handle_command_reply: tid=0 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 client.15531 shutdown 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 msgr2=0x564eeb754de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb754de0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff85c035000 tx=0x7ff85c0048f0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 shutdown_connections 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb767df0 0x564eeb76a210 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb7578e0 0x564eeb755320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff854ff9640 0 client.15531 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74c7b0 0x564eeb754de0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 --2- 172.21.15.76:0/2956583644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb7548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 >> 172.21.15.76:0/2956583644 conn(0x564eeb746900 msgr2=0x564eeb748650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 shutdown_connections 2024-03-23T15:43:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 1 -- 172.21.15.76:0/2956583644 wait complete. 2024-03-23T15:43:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.046+0000 7ff875e73740 20 client.15531 trim_cache size 0 max 16384 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.063+0000 7ff875e73740 1 Processor -- start 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.063+0000 7ff875e73740 1 -- start start 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff875e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74be90 0x564eeb763150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff875e73740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff875e73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb755140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff875e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564eeb638430 con 0x564eeb74be90 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff875e73740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564eeb742550 con 0x564eeb74c7b0 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff875e73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564eeb759cd0 con 0x564eeb7578e0 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74be90 0x564eeb763150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74be90 0x564eeb763150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50050/0 (socket says 172.21.15.76:50050) 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 -- 172.21.15.76:0/3408283301 learned_addr learned my addr 172.21.15.76:0/3408283301 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87082a640 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87182c640 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb755140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 -- 172.21.15.76:0/3408283301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 msgr2=0x564eeb755140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb755140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 -- 172.21.15.76:0/3408283301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 msgr2=0x564eeb7459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 -- 172.21.15.76:0/3408283301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eeb64db20 con 0x564eeb74be90 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff87102b640 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74be90 0x564eeb763150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff8680a30e0 tx=0x7ff86809aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754457af5ea1e7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff856ffd640 1 -- 172.21.15.76:0/3408283301 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868010b80 con 0x564eeb74be90 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff856ffd640 1 -- 172.21.15.76:0/3408283301 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff868097590 con 0x564eeb74be90 2024-03-23T15:43:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.064+0000 7ff856ffd640 1 -- 172.21.15.76:0/3408283301 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8680cc1e0 con 0x564eeb74be90 2024-03-23T15:43:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 -- 172.21.15.76:0/3408283301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74be90 msgr2=0x564eeb763150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74be90 0x564eeb763150 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff8680a30e0 tx=0x7ff86809aa60 comp rx=0 tx=0).stop 2024-03-23T15:43:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 -- 172.21.15.76:0/3408283301 shutdown_connections 2024-03-23T15:43:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb7578e0 0x564eeb755140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 --2- 172.21.15.76:0/3408283301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb74be90 0x564eeb763150 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 -- 172.21.15.76:0/3408283301 >> 172.21.15.76:0/3408283301 conn(0x564eeb746900 msgr2=0x564eeb7452d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 -- 172.21.15.76:0/3408283301 shutdown_connections 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.065+0000 7ff875e73740 1 -- 172.21.15.76:0/3408283301 wait complete. 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 -- start start 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 Processor -- start 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb751490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb7578e0 0x564eeb751f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564eeb7cd130 con 0x564eeb7578e0 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564eeb7cd270 con 0x564eeb74c7b0 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564eeb7cd3b0 con 0x564eeb74be90 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff875e73740 10 client.0 _ll_register_callbacks cb 0x7ff85c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff87182c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb7578e0 0x564eeb751f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff87102b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb751490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.066+0000 7ff87182c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb7578e0 0x564eeb751f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50054/0 (socket says 172.21.15.76:50054) 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87102b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb751490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:38274/0 (socket says 172.21.15.76:38274) 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87182c640 1 -- 172.21.15.76:0/1539300275 learned_addr learned my addr 172.21.15.76:0/1539300275 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87082a640 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87182c640 1 -- 172.21.15.76:0/1539300275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 msgr2=0x564eeb751490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87182c640 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb751490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87182c640 1 -- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 msgr2=0x564eeb7519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87182c640 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87182c640 1 -- 172.21.15.76:0/1539300275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eeb674ea0 con 0x564eeb7578e0 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff87182c640 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb7578e0 0x564eeb751f10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff864011c40 tx=0x7ff864004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812a189660c22d64 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff854ff9640 1 -- 172.21.15.76:0/1539300275 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86402c710 con 0x564eeb7578e0 2024-03-23T15:43:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff854ff9640 1 -- 172.21.15.76:0/1539300275 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff864026040 con 0x564eeb7578e0 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff86809efc0 con 0x564eeb7578e0 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff854ff9640 1 -- 172.21.15.76:0/1539300275 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff86402c870 con 0x564eeb7578e0 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff875e73740 10 client.15537 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564eeb673700 con 0x564eeb7578e0 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.067+0000 7ff854ff9640 1 -- 172.21.15.76:0/1539300275 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86809efc0 con 0x564eeb7578e0 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff854ff9640 1 -- 172.21.15.76:0/1539300275 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff86402abc0 con 0x564eeb7578e0 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 10 client.15537 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 20 client.15537 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb7ced10 0x564eeb7d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 4 client.15537 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564eeb43e890 con 0x564eeb7ced10 2024-03-23T15:43:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff87102b640 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb7ced10 0x564eeb7d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff87102b640 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb7ced10 0x564eeb7d1100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba04f3f24e9b4a server_cookie=41c4a77d248eab31 in_seq=0 out_seq=0 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff854ff9640 10 client.15537 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff854ff9640 1 -- 172.21.15.76:0/1539300275 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564eeb43e890 con 0x564eeb7ced10 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff854ff9640 10 client.15537 handle_command_reply: tid=0 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 1 client.15537 shutdown 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb7578e0 msgr2=0x564eeb751f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.068+0000 7ff875e73740 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb7578e0 0x564eeb751f10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff864011c40 tx=0x7ff864004ab0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 shutdown_connections 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eeb7578e0 0x564eeb751f10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564eeb7ced10 0x564eeb7d1100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564eeb74c7b0 0x564eeb7519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 --2- 172.21.15.76:0/1539300275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564eeb74be90 0x564eeb751490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff854ff9640 0 client.15537 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 >> 172.21.15.76:0/1539300275 conn(0x564eeb746900 msgr2=0x564eeb745b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 shutdown_connections 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 1 -- 172.21.15.76:0/1539300275 wait complete. 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff875e73740 20 client.15537 trim_cache size 0 max 16384 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stdout: "cde0d5da-5eb6-494e-9b96-fecff3db4c7c": { 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/", 2024-03-23T15:43:27.079 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "cde0d5da-5eb6-494e-9b96-fecff3db4c7c", 2024-03-23T15:43:27.080 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-03-23T15:43:27.080 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:43:27.080 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:43:27.080 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:27.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff8580a2350 msgr2=0x7ff8580a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff8580a2350 0x7ff8580a4810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff864005fd0 tx=0x7ff864009e60 comp rx=0 tx=0).stop 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 msgr2=0x7ff86c19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c19c850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff86800e9f0 tx=0x7ff86800eeb0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 shutdown_connections 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff8580a2350 0x7ff8580a4810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff86c106660 0x7ff86c196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff86c100f60 0x7ff86c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 --2- 172.21.15.76:0/815232812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86c1005a0 0x7ff86c19c850 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 >> 172.21.15.76:0/815232812 conn(0x7ff86c077710 msgr2=0x7ff86c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 shutdown_connections 2024-03-23T15:43:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.069+0000 7ff87328f640 1 -- 172.21.15.76:0/815232812 wait complete. 2024-03-23T15:43:27.086 INFO:tasks.cephfs.filesystem:scrub status for tag:cde0d5da-5eb6-494e-9b96-fecff3db4c7c - {'path': '/', 'tag': 'cde0d5da-5eb6-494e-9b96-fecff3db4c7c', 'options': 'recursive,force'} 2024-03-23T15:43:27.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.110+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4180111850 con 0x7f416c0a2470 2024-03-23T15:43:27.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.167+0000 7f417cff9640 1 -- 172.21.15.76:0/74328691 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4180111850 con 0x7f416c0a2470 2024-03-23T15:43:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:Schedule set for path / 2024-03-23T15:43:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f416c0a2470 msgr2=0x7f416c0a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f416c0a2470 0x7f416c0a4930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4168009230 tx=0x7f4168005e80 comp rx=0 tx=0).stop 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f418019c750 msgr2=0x7f418019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f418019c750 0x7f418019cc00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f417800ebc0 tx=0x7f417800c6a0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 shutdown_connections 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f416c0a2470 0x7f416c0a4930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f418019c750 0x7f418019cc00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4180069730 0x7f41801983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 --2- 172.21.15.76:0/74328691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4180068d90 0x7f4180197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 >> 172.21.15.76:0/74328691 conn(0x7f4180077460 msgr2=0x7f41800fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 shutdown_connections 2024-03-23T15:43:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.169+0000 7f418528f640 1 -- 172.21.15.76:0/74328691 wait complete. 2024-03-23T15:43:27.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-23T15:43:27.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 -- 172.21.15.76:0/3137976678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c1076d0 msgr2=0x7f5f5c107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/3137976678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c1076d0 0x7f5f5c107ab0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009930 tx=0x7f5f5002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 -- 172.21.15.76:0/3137976678 shutdown_connections 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/3137976678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5c06a910 0x7f5f5c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/3137976678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c069f50 0x7f5f5c06a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/3137976678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c1076d0 0x7f5f5c107ab0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 -- 172.21.15.76:0/3137976678 >> 172.21.15.76:0/3137976678 conn(0x7f5f5c077330 msgr2=0x7f5f5c077740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 -- 172.21.15.76:0/3137976678 shutdown_connections 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.326+0000 7f5f62a9d640 1 -- 172.21.15.76:0/3137976678 wait complete. 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 Processor -- start 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 -- start start 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c069f50 0x7f5f5c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c06a910 0x7f5f5c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5c1076d0 0x7f5f5c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c198130 con 0x7f5f5c06a910 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1982a0 con 0x7f5f5c1076d0 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f62a9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c198460 con 0x7f5f5c069f50 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c06a910 0x7f5f5c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c06a910 0x7f5f5c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50082/0 (socket says 172.21.15.76:50082) 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 -- 172.21.15.76:0/616452583 learned_addr learned my addr 172.21.15.76:0/616452583 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f60839640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c069f50 0x7f5f5c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f6103a640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5c1076d0 0x7f5f5c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 -- 172.21.15.76:0/616452583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c069f50 msgr2=0x7f5f5c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c069f50 0x7f5f5c19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 -- 172.21.15.76:0/616452583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5c1076d0 msgr2=0x7f5f5c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5c1076d0 0x7f5f5c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 -- 172.21.15.76:0/616452583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f50009560 con 0x7f5f5c06a910 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.327+0000 7f5f5bfff640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c06a910 0x7f5f5c19de70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800e9f0 tx=0x7f5f4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.328+0000 7f5f59ffb640 1 -- 172.21.15.76:0/616452583 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48009720 con 0x7f5f5c06a910 2024-03-23T15:43:27.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.328+0000 7f5f59ffb640 1 -- 172.21.15.76:0/616452583 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f4800cd80 con 0x7f5f5c06a910 2024-03-23T15:43:27.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.328+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c198740 con 0x7f5f5c06a910 2024-03-23T15:43:27.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.328+0000 7f5f59ffb640 1 -- 172.21.15.76:0/616452583 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48010690 con 0x7f5f5c06a910 2024-03-23T15:43:27.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.328+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c1a49d0 con 0x7f5f5c06a910 2024-03-23T15:43:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.329+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5c102e00 con 0x7f5f5c06a910 2024-03-23T15:43:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.330+0000 7f5f59ffb640 1 -- 172.21.15.76:0/616452583 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f5f480046c0 con 0x7f5f5c06a910 2024-03-23T15:43:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.330+0000 7f5f59ffb640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f4c0a23f0 0x7f5f4c0a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.330+0000 7f5f60839640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f4c0a23f0 0x7f5f4c0a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.330+0000 7f5f59ffb640 1 -- 172.21.15.76:0/616452583 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5f48014070 con 0x7f5f5c06a910 2024-03-23T15:43:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.331+0000 7f5f60839640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f4c0a23f0 0x7f5f4c0a48b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005ab0 tx=0x7f5f50005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.334+0000 7f5f59ffb640 1 -- 172.21.15.76:0/616452583 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f5f48097100 con 0x7f5f5c06a910 2024-03-23T15:43:27.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.530+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f5f5c109ca0 con 0x7f5f4c0a23f0 2024-03-23T15:43:27.548 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:27 smithi183 ceph-mon[36296]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:27.548 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:27 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1832998324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:27.548 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:27 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3142516292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:27.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:27 smithi192 ceph-mon[36653]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:27.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:27 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1832998324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:27.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:27 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3142516292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:27.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:27 smithi076 ceph-mon[32176]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:43:27.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:27 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1832998324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:27.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:27 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3142516292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:43:27.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.585+0000 7f5f59ffb640 1 -- 172.21.15.76:0/616452583 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f5f5c109ca0 con 0x7f5f4c0a23f0 2024-03-23T15:43:27.588 INFO:teuthology.orchestra.run.smithi076.stdout:Retention added to path / 2024-03-23T15:43:27.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f4c0a23f0 msgr2=0x7f5f4c0a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f4c0a23f0 0x7f5f4c0a48b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005ab0 tx=0x7f5f50005fd0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c06a910 msgr2=0x7f5f5c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c06a910 0x7f5f5c19de70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800e9f0 tx=0x7f5f4800edf0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 shutdown_connections 2024-03-23T15:43:27.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f4c0a23f0 0x7f5f4c0a48b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f5c1076d0 0x7f5f5c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f5c06a910 0x7f5f5c19de70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 --2- 172.21.15.76:0/616452583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c069f50 0x7f5f5c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 >> 172.21.15.76:0/616452583 conn(0x7f5f5c077330 msgr2=0x7f5f5c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 shutdown_connections 2024-03-23T15:43:27.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.587+0000 7f5f62a9d640 1 -- 172.21.15.76:0/616452583 wait complete. 2024-03-23T15:43:27.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-23T15:43:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/1613054225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 msgr2=0x7f6cc0100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/1613054225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc0100ce0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009900 tx=0x7f6cb402bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/1613054225 shutdown_connections 2024-03-23T15:43:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/1613054225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cc0101220 0x7f6cc010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/1613054225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc0100ce0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/1613054225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc01055b0 0x7f6cc0105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/1613054225 >> 172.21.15.76:0/1613054225 conn(0x7f6cc0077460 msgr2=0x7f6cc0077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/1613054225 shutdown_connections 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/1613054225 wait complete. 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 Processor -- start 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.701+0000 7f6cc5e8f640 1 -- start start 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc0101220 0x7f6cc019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cc01055b0 0x7f6cc0196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0197180 con 0x7f6cc0100860 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01972f0 con 0x7f6cc0101220 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01974b0 con 0x7f6cc01055b0 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50104/0 (socket says 172.21.15.76:50104) 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 -- 172.21.15.76:0/3845147439 learned_addr learned my addr 172.21.15.76:0/3845147439 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbffff640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cc01055b0 0x7f6cc0196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbeffd640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc0101220 0x7f6cc019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 -- 172.21.15.76:0/3845147439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cc01055b0 msgr2=0x7f6cc0196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cc01055b0 0x7f6cc0196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 -- 172.21.15.76:0/3845147439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc0101220 msgr2=0x7f6cc019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc0101220 0x7f6cc019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 -- 172.21.15.76:0/3845147439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb4009560 con 0x7f6cc0100860 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbf7fe640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc019c930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000e9f0 tx=0x7f6cb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbcff9640 1 -- 172.21.15.76:0/3845147439 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0009700 con 0x7f6cc0100860 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbcff9640 1 -- 172.21.15.76:0/3845147439 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cb000cd80 con 0x7f6cc0100860 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc0197790 con 0x7f6cc0100860 2024-03-23T15:43:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cbcff9640 1 -- 172.21.15.76:0/3845147439 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0018920 con 0x7f6cc0100860 2024-03-23T15:43:27.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.702+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc01a39d0 con 0x7f6cc0100860 2024-03-23T15:43:27.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.703+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc004f470 con 0x7f6cc0100860 2024-03-23T15:43:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.705+0000 7f6cbcff9640 1 -- 172.21.15.76:0/3845147439 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f6cb0018a80 con 0x7f6cc0100860 2024-03-23T15:43:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.706+0000 7f6cbcff9640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6ca80a23f0 0x7f6ca80a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.706+0000 7f6cbeffd640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6ca80a23f0 0x7f6ca80a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.706+0000 7f6cbcff9640 1 -- 172.21.15.76:0/3845147439 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6cb0014070 con 0x7f6cc0100860 2024-03-23T15:43:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.706+0000 7f6cbeffd640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6ca80a23f0 0x7f6ca80a48b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4006010 tx=0x7f6cb4005e40 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:27.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.709+0000 7f6cbcff9640 1 -- 172.21.15.76:0/3845147439 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6cb0097580 con 0x7f6cc0100860 2024-03-23T15:43:27.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.904+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6cc0110850 con 0x7f6ca80a23f0 2024-03-23T15:43:27.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.911+0000 7f6cbcff9640 1 -- 172.21.15.76:0/3845147439 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f6cc0110850 con 0x7f6ca80a23f0 2024-03-23T15:43:27.915 INFO:teuthology.orchestra.run.smithi076.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-23T00:00:00", "created": "2024-03-23T15:43:27", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6ca80a23f0 msgr2=0x7f6ca80a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6ca80a23f0 0x7f6ca80a48b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4006010 tx=0x7f6cb4005e40 comp rx=0 tx=0).stop 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 msgr2=0x7f6cc019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc019c930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000e9f0 tx=0x7f6cb000eeb0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 shutdown_connections 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6ca80a23f0 0x7f6ca80a48b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cc01055b0 0x7f6cc0196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc0101220 0x7f6cc019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 --2- 172.21.15.76:0/3845147439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cc0100860 0x7f6cc019c930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:27.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 >> 172.21.15.76:0/3845147439 conn(0x7f6cc0077460 msgr2=0x7f6cc00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:27.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 shutdown_connections 2024-03-23T15:43:27.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:27.914+0000 7f6cc5e8f640 1 -- 172.21.15.76:0/3845147439 wait complete. 2024-03-23T15:43:27.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-23T15:43:28.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 -- 172.21.15.76:0/988042954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 msgr2=0x7f1aa810d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 --2- 172.21.15.76:0/988042954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa810d3d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009ca0 tx=0x7f1aa002c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 -- 172.21.15.76:0/988042954 shutdown_connections 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 --2- 172.21.15.76:0/988042954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa810d3d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 --2- 172.21.15.76:0/988042954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa8100860 0x7f1aa8100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 --2- 172.21.15.76:0/988042954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa8106740 0x7f1aa8106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 -- 172.21.15.76:0/988042954 >> 172.21.15.76:0/988042954 conn(0x7f1aa8077450 msgr2=0x7f1aa8077860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 -- 172.21.15.76:0/988042954 shutdown_connections 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.028+0000 7f1aae55c640 1 -- 172.21.15.76:0/988042954 wait complete. 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 Processor -- start 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 -- start start 2024-03-23T15:43:28.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa8100860 0x7f1aa81024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa8102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa8106740 0x7f1aa8102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81a6080 con 0x7f1aa8101200 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81a61f0 con 0x7f1aa8100860 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aae55c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81a63b0 con 0x7f1aa8106740 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa8102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa8102a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50114/0 (socket says 172.21.15.76:50114) 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 -- 172.21.15.76:0/45586533 learned_addr learned my addr 172.21.15.76:0/45586533 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:28.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aacaf9640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa8106740 0x7f1aa8102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa8100860 0x7f1aa81024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 -- 172.21.15.76:0/45586533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa8106740 msgr2=0x7f1aa8102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa8106740 0x7f1aa8102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 -- 172.21.15.76:0/45586533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa8100860 msgr2=0x7f1aa81024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa8100860 0x7f1aa81024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.029+0000 7f1aa77fe640 1 -- 172.21.15.76:0/45586533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa0009900 con 0x7f1aa8101200 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.030+0000 7f1aa77fe640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa8102a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9f0 tx=0x7f1a9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.030+0000 7f1aa57fa640 1 -- 172.21.15.76:0/45586533 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c009700 con 0x7f1aa8101200 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.030+0000 7f1aa57fa640 1 -- 172.21.15.76:0/45586533 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a9c00cd80 con 0x7f1aa8101200 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.030+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa81a6690 con 0x7f1aa8101200 2024-03-23T15:43:28.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.030+0000 7f1aa57fa640 1 -- 172.21.15.76:0/45586533 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c018920 con 0x7f1aa8101200 2024-03-23T15:43:28.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.030+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa81a6ba0 con 0x7f1aa8101200 2024-03-23T15:43:28.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.031+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa8101d50 con 0x7f1aa8101200 2024-03-23T15:43:28.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.032+0000 7f1aa57fa640 1 -- 172.21.15.76:0/45586533 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f1a9c018a80 con 0x7f1aa8101200 2024-03-23T15:43:28.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.032+0000 7f1aa57fa640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a900a23f0 0x7f1a900a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:28.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.032+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a900a23f0 0x7f1a900a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:28.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.032+0000 7f1aa57fa640 1 -- 172.21.15.76:0/45586533 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1a9c014070 con 0x7f1aa8101200 2024-03-23T15:43:28.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.033+0000 7f1aa7fff640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a900a23f0 0x7f1a900a48b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009b50 tx=0x7f1a98005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:28.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.037+0000 7f1aa57fa640 1 -- 172.21.15.76:0/45586533 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f1a9c053280 con 0x7f1aa8101200 2024-03-23T15:43:28.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.230+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1aa8110700 con 0x7f1a900a23f0 2024-03-23T15:43:28.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.237+0000 7f1aa57fa640 1 -- 172.21.15.76:0/45586533 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1aa8110700 con 0x7f1a900a23f0 2024-03-23T15:43:28.240 INFO:teuthology.orchestra.run.smithi076.stdout:/ 1m 6m3h 2024-03-23T15:43:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a900a23f0 msgr2=0x7f1a900a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a900a23f0 0x7f1a900a48b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009b50 tx=0x7f1a98005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 msgr2=0x7f1aa8102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa8102a20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e9f0 tx=0x7f1a9c00eeb0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 shutdown_connections 2024-03-23T15:43:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1a900a23f0 0x7f1a900a48b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1aa8106740 0x7f1aa8102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa8101200 0x7f1aa8102a20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 --2- 172.21.15.76:0/45586533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1aa8100860 0x7f1aa81024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 >> 172.21.15.76:0/45586533 conn(0x7f1aa8077450 msgr2=0x7f1aa8104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 shutdown_connections 2024-03-23T15:43:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:28.240+0000 7f1aae55c640 1 -- 172.21.15.76:0/45586533 wait complete. 2024-03-23T15:43:28.253 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-23T15:43:28.281 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-23T15:43:28.292 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-23T15:43:28.303 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:43:28.304 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-23T15:43:28.323 INFO:teuthology.run_tasks:Running task workunit... 2024-03-23T15:43:28.336 INFO:tasks.workunit:Pulling workunits from ref d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b 2024-03-23T15:43:28.336 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-23T15:43:28.336 INFO:tasks.workunit:timeout=6h 2024-03-23T15:43:28.336 INFO:tasks.workunit:cleanup=True 2024-03-23T15:43:28.337 DEBUG:teuthology.orchestra.run.smithi076:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:28.351 INFO:teuthology.orchestra.run.smithi076.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:28.351 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-23T15:43:28.351 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-23T15:43:28.351 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T15:43:28.351 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-23T15:43:28.352 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-03-23 15:41:24.075695705 +0000 2024-03-23T15:43:28.352 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-03-23 15:41:24.075695705 +0000 2024-03-23T15:43:28.352 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-03-23 15:43:18.019834968 +0000 2024-03-23T15:43:28.352 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-03-23 15:41:24.075695705 +0000 2024-03-23T15:43:28.352 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-23T15:43:28.352 DEBUG:teuthology.orchestra.run.smithi076:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-23T15:43:28.426 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b 2024-03-23T15:43:28.481 INFO:tasks.workunit.client.0.smithi076.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-23T15:43:28.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:28 smithi192 ceph-mon[36653]: scrub queued for path: / 2024-03-23T15:43:28.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:28 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:43:28.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:28 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:43:28.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:28 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:43:28.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:28 smithi192 ceph-mon[36653]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:28.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:28 smithi192 ceph-mon[36653]: scrub summary: idle 2024-03-23T15:43:28.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:28 smithi192 ceph-mon[36653]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:28.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:28 smithi076 ceph-mon[32176]: scrub queued for path: / 2024-03-23T15:43:28.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:28 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:43:28.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:28 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:43:28.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:28 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:43:28.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:28 smithi076 ceph-mon[32176]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:28.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:28 smithi076 ceph-mon[32176]: scrub summary: idle 2024-03-23T15:43:28.579 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:28 smithi076 ceph-mon[32176]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:28.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:28 smithi183 ceph-mon[36296]: scrub queued for path: / 2024-03-23T15:43:28.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:28 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:43:28.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:28 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:43:28.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:28 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:43:28.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:28 smithi183 ceph-mon[36296]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:28.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:28 smithi183 ceph-mon[36296]: scrub summary: idle 2024-03-23T15:43:28.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:28 smithi183 ceph-mon[36296]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:29.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:29 smithi192 ceph-mon[36653]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 733 B/s wr, 22 op/s 2024-03-23T15:43:29.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:29 smithi192 ceph-mon[36653]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:29.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:29 smithi192 ceph-mon[36653]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:29.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:29 smithi076 ceph-mon[32176]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 733 B/s wr, 22 op/s 2024-03-23T15:43:29.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:29 smithi076 ceph-mon[32176]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:29.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:29 smithi076 ceph-mon[32176]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:29.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:29 smithi183 ceph-mon[36296]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 733 B/s wr, 22 op/s 2024-03-23T15:43:29.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:29 smithi183 ceph-mon[36296]: from='client.15549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:29.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:29 smithi183 ceph-mon[36296]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:43:31.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:31 smithi192 ceph-mon[36653]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 570 B/s wr, 17 op/s 2024-03-23T15:43:31.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:31 smithi076 ceph-mon[32176]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 570 B/s wr, 17 op/s 2024-03-23T15:43:31.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:31 smithi183 ceph-mon[36296]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 570 B/s wr, 17 op/s 2024-03-23T15:43:33.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:33 smithi192 ceph-mon[36653]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-03-23T15:43:33.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:33 smithi076 ceph-mon[32176]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-03-23T15:43:33.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:33 smithi183 ceph-mon[36296]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-03-23T15:43:35.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:35 smithi192 ceph-mon[36653]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-03-23T15:43:35.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:35 smithi076 ceph-mon[32176]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-03-23T15:43:35.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:35 smithi183 ceph-mon[36296]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-03-23T15:43:37.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:37 smithi192 ceph-mon[36653]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-03-23T15:43:37.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:37 smithi076 ceph-mon[32176]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-03-23T15:43:37.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:37 smithi183 ceph-mon[36296]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-03-23T15:43:39.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:39 smithi192 ceph-mon[36653]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T15:43:39.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:39 smithi076 ceph-mon[32176]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T15:43:39.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:39 smithi183 ceph-mon[36296]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 -- 172.21.15.76:0/3025693896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8068d90 msgr2=0x7fe7a80691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/3025693896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8068d90 0x7fe7a80691f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe790009900 tx=0x7fe79002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 -- 172.21.15.76:0/3025693896 shutdown_connections 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/3025693896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8069730 0x7fe7a810e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/3025693896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8068d90 0x7fe7a80691f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/3025693896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a8107800 0x7fe7a8107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 -- 172.21.15.76:0/3025693896 >> 172.21.15.76:0/3025693896 conn(0x7fe7a8077460 msgr2=0x7fe7a8077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 -- 172.21.15.76:0/3025693896 shutdown_connections 2024-03-23T15:43:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.463+0000 7fe7ace0e640 1 -- 172.21.15.76:0/3025693896 wait complete. 2024-03-23T15:43:41.467 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 Processor -- start 2024-03-23T15:43:41.467 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 -- start start 2024-03-23T15:43:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8068d90 0x7fe7a8100930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8069730 0x7fe7a8100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a8107800 0x7fe7a81013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81a60b0 con 0x7fe7a8069730 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81a6220 con 0x7fe7a8068d90 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7ace0e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81a63e0 con 0x7fe7a8107800 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8069730 0x7fe7a8100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8069730 0x7fe7a8100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45768/0 (socket says 172.21.15.76:45768) 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe79ffff640 1 -- 172.21.15.76:0/275316146 learned_addr learned my addr 172.21.15.76:0/275316146 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7a659c640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8068d90 0x7fe7a8100930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.464+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a8107800 0x7fe7a81013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79ffff640 1 -- 172.21.15.76:0/275316146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a8107800 msgr2=0x7fe7a81013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79ffff640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a8107800 0x7fe7a81013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79ffff640 1 -- 172.21.15.76:0/275316146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8068d90 msgr2=0x7fe7a8100930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79ffff640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8068d90 0x7fe7a8100930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79ffff640 1 -- 172.21.15.76:0/275316146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe790009560 con 0x7fe7a8069730 2024-03-23T15:43:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe7a659c640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8068d90 0x7fe7a8100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79ffff640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8069730 0x7fe7a8100e70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe790005940 tx=0x7fe79002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79f7fe640 1 -- 172.21.15.76:0/275316146 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790004350 con 0x7fe7a8069730 2024-03-23T15:43:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79f7fe640 1 -- 172.21.15.76:0/275316146 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7900044b0 con 0x7fe7a8069730 2024-03-23T15:43:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a81a6660 con 0x7fe7a8069730 2024-03-23T15:43:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe79f7fe640 1 -- 172.21.15.76:0/275316146 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790003d30 con 0x7fe7a8069730 2024-03-23T15:43:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.465+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a81a6b70 con 0x7fe7a8069730 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.467+0000 7fe79f7fe640 1 -- 172.21.15.76:0/275316146 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7fe790004df0 con 0x7fe7a8069730 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.467+0000 7fe79f7fe640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe7940a23f0 0x7fe7940a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7a659c640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe7940a23f0 0x7fe7940a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe79f7fe640 1 -- 172.21.15.76:0/275316146 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe7900f0320 con 0x7fe7a8069730 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 Processor -- start 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 -- start start 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7a659c640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe7940a23f0 0x7fe7940a48b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe798009f40 tx=0x7fe798005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb2649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb26d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb2700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55addb21f1b0 con 0x55addb2645e0 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55addb1d1f80 con 0x55addb26dcd0 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55addb1cf4b0 con 0x55addb264f00 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb2649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb26d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb2649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45778/0 (socket says 172.21.15.76:45778) 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7a659c640 1 -- 172.21.15.76:0/3572035393 learned_addr learned my addr 172.21.15.76:0/3572035393 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.468+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb2700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79ffff640 1 -- 172.21.15.76:0/3572035393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 msgr2=0x55addb2700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79ffff640 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb2700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79ffff640 1 -- 172.21.15.76:0/3572035393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 msgr2=0x55addb2649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79ffff640 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb2649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79ffff640 1 -- 172.21.15.76:0/3572035393 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55addb160810 con 0x55addb264f00 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe7a659c640 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb2649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79ffff640 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb26d790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe7900eda00 tx=0x7fe7900bc530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3ead51b0d024ea server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79cff9640 1 -- 172.21.15.76:0/3572035393 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900bcbd0 con 0x55addb264f00 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79cff9640 1 -- 172.21.15.76:0/3572035393 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7900c0d20 con 0x55addb264f00 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.469+0000 7fe79cff9640 1 -- 172.21.15.76:0/3572035393 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900c50e0 con 0x55addb264f00 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 -- 172.21.15.76:0/3572035393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 msgr2=0x55addb26d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb26d790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe7900eda00 tx=0x7fe7900bc530 comp rx=0 tx=0).stop 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 -- 172.21.15.76:0/3572035393 shutdown_connections 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb2700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb26d790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/3572035393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb2649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 -- 172.21.15.76:0/3572035393 >> 172.21.15.76:0/3572035393 conn(0x55addb25ef00 msgr2=0x55addb25f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 -- 172.21.15.76:0/3572035393 shutdown_connections 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 -- 172.21.15.76:0/3572035393 wait complete. 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 -- start start 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.470+0000 7fe7af9ef740 1 Processor -- start 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb2645e0 0x55addb26acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb264f00 0x55addb26b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb26dcd0 0x55addb26b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55addb1cac90 con 0x55addb26dcd0 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55addb16fc50 con 0x55addb264f00 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55addae9e250 con 0x55addb2645e0 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7af9ef740 10 client.0 _ll_register_callbacks cb 0x55addb1be290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb26dcd0 0x55addb26b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb2645e0 0x55addb26acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb264f00 0x55addb26b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb26dcd0 0x55addb26b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45790/0 (socket says 172.21.15.76:45790) 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 -- 172.21.15.76:0/1997464700 learned_addr learned my addr 172.21.15.76:0/1997464700 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 -- 172.21.15.76:0/1997464700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb2645e0 msgr2=0x55addb26acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb2645e0 0x55addb26acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 -- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb264f00 msgr2=0x55addb26b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb264f00 0x55addb26b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 -- 172.21.15.76:0/1997464700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a81a6b70 con 0x55addb26dcd0 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a659c640 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb2645e0 0x55addb26acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe79ffff640 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb264f00 0x55addb26b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb26dcd0 0x55addb26b750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0011cf0 tx=0x7fe7a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b5a97b8a3de576 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.471+0000 7fe78dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe78dffb640 1 -- 172.21.15.76:0/1997464700 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0016070 con 0x55addb26dcd0 2024-03-23T15:43:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe78dffb640 1 -- 172.21.15.76:0/1997464700 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe7a0003970 con 0x55addb26dcd0 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55addb1ef790 con 0x55addb26dcd0 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe78dffb640 1 -- 172.21.15.76:0/1997464700 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe7a000fe60 con 0x55addb26dcd0 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 10 client.15567 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55addb1c9310 con 0x55addb26dcd0 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe78dffb640 1 -- 172.21.15.76:0/1997464700 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a0027080 con 0x55addb26dcd0 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe78dffb640 1 -- 172.21.15.76:0/1997464700 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe7a0021030 con 0x55addb26dcd0 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 10 client.15567 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 20 client.15567 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb27d930 0x55addb27fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 4 client.15567 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55addaeca600 con 0x55addb27d930 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.472+0000 7fe7a659c640 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb27d930 0x55addb27fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.473+0000 7fe78dffb640 10 client.15567 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.473+0000 7fe7a659c640 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb27d930 0x55addb27fd50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e027bb052056fbb server_cookie=67f1dc08cf83a6d2 in_seq=0 out_seq=0 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.473+0000 7fe78dffb640 1 -- 172.21.15.76:0/1997464700 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55addaeca600 con 0x55addb27d930 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.473+0000 7fe78dffb640 10 client.15567 handle_command_reply: tid=0 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.473+0000 7fe7af9ef740 1 client.15567 shutdown 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.473+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb26dcd0 msgr2=0x55addb26b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.473+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb26dcd0 0x55addb26b750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0011cf0 tx=0x7fe7a000f790 comp rx=0 tx=0).stop 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 shutdown_connections 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb26dcd0 0x55addb26b750 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb27d930 0x55addb27fd50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb264f00 0x55addb26b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1997464700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb2645e0 0x55addb26acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 >> 172.21.15.76:0/1997464700 conn(0x55addb25ef00 msgr2=0x55addb260c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe78dffb640 0 client.15567 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 shutdown_connections 2024-03-23T15:43:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1997464700 wait complete. 2024-03-23T15:43:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.474+0000 7fe7af9ef740 20 client.15567 trim_cache size 0 max 16384 2024-03-23T15:43:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.488+0000 7fe7af9ef740 1 Processor -- start 2024-03-23T15:43:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.488+0000 7fe7af9ef740 1 -- start start 2024-03-23T15:43:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb26d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb2790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb26b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55addb150660 con 0x55addb2645e0 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55addb270480 con 0x55addb26dcd0 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55addb279610 con 0x55addb264f00 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb26d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb26d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45802/0 (socket says 172.21.15.76:45802) 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7a659c640 1 -- 172.21.15.76:0/1466889084 learned_addr learned my addr 172.21.15.76:0/1466889084 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe79ffff640 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb2790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb26b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe79ffff640 1 -- 172.21.15.76:0/1466889084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 msgr2=0x55addb26b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe79ffff640 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb26b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe79ffff640 1 -- 172.21.15.76:0/1466889084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 msgr2=0x55addb26d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe79ffff640 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb26d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe79ffff640 1 -- 172.21.15.76:0/1466889084 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55addb165810 con 0x55addb264f00 2024-03-23T15:43:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb26b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe7a659c640 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb26d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe79ffff640 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb2790d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe7900c5530 tx=0x7fe7900bc660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a2f5abbad419ab server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.489+0000 7fe78ffff640 1 -- 172.21.15.76:0/1466889084 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900c5370 con 0x55addb264f00 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe78ffff640 1 -- 172.21.15.76:0/1466889084 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7900bcbd0 con 0x55addb264f00 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe78ffff640 1 -- 172.21.15.76:0/1466889084 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900bc310 con 0x55addb264f00 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1466889084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 msgr2=0x55addb2790d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb2790d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe7900c5530 tx=0x7fe7900bc660 comp rx=0 tx=0).stop 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1466889084 shutdown_connections 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb26b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb2790d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/1466889084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb26d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1466889084 >> 172.21.15.76:0/1466889084 conn(0x55addb25ef00 msgr2=0x55addb25f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1466889084 shutdown_connections 2024-03-23T15:43:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.490+0000 7fe7af9ef740 1 -- 172.21.15.76:0/1466889084 wait complete. 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 -- start start 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 Processor -- start 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb275510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb275a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb270ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55addb271270 con 0x55addb2645e0 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55addb2713e0 con 0x55addb26dcd0 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55addb2715a0 con 0x55addb264f00 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7af9ef740 10 client.0 _ll_register_callbacks cb 0x7fe7a0009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb275510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb275510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45808/0 (socket says 172.21.15.76:45808) 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 -- 172.21.15.76:0/251959901 learned_addr learned my addr 172.21.15.76:0/251959901 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe79ffff640 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb275a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a6d9d640 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb270ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 -- 172.21.15.76:0/251959901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 msgr2=0x55addb275a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb275a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 -- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 msgr2=0x55addb270ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb270ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.491+0000 7fe7a659c640 1 -- 172.21.15.76:0/251959901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7980044d0 con 0x55addb2645e0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7a659c640 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb275510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe798003b00 tx=0x7fe798004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6da57ed561f6007 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe78dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe78dffb640 1 -- 172.21.15.76:0/251959901 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798022040 con 0x55addb2645e0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe78dffb640 1 -- 172.21.15.76:0/251959901 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe79802c730 con 0x55addb2645e0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7900c0d20 con 0x55addb2645e0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe78dffb640 1 -- 172.21.15.76:0/251959901 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe798035440 con 0x55addb2645e0 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 10 client.15573 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55addb1c9310 con 0x55addb2645e0 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe78dffb640 1 -- 172.21.15.76:0/251959901 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe798041080 con 0x55addb2645e0 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe78dffb640 1 -- 172.21.15.76:0/251959901 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe79803b030 con 0x55addb2645e0 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 10 client.15573 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 20 client.15573 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb2e40a0 0x55addb2e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 4 client.15573 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.492+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55addaf1b1b0 con 0x55addb2e40a0 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.493+0000 7fe79ffff640 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb2e40a0 0x55addb2e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.493+0000 7fe79ffff640 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb2e40a0 0x55addb2e64c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7abb6c9d62eabec server_cookie=5642767ef1da9cca in_seq=0 out_seq=0 2024-03-23T15:43:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.493+0000 7fe78dffb640 10 client.15573 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe78dffb640 1 -- 172.21.15.76:0/251959901 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55addaf1b1b0 con 0x55addb2e40a0 2024-03-23T15:43:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe78dffb640 10 client.15573 handle_command_reply: tid=0 2024-03-23T15:43:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 client.15573 shutdown 2024-03-23T15:43:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 msgr2=0x55addb275510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb275510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe798003b00 tx=0x7fe798004730 comp rx=0 tx=0).stop 2024-03-23T15:43:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 shutdown_connections 2024-03-23T15:43:41.505 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55addb2e40a0 0x55addb2e64c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55addb26dcd0 0x55addb270ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe78dffb640 0 client.15573 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55addb264f00 0x55addb275a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.502+0000 7fe7af9ef740 1 --2- 172.21.15.76:0/251959901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55addb2645e0 0x55addb275510 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 >> 172.21.15.76:0/251959901 conn(0x55addb25ef00 msgr2=0x55addb25d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 shutdown_connections 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7af9ef740 1 -- 172.21.15.76:0/251959901 wait complete. 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7af9ef740 20 client.15573 trim_cache size 0 max 16384 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:43:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:43:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:43:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:43:41.507 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe7940a23f0 msgr2=0x7fe7940a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe7940a23f0 0x7fe7940a48b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe798009f40 tx=0x7fe798005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8069730 msgr2=0x7fe7a8100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8069730 0x7fe7a8100e70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe790005940 tx=0x7fe79002c670 comp rx=0 tx=0).stop 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 shutdown_connections 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe7940a23f0 0x7fe7940a48b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a8107800 0x7fe7a81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7a8069730 0x7fe7a8100e70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 --2- 172.21.15.76:0/275316146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8068d90 0x7fe7a8100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.503+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 >> 172.21.15.76:0/275316146 conn(0x7fe7a8077460 msgr2=0x7fe7a80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.504+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 shutdown_connections 2024-03-23T15:43:41.508 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:41.504+0000 7fe7ace0e640 1 -- 172.21.15.76:0/275316146 wait complete. 2024-03-23T15:43:41.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:41 smithi192 ceph-mon[36653]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-03-23T15:43:41.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:41 smithi076 ceph-mon[32176]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-03-23T15:43:41.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:41 smithi183 ceph-mon[36296]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-03-23T15:43:43.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:43 smithi192 ceph-mon[36653]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-03-23T15:43:43.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:43 smithi076 ceph-mon[32176]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-03-23T15:43:43.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:43 smithi183 ceph-mon[36296]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-03-23T15:43:45.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:45 smithi192 ceph-mon[36653]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:45.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:45 smithi076 ceph-mon[32176]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:45.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:45 smithi183 ceph-mon[36296]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:47.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:47 smithi192 ceph-mon[36653]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:47.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:47 smithi076 ceph-mon[32176]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:47.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:47 smithi183 ceph-mon[36296]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:43:49.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:49 smithi192 ceph-mon[36653]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T15:43:49.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:49 smithi076 ceph-mon[32176]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T15:43:49.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:49 smithi183 ceph-mon[36296]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T15:43:51.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:51 smithi192 ceph-mon[36653]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:51.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:51 smithi076 ceph-mon[32176]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:51.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:51 smithi183 ceph-mon[36296]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:53.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:53 smithi192 ceph-mon[36653]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T15:43:53.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:53 smithi076 ceph-mon[32176]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T15:43:53.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:53 smithi183 ceph-mon[36296]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-23T15:43:54.630 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 -- 172.21.15.76:0/1759114755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81077f0 msgr2=0x7f3bf8107bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 --2- 172.21.15.76:0/1759114755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81077f0 0x7f3bf8107bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009900 tx=0x7f3be802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 -- 172.21.15.76:0/1759114755 shutdown_connections 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 --2- 172.21.15.76:0/1759114755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bf81001a0 0x7f3bf810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 --2- 172.21.15.76:0/1759114755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bf80ff7e0 0x7f3bf80ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 --2- 172.21.15.76:0/1759114755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81077f0 0x7f3bf8107bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 -- 172.21.15.76:0/1759114755 >> 172.21.15.76:0/1759114755 conn(0x7f3bf80767a0 msgr2=0x7f3bf80fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 -- 172.21.15.76:0/1759114755 shutdown_connections 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.627+0000 7f3bfe842640 1 -- 172.21.15.76:0/1759114755 wait complete. 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 Processor -- start 2024-03-23T15:43:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 -- start start 2024-03-23T15:43:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bf80ff7e0 0x7f3bf8101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81001a0 0x7f3bf8101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bf81077f0 0x7f3bf8101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf81a5f90 con 0x7f3bf80ff7e0 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf81a6100 con 0x7f3bf81001a0 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfe842640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf81a62c0 con 0x7f3bf81077f0 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bf80ff7e0 0x7f3bf8101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bf80ff7e0 0x7f3bf8101250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39014/0 (socket says 172.21.15.76:39014) 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfcddf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bf81077f0 0x7f3bf8101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81001a0 0x7f3bf8101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bf7fff640 1 -- 172.21.15.76:0/4215501763 learned_addr learned my addr 172.21.15.76:0/4215501763 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfcddf640 1 -- 172.21.15.76:0/4215501763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81001a0 msgr2=0x7f3bf8101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81001a0 0x7f3bf8101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfcddf640 1 -- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bf80ff7e0 msgr2=0x7f3bf8101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bf80ff7e0 0x7f3bf8101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bfcddf640 1 -- 172.21.15.76:0/4215501763 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be8009560 con 0x7f3bf81077f0 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.628+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81001a0 0x7f3bf8101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.629+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bf81077f0 0x7f3bf8101cd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009fd0 tx=0x7f3bf000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.629+0000 7f3bf57fa640 1 -- 172.21.15.76:0/4215501763 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0004d20 con 0x7f3bf81077f0 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.629+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf8197970 con 0x7f3bf81077f0 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.629+0000 7f3bf57fa640 1 -- 172.21.15.76:0/4215501763 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf0004e80 con 0x7f3bf81077f0 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.629+0000 7f3bf57fa640 1 -- 172.21.15.76:0/4215501763 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0005960 con 0x7f3bf81077f0 2024-03-23T15:43:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.629+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf8197e80 con 0x7f3bf81077f0 2024-03-23T15:43:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.631+0000 7f3bf57fa640 1 -- 172.21.15.76:0/4215501763 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f3bf0005020 con 0x7f3bf81077f0 2024-03-23T15:43:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.631+0000 7f3bf57fa640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3be00a23f0 0x7f3be00a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.632+0000 7f3bf57fa640 1 -- 172.21.15.76:0/4215501763 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3bf00d6ea0 con 0x7f3bf81077f0 2024-03-23T15:43:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.632+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3be00a23f0 0x7f3be00a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.632+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3be00a23f0 0x7f3be00a48b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3be8005ec0 tx=0x7f3be8005a20 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.632+0000 7f3c0141a740 1 Processor -- start 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.632+0000 7f3c0141a740 1 -- start start 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b83f00 0x5621c4b8c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3c0141a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5621c4b3e1b0 con 0x5621c4b835e0 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3c0141a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621c4af0f80 con 0x5621c4b8ccd0 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3c0141a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5621c4aee4b0 con 0x5621c4b83f00 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b839c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39020/0 (socket says 172.21.15.76:39020) 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 -- 172.21.15.76:0/3385653246 learned_addr learned my addr 172.21.15.76:0/3385653246 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b83f00 0x5621c4b8c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 -- 172.21.15.76:0/3385653246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b83f00 msgr2=0x5621c4b8c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b83f00 0x5621c4b8c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 -- 172.21.15.76:0/3385653246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 msgr2=0x5621c4b8f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 -- 172.21.15.76:0/3385653246 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c4a7f810 con 0x5621c4b835e0 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b839c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3be8004a00 tx=0x7f3be803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1f43660088022a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.633+0000 7f3bde7fc640 1 -- 172.21.15.76:0/3385653246 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be802c920 con 0x5621c4b835e0 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3bde7fc640 1 -- 172.21.15.76:0/3385653246 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be802ca80 con 0x5621c4b835e0 2024-03-23T15:43:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3bde7fc640 1 -- 172.21.15.76:0/3385653246 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be80035f0 con 0x5621c4b835e0 2024-03-23T15:43:54.640 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 -- 172.21.15.76:0/3385653246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 msgr2=0x5621c4b839c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.640 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b839c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3be8004a00 tx=0x7f3be803f000 comp rx=0 tx=0).stop 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 -- 172.21.15.76:0/3385653246 shutdown_connections 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b83f00 0x5621c4b8c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3385653246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b839c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 -- 172.21.15.76:0/3385653246 >> 172.21.15.76:0/3385653246 conn(0x5621c4b7df00 msgr2=0x5621c4b7e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 -- 172.21.15.76:0/3385653246 shutdown_connections 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.634+0000 7f3c0141a740 1 -- 172.21.15.76:0/3385653246 wait complete. 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 -- start start 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 Processor -- start 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b89cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b8a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5621c4ae9c90 con 0x5621c4b83f00 2024-03-23T15:43:54.641 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621c4a8ec50 con 0x5621c4b8ccd0 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5621c47bd250 con 0x5621c4b835e0 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3c0141a740 10 client.0 _ll_register_callbacks cb 0x5621c4add290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3bf77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b8a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b89cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3bf77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b8a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39032/0 (socket says 172.21.15.76:39032) 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3211031028 learned_addr learned my addr 172.21.15.76:0/3211031028 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.635+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3211031028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 msgr2=0x5621c4b89cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b89cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 msgr2=0x5621c4b8a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3211031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf810ee10 con 0x5621c4b83f00 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b89cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b8a210 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bec012b10 tx=0x7f3bec00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87b9e19f5a6c21f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bd77fe640 1 -- 172.21.15.76:0/3211031028 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec017070 con 0x5621c4b83f00 2024-03-23T15:43:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bd77fe640 1 -- 172.21.15.76:0/3211031028 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3bec00fb40 con 0x5621c4b83f00 2024-03-23T15:43:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621c4b0e790 con 0x5621c4b83f00 2024-03-23T15:43:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.636+0000 7f3bd77fe640 1 -- 172.21.15.76:0/3211031028 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3bec00fd40 con 0x5621c4b83f00 2024-03-23T15:43:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 10 client.15585 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5621c4ae8310 con 0x5621c4b83f00 2024-03-23T15:43:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3bd77fe640 1 -- 172.21.15.76:0/3211031028 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5621c4b0e790 con 0x5621c4b83f00 2024-03-23T15:43:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3bd77fe640 1 -- 172.21.15.76:0/3211031028 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3bec010430 con 0x5621c4b83f00 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 10 client.15585 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 20 client.15585 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b9cb10 0x5621c4b9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 4 client.15585 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.638+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621c47e9600 con 0x5621c4b9cb10 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.639+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b9cb10 0x5621c4b9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.639+0000 7f3bd77fe640 10 client.15585 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.639+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b9cb10 0x5621c4b9ef30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcee238b390a1bef server_cookie=dd59bfb4690e3b37 in_seq=0 out_seq=0 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3bd77fe640 1 -- 172.21.15.76:0/3211031028 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5621c47e9600 con 0x5621c4b9cb10 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3bd77fe640 10 client.15585 handle_command_reply: tid=0 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 client.15585 shutdown 2024-03-23T15:43:54.646 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 msgr2=0x5621c4b8a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b8a210 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bec012b10 tx=0x7f3bec00f3c0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 shutdown_connections 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b9cb10 0x5621c4b9ef30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b8ccd0 0x5621c4b8a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b8a210 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3211031028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b89cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3bd77fe640 0 client.15585 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 >> 172.21.15.76:0/3211031028 conn(0x5621c4b7df00 msgr2=0x5621c4b7fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 shutdown_connections 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 1 -- 172.21.15.76:0/3211031028 wait complete. 2024-03-23T15:43:54.647 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.640+0000 7f3c0141a740 20 client.15585 trim_cache size 0 max 16384 2024-03-23T15:43:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.655+0000 7f3c0141a740 1 Processor -- start 2024-03-23T15:43:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.655+0000 7f3c0141a740 1 -- start start 2024-03-23T15:43:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.655+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b7df00 0x5621c4b8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.655+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b7cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.659 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b7df00 0x5621c4b8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b98220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41712/0 (socket says 172.21.15.76:41712) 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3462501700 learned_addr learned my addr 172.21.15.76:0/3462501700 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5621c4b84b30 con 0x5621c4b83f00 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621c4a6f660 con 0x5621c4b7df00 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5621c4b82fd0 con 0x5621c4b835e0 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b7cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3462501700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b7df00 msgr2=0x5621c4b8c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b7df00 0x5621c4b8c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3462501700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 msgr2=0x5621c4b7cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b7cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 -- 172.21.15.76:0/3462501700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c4a84810 con 0x5621c4b835e0 2024-03-23T15:43:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.656+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b98220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bec000c00 tx=0x7f3bec00f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eded95ab5a52918 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3bddffb640 1 -- 172.21.15.76:0/3462501700 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec017070 con 0x5621c4b835e0 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3bddffb640 1 -- 172.21.15.76:0/3462501700 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bec005d30 con 0x5621c4b835e0 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3bddffb640 1 -- 172.21.15.76:0/3462501700 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0054d0 con 0x5621c4b835e0 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 msgr2=0x5621c4b98220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b98220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bec000c00 tx=0x7f3bec00f940 comp rx=0 tx=0).stop 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 shutdown_connections 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b83f00 0x5621c4b7cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b835e0 0x5621c4b98220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3c0141a740 1 --2- 172.21.15.76:0/3462501700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b7df00 0x5621c4b8c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.657+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 >> 172.21.15.76:0/3462501700 conn(0x5621c4b78ec0 msgr2=0x5621c4b7c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 shutdown_connections 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 -- 172.21.15.76:0/3462501700 wait complete. 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 -- start start 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 Processor -- start 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b7df00 0x5621c4b90820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b90d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b83f00 0x5621c4c01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5621c4b914a0 con 0x5621c4b835e0 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621c4c02390 con 0x5621c4b83f00 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5621c4c02520 con 0x5621c4b7df00 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.658+0000 7f3c0141a740 10 client.0 _ll_register_callbacks cb 0x7f3bec009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b83f00 0x5621c4c01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b83f00 0x5621c4c01e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59286/0 (socket says 172.21.15.76:59286) 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 -- 172.21.15.76:0/1207071386 learned_addr learned my addr 172.21.15.76:0/1207071386 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b90d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b7df00 0x5621c4b90820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 -- 172.21.15.76:0/1207071386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b7df00 msgr2=0x5621c4b90820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b7df00 0x5621c4b90820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 -- 172.21.15.76:0/1207071386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 msgr2=0x5621c4b90d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b90d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 -- 172.21.15.76:0/1207071386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c4ae8310 con 0x5621c4b83f00 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bf77fe640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b90d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bfcddf640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b83f00 0x5621c4c01e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00a00d0 tx=0x7f3bf00d6040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c0c18a8ac5a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bd77fe640 1 -- 172.21.15.76:0/1207071386 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00c32b0 con 0x5621c4b83f00 2024-03-23T15:43:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bd77fe640 1 -- 172.21.15.76:0/1207071386 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3bf00a78a0 con 0x5621c4b83f00 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be802ca80 con 0x5621c4b83f00 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3bd77fe640 1 -- 172.21.15.76:0/1207071386 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3bf00bc2a0 con 0x5621c4b83f00 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.659+0000 7f3c0141a740 10 client.25376 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5621c4aaa5a0 con 0x5621c4b83f00 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3bd77fe640 1 -- 172.21.15.76:0/1207071386 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf0086080 con 0x5621c4b83f00 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3bd77fe640 1 -- 172.21.15.76:0/1207071386 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3bf0026060 con 0x5621c4b83f00 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3c0141a740 10 client.25376 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3c0141a740 10 client.25376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3c0141a740 20 client.25376 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3c0141a740 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b8ccd0 0x5621c4c05d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3c0141a740 4 client.25376 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621c483a1b0 con 0x5621c4b8ccd0 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b8ccd0 0x5621c4c05d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3bd77fe640 10 client.25376 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.660+0000 7f3bf7fff640 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b8ccd0 0x5621c4c05d40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1813e12522949e server_cookie=f2bf973a35c8718c in_seq=0 out_seq=0 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.663+0000 7f3bd77fe640 1 -- 172.21.15.76:0/1207071386 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621c483a1b0 con 0x5621c4b8ccd0 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.663+0000 7f3bd77fe640 10 client.25376 handle_command_reply: tid=0 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.663+0000 7f3c0141a740 1 client.25376 shutdown 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.663+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b83f00 msgr2=0x5621c4c01e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.663+0000 7f3c0141a740 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b83f00 0x5621c4c01e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00a00d0 tx=0x7f3bf00d6040 comp rx=0 tx=0).stop 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 shutdown_connections 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5621c4b8ccd0 0x5621c4c05d40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621c4b83f00 0x5621c4c01e50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5621c4b835e0 0x5621c4b90d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 --2- 172.21.15.76:0/1207071386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621c4b7df00 0x5621c4b90820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3bd77fe640 0 client.25376 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 >> 172.21.15.76:0/1207071386 conn(0x5621c4b78ec0 msgr2=0x5621c4b7cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 shutdown_connections 2024-03-23T15:43:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 1 -- 172.21.15.76:0/1207071386 wait complete. 2024-03-23T15:43:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3c0141a740 20 client.25376 trim_cache size 0 max 16384 2024-03-23T15:43:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:43:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:43:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:43:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:43:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3be00a23f0 msgr2=0x7f3be00a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3bfe842640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3be00a23f0 0x7f3be00a48b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3be8005ec0 tx=0x7f3be8005a20 comp rx=0 tx=0).stop 2024-03-23T15:43:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bf81077f0 msgr2=0x7f3bf8101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.664+0000 7f3bfe842640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bf81077f0 0x7f3bf8101cd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009fd0 tx=0x7f3bf000ef10 comp rx=0 tx=0).stop 2024-03-23T15:43:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 shutdown_connections 2024-03-23T15:43:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3be00a23f0 0x7f3be00a48b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bf81077f0 0x7f3bf8101cd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bf81001a0 0x7f3bf8101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 --2- 172.21.15.76:0/4215501763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3bf80ff7e0 0x7f3bf8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 >> 172.21.15.76:0/4215501763 conn(0x7f3bf80767a0 msgr2=0x7f3bf80fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 shutdown_connections 2024-03-23T15:43:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:43:54.665+0000 7f3bfe842640 1 -- 172.21.15.76:0/4215501763 wait complete. 2024-03-23T15:43:55.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:55 smithi192 ceph-mon[36653]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:55.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:55 smithi076 ceph-mon[32176]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:55.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:55 smithi183 ceph-mon[36296]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:57.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T15:43:57.088 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T15:43:57.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 -- 172.21.15.76:0/3333999311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 msgr2=0x7f4694107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 --2- 172.21.15.76:0/3333999311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 0x7f4694107a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f469002bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:57.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 -- 172.21.15.76:0/3333999311 shutdown_connections 2024-03-23T15:43:57.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 --2- 172.21.15.76:0/3333999311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 0x7f469410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 --2- 172.21.15.76:0/3333999311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46940688f0 0x7f4694068d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 --2- 172.21.15.76:0/3333999311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 0x7f4694107a70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 -- 172.21.15.76:0/3333999311 >> 172.21.15.76:0/3333999311 conn(0x7f4694077710 msgr2=0x7f4694077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 -- 172.21.15.76:0/3333999311 shutdown_connections 2024-03-23T15:43:57.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.196+0000 7f469c60e640 1 -- 172.21.15.76:0/3333999311 wait complete. 2024-03-23T15:43:57.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 Processor -- start 2024-03-23T15:43:57.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 -- start start 2024-03-23T15:43:57.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46940688f0 0x7f4694197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 0x7f4694198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 0x7f469419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f469419cc00 con 0x7f46940688f0 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f469419cd70 con 0x7f4694107690 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469c60e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f469419cf30 con 0x7f46940692b0 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469a3aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46940688f0 0x7f4694197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 0x7f4694198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 0x7f4694198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41746/0 (socket says 172.21.15.76:41746) 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 -- 172.21.15.76:0/2737000658 learned_addr learned my addr 172.21.15.76:0/2737000658 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f469abab640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 0x7f469419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 -- 172.21.15.76:0/2737000658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 msgr2=0x7f469419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 0x7f469419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 -- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46940688f0 msgr2=0x7f4694197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46940688f0 0x7f4694197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.197+0000 7f4699ba9640 1 -- 172.21.15.76:0/2737000658 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c009840 con 0x7f46940692b0 2024-03-23T15:43:57.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.198+0000 7f4699ba9640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 0x7f4694198280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f468400e9f0 tx=0x7f468400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.198+0000 7f468b7fe640 1 -- 172.21.15.76:0/2737000658 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684009700 con 0x7f46940692b0 2024-03-23T15:43:57.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.198+0000 7f468b7fe640 1 -- 172.21.15.76:0/2737000658 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468400ccc0 con 0x7f46940692b0 2024-03-23T15:43:57.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.198+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4690009560 con 0x7f46940692b0 2024-03-23T15:43:57.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.199+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a48b0 con 0x7f46940692b0 2024-03-23T15:43:57.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.199+0000 7f468b7fe640 1 -- 172.21.15.76:0/2737000658 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684018820 con 0x7f46940692b0 2024-03-23T15:43:57.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.201+0000 7f468b7fe640 1 -- 172.21.15.76:0/2737000658 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f4684010430 con 0x7f46940692b0 2024-03-23T15:43:57.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 Processor -- start 2024-03-23T15:43:57.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 -- start start 2024-03-23T15:43:57.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f468b7fe640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f46800a2350 0x7f46800a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda45b1d0 0x555eda45b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda464190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda4646d0 0x555eda466ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555eda37ff30 con 0x555eda4646d0 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555eda3c9810 con 0x555eda45b1d0 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469f1ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555eda3c6d40 con 0x555eda45baf0 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469abab640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f46800a2350 0x7f46800a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda464190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda464190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41756/0 (socket says 172.21.15.76:41756) 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f4699ba9640 1 -- 172.21.15.76:0/847268504 learned_addr learned my addr 172.21.15.76:0/847268504 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.202+0000 7f469abab640 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda4646d0 0x555eda466ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4699ba9640 1 -- 172.21.15.76:0/847268504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda45b1d0 msgr2=0x555eda45b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4699ba9640 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda45b1d0 0x555eda45b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4699ba9640 1 -- 172.21.15.76:0/847268504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda4646d0 msgr2=0x555eda466ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4699ba9640 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda4646d0 0x555eda466ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4699ba9640 1 -- 172.21.15.76:0/847268504 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eda383ea0 con 0x555eda45baf0 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4699ba9640 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda464190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46840a31a0 tx=0x7f468406a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8928a46e71e66bad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f468b7fe640 1 -- 172.21.15.76:0/2737000658 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4684014070 con 0x7f46940692b0 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f469abab640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f46800a2350 0x7f46800a4810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f468c005fd0 tx=0x7f468c0114d0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4688bfe640 1 -- 172.21.15.76:0/847268504 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684065c10 con 0x555eda45baf0 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4688bfe640 1 -- 172.21.15.76:0/847268504 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4684010a10 con 0x555eda45baf0 2024-03-23T15:43:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.203+0000 7f4688bfe640 1 -- 172.21.15.76:0/847268504 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840ad400 con 0x555eda45baf0 2024-03-23T15:43:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 -- 172.21.15.76:0/847268504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 msgr2=0x555eda464190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda464190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46840a31a0 tx=0x7f468406a0d0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 -- 172.21.15.76:0/847268504 shutdown_connections 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda4646d0 0x555eda466ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda464190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 --2- 172.21.15.76:0/847268504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda45b1d0 0x555eda45b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 -- 172.21.15.76:0/847268504 >> 172.21.15.76:0/847268504 conn(0x555eda455900 msgr2=0x555eda455ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 -- 172.21.15.76:0/847268504 shutdown_connections 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 -- 172.21.15.76:0/847268504 wait complete. 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 -- start start 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.204+0000 7f469f1ec740 1 Processor -- start 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda461710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda461c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda462190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469f1ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555eda3c2520 con 0x555eda45b1d0 2024-03-23T15:43:57.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469f1ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555eda095430 con 0x555eda4646d0 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469f1ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555eda051e00 con 0x555eda45baf0 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469f1ec740 10 client.0 _ll_register_callbacks cb 0x555eda360d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda461c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda461c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41768/0 (socket says 172.21.15.76:41768) 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469a3aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda461710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 -- 172.21.15.76:0/3042966874 learned_addr learned my addr 172.21.15.76:0/3042966874 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f469abab640 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda462190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 -- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 msgr2=0x555eda462190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda462190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 -- 172.21.15.76:0/3042966874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 msgr2=0x555eda461710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda461710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 -- 172.21.15.76:0/3042966874 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a48b0 con 0x555eda45baf0 2024-03-23T15:43:57.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f4699ba9640 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda461c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4684097390 tx=0x7f4684010c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7649daf9fc6ac77 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f467dffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.205+0000 7f467dffb640 1 -- 172.21.15.76:0/3042966874 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468409ad00 con 0x555eda45baf0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555eda0c17e0 con 0x555eda45baf0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f467dffb640 1 -- 172.21.15.76:0/3042966874 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f46840ad3e0 con 0x555eda45baf0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f467dffb640 1 -- 172.21.15.76:0/3042966874 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f468409b630 con 0x555eda45baf0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 10 client.25405 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555eda3c0ba0 con 0x555eda45baf0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f467dffb640 1 -- 172.21.15.76:0/3042966874 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555eda0c17e0 con 0x555eda45baf0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f467dffb640 1 -- 172.21.15.76:0/3042966874 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f46840cc040 con 0x555eda45baf0 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 10 client.25405 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 20 client.25405 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda478ab0 0x555eda47aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 4 client.25405 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eda0fe720 con 0x555eda478ab0 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.206+0000 7f469a3aa640 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda478ab0 0x555eda47aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.207+0000 7f469a3aa640 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda478ab0 0x555eda47aef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7eedc6b65d9836f server_cookie=eefda02fde4ca528 in_seq=0 out_seq=0 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.207+0000 7f467dffb640 10 client.25405 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.207+0000 7f467dffb640 1 -- 172.21.15.76:0/3042966874 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555eda0fe720 con 0x555eda478ab0 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.207+0000 7f467dffb640 10 client.25405 handle_command_reply: tid=0 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.207+0000 7f469f1ec740 1 client.25405 shutdown 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 msgr2=0x555eda461c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda461c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4684097390 tx=0x7f4684010c00 comp rx=0 tx=0).stop 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 shutdown_connections 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda478ab0 0x555eda47aef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda462190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda461c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 --2- 172.21.15.76:0/3042966874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda461710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 >> 172.21.15.76:0/3042966874 conn(0x555eda455900 msgr2=0x555eda457650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f467dffb640 0 client.25405 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:57.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 shutdown_connections 2024-03-23T15:43:57.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 1 -- 172.21.15.76:0/3042966874 wait complete. 2024-03-23T15:43:57.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.208+0000 7f469f1ec740 20 client.25405 trim_cache size 0 max 16384 2024-03-23T15:43:57.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 Processor -- start 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 -- start start 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda463dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda460870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda46b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555eda347430 con 0x555eda45b1d0 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555eda461500 con 0x555eda4646d0 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469f1ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555eda4474d0 con 0x555eda45baf0 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469a3aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda463dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda460870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469a3aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda463dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39088/0 (socket says 172.21.15.76:39088) 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469a3aa640 1 -- 172.21.15.76:0/4130691766 learned_addr learned my addr 172.21.15.76:0/4130691766 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f4699ba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda460870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41780/0 (socket says 172.21.15.76:41780) 2024-03-23T15:43:57.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.225+0000 7f469abab640 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda46b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f4699ba9640 1 -- 172.21.15.76:0/4130691766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 msgr2=0x555eda46b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f4699ba9640 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda46b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f4699ba9640 1 -- 172.21.15.76:0/4130691766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 msgr2=0x555eda463dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f4699ba9640 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda463dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f4699ba9640 1 -- 172.21.15.76:0/4130691766 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eda35cb20 con 0x555eda45baf0 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f4699ba9640 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda460870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f468409aec0 tx=0x7f4684010c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a179535fc586cd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f467ffff640 1 -- 172.21.15.76:0/4130691766 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468406e4f0 con 0x555eda45baf0 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f467ffff640 1 -- 172.21.15.76:0/4130691766 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468406b040 con 0x555eda45baf0 2024-03-23T15:43:57.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.226+0000 7f467ffff640 1 -- 172.21.15.76:0/4130691766 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840cbca0 con 0x555eda45baf0 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 -- 172.21.15.76:0/4130691766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 msgr2=0x555eda460870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda460870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f468409aec0 tx=0x7f4684010c70 comp rx=0 tx=0).stop 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 -- 172.21.15.76:0/4130691766 shutdown_connections 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda46b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda460870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 --2- 172.21.15.76:0/4130691766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda463dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 -- 172.21.15.76:0/4130691766 >> 172.21.15.76:0/4130691766 conn(0x555eda455900 msgr2=0x555eda4542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 -- 172.21.15.76:0/4130691766 shutdown_connections 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 -- 172.21.15.76:0/4130691766 wait complete. 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 -- start start 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.227+0000 7f469f1ec740 1 Processor -- start 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda4675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda467b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda468060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555eda46b4c0 con 0x555eda45b1d0 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555eda46f8d0 con 0x555eda4646d0 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555eda46fa90 con 0x555eda45baf0 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 10 client.0 _ll_register_callbacks cb 0x7f468c0118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda4675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda4675e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39096/0 (socket says 172.21.15.76:39096) 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 -- 172.21.15.76:0/1009284740 learned_addr learned my addr 172.21.15.76:0/1009284740 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f4699ba9640 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda467b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469abab640 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda468060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 -- 172.21.15.76:0/1009284740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 msgr2=0x555eda467b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda467b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 -- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 msgr2=0x555eda468060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda468060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 -- 172.21.15.76:0/1009284740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eda3c0ba0 con 0x555eda45b1d0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469a3aa640 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda4675e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f469002cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc49929553534be server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f467dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f467dffb640 1 -- 172.21.15.76:0/1009284740 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690045040 con 0x555eda45b1d0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f467dffb640 1 -- 172.21.15.76:0/1009284740 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4690004990 con 0x555eda45b1d0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.228+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4684010a10 con 0x555eda45b1d0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f467dffb640 1 -- 172.21.15.76:0/1009284740 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4690047a30 con 0x555eda45b1d0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 10 client.15609 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555eda382700 con 0x555eda45b1d0 2024-03-23T15:43:57.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f467dffb640 1 -- 172.21.15.76:0/1009284740 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4684010a10 con 0x555eda45b1d0 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f467dffb640 1 -- 172.21.15.76:0/1009284740 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f469003c040 con 0x555eda45b1d0 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 10 client.15609 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 20 client.15609 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda4e3690 0x555eda4e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 4 client.15609 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555eda14d890 con 0x555eda4e3690 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.229+0000 7f4699ba9640 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda4e3690 0x555eda4e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f467dffb640 10 client.15609 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f4699ba9640 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda4e3690 0x555eda4e5ab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e74a9aee08ff741 server_cookie=477468d5b94cde1b in_seq=0 out_seq=0 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f467dffb640 1 -- 172.21.15.76:0/1009284740 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555eda14d890 con 0x555eda4e3690 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f467dffb640 10 client.15609 handle_command_reply: tid=0 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 client.15609 shutdown 2024-03-23T15:43:57.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 msgr2=0x555eda4675e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda4675e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f469002cc40 comp rx=0 tx=0).stop 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 shutdown_connections 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x555eda4e3690 0x555eda4e5ab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555eda4646d0 0x555eda468060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f467dffb640 0 client.15609 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eda45baf0 0x555eda467b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 --2- 172.21.15.76:0/1009284740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555eda45b1d0 0x555eda4675e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 >> 172.21.15.76:0/1009284740 conn(0x555eda455900 msgr2=0x555eda461640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 shutdown_connections 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 1 -- 172.21.15.76:0/1009284740 wait complete. 2024-03-23T15:43:57.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.230+0000 7f469f1ec740 20 client.15609 trim_cache size 0 max 16384 2024-03-23T15:43:57.240 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:57.240 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-03-23T15:43:57.240 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-03-23T15:43:57.240 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:57.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f46800a2350 msgr2=0x7f46800a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f46800a2350 0x7f46800a4810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f468c005fd0 tx=0x7f468c0114d0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 msgr2=0x7f4694198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 0x7f4694198280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f468400e9f0 tx=0x7f468400edf0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 shutdown_connections 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f46800a2350 0x7f46800a4810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4694107690 0x7f469419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46940692b0 0x7f4694198280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 --2- 172.21.15.76:0/2737000658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46940688f0 0x7f4694197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 >> 172.21.15.76:0/2737000658 conn(0x7f4694077710 msgr2=0x7f46940fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 shutdown_connections 2024-03-23T15:43:57.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.231+0000 7f469c60e640 1 -- 172.21.15.76:0/2737000658 wait complete. 2024-03-23T15:43:57.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T15:43:57.246 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:43:57.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.348+0000 7f6045cb5640 1 -- 172.21.15.76:0/58168464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604010e0b0 msgr2=0x7f60401104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.348+0000 7f6045cb5640 1 --2- 172.21.15.76:0/58168464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604010e0b0 0x7f60401104a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6038009ca0 tx=0x7f603802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 -- 172.21.15.76:0/58168464 shutdown_connections 2024-03-23T15:43:57.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 --2- 172.21.15.76:0/58168464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604010e0b0 0x7f60401104a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 --2- 172.21.15.76:0/58168464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6040100eb0 0x7f604010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 --2- 172.21.15.76:0/58168464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6040100500 0x7f60401008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 -- 172.21.15.76:0/58168464 >> 172.21.15.76:0/58168464 conn(0x7f6040077710 msgr2=0x7f6040077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 -- 172.21.15.76:0/58168464 shutdown_connections 2024-03-23T15:43:57.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 -- 172.21.15.76:0/58168464 wait complete. 2024-03-23T15:43:57.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.349+0000 7f6045cb5640 1 Processor -- start 2024-03-23T15:43:57.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f6045cb5640 1 -- start start 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f6045cb5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6040100500 0x7f6040199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f6045cb5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040100eb0 0x7f604019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f6045cb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604010e0b0 0x7f604019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f6045cb5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f604019ee30 con 0x7f604010e0b0 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040100eb0 0x7f604019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6040100500 0x7f6040199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f6045cb5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f604019efa0 con 0x7f6040100eb0 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f6045cb5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f604019f160 con 0x7f6040100500 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040100eb0 0x7f604019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59384/0 (socket says 172.21.15.76:59384) 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 -- 172.21.15.76:0/3111012880 learned_addr learned my addr 172.21.15.76:0/3111012880 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603ffff640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604010e0b0 0x7f604019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 -- 172.21.15.76:0/3111012880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6040100500 msgr2=0x7f6040199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6040100500 0x7f6040199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 -- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604010e0b0 msgr2=0x7f604019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604010e0b0 0x7f604019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 -- 172.21.15.76:0/3111012880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6038009900 con 0x7f6040100eb0 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.350+0000 7f603effd640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040100eb0 0x7f604019a4b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f602800e9f0 tx=0x7f602800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.351+0000 7f603cff9640 1 -- 172.21.15.76:0/3111012880 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028009720 con 0x7f6040100eb0 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.351+0000 7f603cff9640 1 -- 172.21.15.76:0/3111012880 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602800cd80 con 0x7f6040100eb0 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.351+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604019f440 con 0x7f6040100eb0 2024-03-23T15:43:57.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.351+0000 7f603cff9640 1 -- 172.21.15.76:0/3111012880 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028010690 con 0x7f6040100eb0 2024-03-23T15:43:57.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.351+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401a6cc0 con 0x7f6040100eb0 2024-03-23T15:43:57.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.352+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6040068500 con 0x7f6040100eb0 2024-03-23T15:43:57.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.353+0000 7f603cff9640 1 -- 172.21.15.76:0/3111012880 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f60280107f0 con 0x7f6040100eb0 2024-03-23T15:43:57.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.354+0000 7f603cff9640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f602c0a23d0 0x7f602c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.354+0000 7f603cff9640 1 -- 172.21.15.76:0/3111012880 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6028014070 con 0x7f6040100eb0 2024-03-23T15:43:57.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.354+0000 7f603f7fe640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f602c0a23d0 0x7f602c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.355+0000 7f603f7fe640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f602c0a23d0 0x7f602c0a4890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6030009bd0 tx=0x7f6030005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.359+0000 7f603cff9640 1 -- 172.21.15.76:0/3111012880 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6028096ff0 con 0x7f6040100eb0 2024-03-23T15:43:57.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:57 smithi192 ceph-mon[36653]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:57.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:57 smithi076 ceph-mon[32176]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:57.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.581+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6040109240 con 0x7f6040100eb0 2024-03-23T15:43:57.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.581+0000 7f603cff9640 1 -- 172.21.15.76:0/3111012880 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11872 (secure 0 0 0) 0x7f602809a6b0 con 0x7f6040100eb0 2024-03-23T15:43:57.585 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:43:57.585 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:43:18.737936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24463,24449]},"id":1}]} 2024-03-23T15:43:57.585 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 22 2024-03-23T15:43:57.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f602c0a23d0 msgr2=0x7f602c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f602c0a23d0 0x7f602c0a4890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6030009bd0 tx=0x7f6030005f90 comp rx=0 tx=0).stop 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040100eb0 msgr2=0x7f604019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040100eb0 0x7f604019a4b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f602800e9f0 tx=0x7f602800edf0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 shutdown_connections 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f602c0a23d0 0x7f602c0a4890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604010e0b0 0x7f604019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040100eb0 0x7f604019a4b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 --2- 172.21.15.76:0/3111012880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6040100500 0x7f6040199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 >> 172.21.15.76:0/3111012880 conn(0x7f6040077710 msgr2=0x7f604010f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 shutdown_connections 2024-03-23T15:43:57.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.584+0000 7f6045cb5640 1 -- 172.21.15.76:0/3111012880 wait complete. 2024-03-23T15:43:57.602 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T15:43:57.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:57 smithi183 ceph-mon[36296]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-03-23T15:43:57.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 -- 172.21.15.76:0/2202171541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c80692d0 msgr2=0x7f08c810e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 --2- 172.21.15.76:0/2202171541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c80692d0 0x7f08c810e570 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009cd0 tx=0x7f08c002c0f0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 -- 172.21.15.76:0/2202171541 shutdown_connections 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 --2- 172.21.15.76:0/2202171541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c80692d0 0x7f08c810e570 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 --2- 172.21.15.76:0/2202171541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08c8068910 0x7f08c8068d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 --2- 172.21.15.76:0/2202171541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08c8107760 0x7f08c8107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 -- 172.21.15.76:0/2202171541 >> 172.21.15.76:0/2202171541 conn(0x7f08c8077710 msgr2=0x7f08c8077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 -- 172.21.15.76:0/2202171541 shutdown_connections 2024-03-23T15:43:57.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.722+0000 7f08cce72640 1 -- 172.21.15.76:0/2202171541 wait complete. 2024-03-23T15:43:57.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 Processor -- start 2024-03-23T15:43:57.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 -- start start 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c8068910 0x7f08c819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08c80692d0 0x7f08c819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08c8107760 0x7f08c8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8198120 con 0x7f08c8068910 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8198290 con 0x7f08c8107760 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08cce72640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8198450 con 0x7f08c80692d0 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08c659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c8068910 0x7f08c819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08c659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c8068910 0x7f08c819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39116/0 (socket says 172.21.15.76:39116) 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08c659c640 1 -- 172.21.15.76:0/975772076 learned_addr learned my addr 172.21.15.76:0/975772076 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.723+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08c8107760 0x7f08c8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08c80692d0 0x7f08c819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c659c640 1 -- 172.21.15.76:0/975772076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08c80692d0 msgr2=0x7f08c819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c659c640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08c80692d0 0x7f08c819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c659c640 1 -- 172.21.15.76:0/975772076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08c8107760 msgr2=0x7f08c8197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c659c640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08c8107760 0x7f08c8197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c659c640 1 -- 172.21.15.76:0/975772076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b8009490 con 0x7f08c8068910 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08c8107760 0x7f08c8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08c659c640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c8068910 0x7f08c819d920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f08b800ea60 tx=0x7f08b800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08b77fe640 1 -- 172.21.15.76:0/975772076 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800cde0 con 0x7f08c8068910 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08b77fe640 1 -- 172.21.15.76:0/975772076 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b80035f0 con 0x7f08c8068910 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c0009900 con 0x7f08c8068910 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08b77fe640 1 -- 172.21.15.76:0/975772076 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8010690 con 0x7f08c8068910 2024-03-23T15:43:57.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.724+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c81a47e0 con 0x7f08c8068910 2024-03-23T15:43:57.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 Processor -- start 2024-03-23T15:43:57.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 -- start start 2024-03-23T15:43:57.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da831637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da8316ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da830d1840 con 0x55da83163d10 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da830ced70 con 0x55da8316c8f0 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55da830ca550 con 0x55da831633f0 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08c6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da831637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08c6d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da831637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41832/0 (socket says 172.21.15.76:41832) 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.727+0000 7f08c6d9d640 1 -- 172.21.15.76:0/2415953734 learned_addr learned my addr 172.21.15.76:0/2415953734 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da8316ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08b77fe640 1 -- 172.21.15.76:0/975772076 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f08b8003af0 con 0x7f08c8068910 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c5d9b640 1 -- 172.21.15.76:0/2415953734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 msgr2=0x55da831637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da831637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c5d9b640 1 -- 172.21.15.76:0/2415953734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 msgr2=0x55da8316c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c5d9b640 1 -- 172.21.15.76:0/2415953734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da8305fde0 con 0x55da8316c8f0 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da831637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da8316ece0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08c8198c10 tx=0x7f08bc0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77eec30058cf62e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08b4ff9640 1 -- 172.21.15.76:0/2415953734 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc011de0 con 0x55da8316c8f0 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08b4ff9640 1 -- 172.21.15.76:0/2415953734 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc012930 con 0x55da8316c8f0 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08b4ff9640 1 -- 172.21.15.76:0/2415953734 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00fa30 con 0x55da8316c8f0 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.728+0000 7f08b77fe640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f08b00a2350 0x7f08b00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08b77fe640 1 -- 172.21.15.76:0/975772076 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f08b8014070 con 0x7f08c8068910 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f08b00a2350 0x7f08b00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08cfa4d740 1 -- 172.21.15.76:0/2415953734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 msgr2=0x55da8316ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da8316ece0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08c8198c10 tx=0x7f08bc0113e0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f08b00a2350 0x7f08b00a4810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08c00348f0 tx=0x7f08c0005c50 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08cfa4d740 1 -- 172.21.15.76:0/2415953734 shutdown_connections 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da8316ece0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/2415953734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da831637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.729+0000 7f08cfa4d740 1 -- 172.21.15.76:0/2415953734 >> 172.21.15.76:0/2415953734 conn(0x55da8315db20 msgr2=0x55da8315df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 -- 172.21.15.76:0/2415953734 shutdown_connections 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 -- 172.21.15.76:0/2415953734 wait complete. 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 -- start start 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 Processor -- start 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da8316a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da82d9d430 con 0x55da83163d10 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da82d59e00 con 0x55da831633f0 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55da8304f590 con 0x55da8316c8f0 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08cfa4d740 10 client.0 _ll_register_callbacks cb 0x55da830befc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08c5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08c5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83169ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39132/0 (socket says 172.21.15.76:39132) 2024-03-23T15:43:57.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08c5d9b640 1 -- 172.21.15.76:0/1824124289 learned_addr learned my addr 172.21.15.76:0/1824124289 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08c659c640 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.730+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da8316a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08c659c640 1 -- 172.21.15.76:0/1824124289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 msgr2=0x55da8316a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08c659c640 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da8316a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08c659c640 1 -- 172.21.15.76:0/1824124289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 msgr2=0x55da83169ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08c659c640 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83169ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08c659c640 1 -- 172.21.15.76:0/1824124289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08bc0097a0 con 0x55da831633f0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08c659c640 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83169960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08b80cbe20 tx=0x7f08b809ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fabbc32f0f2b4be9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08adffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08adffb640 1 -- 172.21.15.76:0/1824124289 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8003960 con 0x55da831633f0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08adffb640 1 -- 172.21.15.76:0/1824124289 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f08b809f170 con 0x55da831633f0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da82dc97e0 con 0x55da831633f0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08adffb640 1 -- 172.21.15.76:0/1824124289 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f08b80a37c0 con 0x55da831633f0 2024-03-23T15:43:57.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08cfa4d740 10 client.25409 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55da830c8bd0 con 0x55da831633f0 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08adffb640 1 -- 172.21.15.76:0/1824124289 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b80b9020 con 0x55da831633f0 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.731+0000 7f08adffb640 1 -- 172.21.15.76:0/1824124289 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f08b809a470 con 0x55da831633f0 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08cfa4d740 10 client.25409 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08cfa4d740 10 client.25409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08cfa4d740 20 client.25409 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da8317c940 0x55da8317eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08cfa4d740 4 client.25409 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da82e06720 con 0x55da8317c940 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da8317c940 0x55da8317eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08adffb640 10 client.25409 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:57.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.732+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da8317c940 0x55da8317eda0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28da9d852dcd44a6 server_cookie=20c6ff834e4689c9 in_seq=0 out_seq=0 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08adffb640 1 -- 172.21.15.76:0/1824124289 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55da82e06720 con 0x55da8317c940 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08adffb640 10 client.25409 handle_command_reply: tid=0 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 client.25409 shutdown 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 msgr2=0x55da83169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83169960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08b80cbe20 tx=0x7f08b809ac50 comp rx=0 tx=0).stop 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 shutdown_connections 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da8316a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da8317c940 0x55da8317eda0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1824124289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83169960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08adffb640 0 client.25409 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 >> 172.21.15.76:0/1824124289 conn(0x55da8315db20 msgr2=0x55da8315f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.733+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 shutdown_connections 2024-03-23T15:43:57.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.734+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1824124289 wait complete. 2024-03-23T15:43:57.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.734+0000 7f08cfa4d740 20 client.25409 trim_cache size 0 max 16384 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 Processor -- start 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 -- start start 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da83178180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da831846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da8315eba0 con 0x55da83163d10 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da831786f0 con 0x55da8316c8f0 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55da8315d680 con 0x55da831633f0 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39146/0 (socket says 172.21.15.76:39146) 2024-03-23T15:43:57.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 -- 172.21.15.76:0/1818521234 learned_addr learned my addr 172.21.15.76:0/1818521234 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da831846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c659c640 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da83178180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 -- 172.21.15.76:0/1818521234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 msgr2=0x55da83178180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da83178180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 -- 172.21.15.76:0/1818521234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 msgr2=0x55da831846b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da831846b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 -- 172.21.15.76:0/1818521234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da83064c80 con 0x55da83163d10 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.747+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316a200 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009770 tx=0x7f08bc0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7052f338ea9c84f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.748+0000 7f08affff640 1 -- 172.21.15.76:0/1818521234 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc012da0 con 0x55da83163d10 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.748+0000 7f08affff640 1 -- 172.21.15.76:0/1818521234 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc02ca30 con 0x55da83163d10 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.748+0000 7f08affff640 1 -- 172.21.15.76:0/1818521234 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc02c6f0 con 0x55da83163d10 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.748+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1818521234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 msgr2=0x55da8316a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.748+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316a200 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009770 tx=0x7f08bc0113e0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.748+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1818521234 shutdown_connections 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da8316c8f0 0x55da831846b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da8316a200 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/1818521234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da831633f0 0x55da83178180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1818521234 >> 172.21.15.76:0/1818521234 conn(0x55da8315db20 msgr2=0x55da8315c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1818521234 shutdown_connections 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 -- 172.21.15.76:0/1818521234 wait complete. 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 -- start start 2024-03-23T15:43:57.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 Processor -- start 2024-03-23T15:43:57.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83170920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83170e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da831dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da831dc1f0 con 0x55da83163d10 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da831dc360 con 0x55da831633f0 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55da831dc520 con 0x55da8316c8f0 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08cfa4d740 10 client.0 _ll_register_callbacks cb 0x7f08b800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08c5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83170e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08c659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83170920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08c5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83170e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39156/0 (socket says 172.21.15.76:39156) 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08c659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83170920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59446/0 (socket says 172.21.15.76:59446) 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.749+0000 7f08c5d9b640 1 -- 172.21.15.76:0/910251251 learned_addr learned my addr 172.21.15.76:0/910251251 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c6d9d640 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da831dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c659c640 1 -- 172.21.15.76:0/910251251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 msgr2=0x55da831dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c659c640 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da831dbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c659c640 1 -- 172.21.15.76:0/910251251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 msgr2=0x55da83170e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c659c640 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83170e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c659c640 1 -- 172.21.15.76:0/910251251 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da8305fde0 con 0x55da831633f0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83170e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08c659c640 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83170920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08b800ea30 tx=0x7f08b80cbd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3022be6db17dc8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08adffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08adffb640 1 -- 172.21.15.76:0/910251251 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004200 con 0x55da831633f0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08adffb640 1 -- 172.21.15.76:0/910251251 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f08b809b720 con 0x55da831633f0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08bc012930 con 0x55da831633f0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08adffb640 1 -- 172.21.15.76:0/910251251 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f08b8004360 con 0x55da831633f0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08cfa4d740 10 client.25415 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.750+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55da8308a920 con 0x55da831633f0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08adffb640 1 -- 172.21.15.76:0/910251251 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc012930 con 0x55da831633f0 2024-03-23T15:43:57.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08adffb640 1 -- 172.21.15.76:0/910251251 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f08b8096f70 con 0x55da831633f0 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08cfa4d740 10 client.25415 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08cfa4d740 10 client.25415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08cfa4d740 20 client.25415 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da831e2d60 0x55da831e5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08cfa4d740 4 client.25415 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.751+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da82e55890 con 0x55da831e2d60 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da831e2d60 0x55da831e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08c5d9b640 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da831e2d60 0x55da831e5180 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ba215bdadfc49fc server_cookie=7c7b8f47b60b0abf in_seq=0 out_seq=0 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08adffb640 10 client.25415 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08adffb640 1 -- 172.21.15.76:0/910251251 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da82e55890 con 0x55da831e2d60 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08adffb640 10 client.25415 handle_command_reply: tid=0 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 client.25415 shutdown 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 msgr2=0x55da83170920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83170920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08b800ea30 tx=0x7f08b80cbd90 comp rx=0 tx=0).stop 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 shutdown_connections 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da8316c8f0 0x55da831dbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x55da831e2d60 0x55da831e5180 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da83163d10 0x55da83170e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 --2- 172.21.15.76:0/910251251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da831633f0 0x55da83170920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 >> 172.21.15.76:0/910251251 conn(0x55da8315db20 msgr2=0x55da8315f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.752+0000 7f08adffb640 0 client.25415 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 shutdown_connections 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cfa4d740 1 -- 172.21.15.76:0/910251251 wait complete. 2024-03-23T15:43:57.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cfa4d740 20 client.25415 trim_cache size 0 max 16384 2024-03-23T15:43:57.763 INFO:teuthology.orchestra.run.smithi076.stdout:[] 2024-03-23T15:43:57.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f08b00a2350 msgr2=0x7f08b00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f08b00a2350 0x7f08b00a4810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08c00348f0 tx=0x7f08c0005c50 comp rx=0 tx=0).stop 2024-03-23T15:43:57.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c8068910 msgr2=0x7f08c819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c8068910 0x7f08c819d920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f08b800ea60 tx=0x7f08b800ef20 comp rx=0 tx=0).stop 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 shutdown_connections 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f08b00a2350 0x7f08b00a4810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08c8107760 0x7f08c8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08c80692d0 0x7f08c819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 --2- 172.21.15.76:0/975772076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c8068910 0x7f08c819d920 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 >> 172.21.15.76:0/975772076 conn(0x7f08c8077710 msgr2=0x7f08c80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 shutdown_connections 2024-03-23T15:43:57.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.753+0000 7f08cce72640 1 -- 172.21.15.76:0/975772076 wait complete. 2024-03-23T15:43:57.770 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T15:43:57.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/917023394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd44073ac0 msgr2=0x7fbd44073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/917023394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd44073ac0 0x7fbd44073ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd4002bcd0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/917023394 shutdown_connections 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/917023394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 0x7fbd44110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/917023394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd440743e0 0x7fbd4410da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/917023394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd44073ac0 0x7fbd44073ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/917023394 >> 172.21.15.76:0/917023394 conn(0x7fbd440fd1b0 msgr2=0x7fbd440ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/917023394 shutdown_connections 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.886+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/917023394 wait complete. 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.887+0000 7fbd4bbd7640 1 Processor -- start 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.887+0000 7fbd4bbd7640 1 -- start start 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.887+0000 7fbd4bbd7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd44073ac0 0x7fbd44199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.887+0000 7fbd4bbd7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd440743e0 0x7fbd4419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.887+0000 7fbd4bbd7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 0x7fbd4419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.887+0000 7fbd4a174640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 0x7fbd4419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd49973640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd44073ac0 0x7fbd44199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 0x7fbd4419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39164/0 (socket says 172.21.15.76:39164) 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 -- 172.21.15.76:0/1133226442 learned_addr learned my addr 172.21.15.76:0/1133226442 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd49172640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd440743e0 0x7fbd4419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.887+0000 7fbd4bbd7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4419ee60 con 0x7fbd4410df60 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4419efd0 con 0x7fbd440743e0 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4419f190 con 0x7fbd44073ac0 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 -- 172.21.15.76:0/1133226442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd44073ac0 msgr2=0x7fbd44199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd44073ac0 0x7fbd44199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 -- 172.21.15.76:0/1133226442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd440743e0 msgr2=0x7fbd4419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd440743e0 0x7fbd4419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 -- 172.21.15.76:0/1133226442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd40009560 con 0x7fbd4410df60 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4a174640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 0x7fbd4419e860 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00eb10 tx=0x7fbd3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd3affd640 1 -- 172.21.15.76:0/1133226442 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004b20 con 0x7fbd4410df60 2024-03-23T15:43:57.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4419f470 con 0x7fbd4410df60 2024-03-23T15:43:57.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd3affd640 1 -- 172.21.15.76:0/1133226442 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c004c80 con 0x7fbd4410df60 2024-03-23T15:43:57.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd441a6cf0 con 0x7fbd4410df60 2024-03-23T15:43:57.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.888+0000 7fbd3affd640 1 -- 172.21.15.76:0/1133226442 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c010670 con 0x7fbd4410df60 2024-03-23T15:43:57.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.892+0000 7fbd4e7b9740 1 Processor -- start 2024-03-23T15:43:57.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.892+0000 7fbd3affd640 1 -- 172.21.15.76:0/1133226442 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7fbd3c005020 con 0x7fbd4410df60 2024-03-23T15:43:57.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.892+0000 7fbd4e7b9740 1 -- start start 2024-03-23T15:43:57.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae43f0 0x5587faae47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae4d10 0x5587faaed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaefce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5587faa52840 con 0x5587faae4d10 2024-03-23T15:43:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587faa4fd70 con 0x5587faae43f0 2024-03-23T15:43:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5587faa4b550 con 0x5587faaed8f0 2024-03-23T15:43:57.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49973640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae4d10 0x5587faaed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae43f0 0x5587faae47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae43f0 0x5587faae47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59476/0 (socket says 172.21.15.76:59476) 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 -- 172.21.15.76:0/341329248 learned_addr learned my addr 172.21.15.76:0/341329248 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49973640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae4d10 0x5587faaed3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39166/0 (socket says 172.21.15.76:39166) 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaefce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 -- 172.21.15.76:0/341329248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 msgr2=0x5587faaefce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaefce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 -- 172.21.15.76:0/341329248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae4d10 msgr2=0x5587faaed3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae4d10 0x5587faaed3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 -- 172.21.15.76:0/341329248 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587fa9e0de0 con 0x5587faae43f0 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49973640 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae4d10 0x5587faaed3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.893+0000 7fbd49172640 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae43f0 0x5587faae47d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009f80 tx=0x7fbd34009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b4a1e7d713edaf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd2ffff640 1 -- 172.21.15.76:0/341329248 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34004250 con 0x5587faae43f0 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd2ffff640 1 -- 172.21.15.76:0/341329248 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd340043b0 con 0x5587faae43f0 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd2ffff640 1 -- 172.21.15.76:0/341329248 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd340035f0 con 0x5587faae43f0 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd3affd640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fbd300a2350 0x7fbd300a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd49973640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fbd300a2350 0x7fbd300a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd3affd640 1 -- 172.21.15.76:0/1133226442 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd3c014070 con 0x7fbd4410df60 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/341329248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae43f0 msgr2=0x5587faae47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae43f0 0x5587faae47d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009f80 tx=0x7fbd34009930 comp rx=0 tx=0).stop 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/341329248 shutdown_connections 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaefce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae4d10 0x5587faaed3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/341329248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae43f0 0x5587faae47d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.894+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/341329248 >> 172.21.15.76:0/341329248 conn(0x5587faadeb20 msgr2=0x5587faadef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.895+0000 7fbd49973640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fbd300a2350 0x7fbd300a4810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006010 tx=0x7fbd40005e40 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.895+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/341329248 shutdown_connections 2024-03-23T15:43:57.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.895+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/341329248 wait complete. 2024-03-23T15:43:57.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.895+0000 7fbd4e7b9740 1 -- start start 2024-03-23T15:43:57.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.895+0000 7fbd4e7b9740 1 Processor -- start 2024-03-23T15:43:57.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaeae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaeb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39182/0 (socket says 172.21.15.76:39182) 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49172640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaeae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 -- 172.21.15.76:0/1806242066 learned_addr learned my addr 172.21.15.76:0/1806242066 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5587fa71e430 con 0x5587faae43f0 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587fa6dae00 con 0x5587faae4d10 2024-03-23T15:43:57.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5587fa9d0590 con 0x5587faaed8f0 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4e7b9740 10 client.0 _ll_register_callbacks cb 0x5587faa3ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd4a174640 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaeb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 -- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 msgr2=0x5587faaeb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaeb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 -- 172.21.15.76:0/1806242066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 msgr2=0x5587faaeae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaeae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 -- 172.21.15.76:0/1806242066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd34009df0 con 0x5587faae43f0 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49172640 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaeae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd49973640 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaea950 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbd40035000 tx=0x7fbd40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202e147be1cdc281 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.896+0000 7fbd2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/1806242066 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40004af0 con 0x5587faae43f0 2024-03-23T15:43:57.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/1806242066 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd400042c0 con 0x5587faae43f0 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587fa74a7e0 con 0x5587faae43f0 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/1806242066 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbd4002c6b0 con 0x5587faae43f0 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 10 client.15642 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5587faa49bd0 con 0x5587faae43f0 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/1806242066 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd40061080 con 0x5587faae43f0 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/1806242066 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbd4005b030 con 0x5587faae43f0 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 10 client.15642 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 20 client.15642 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587faaffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 4 client.15642 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:57.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.897+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587fa787720 con 0x5587faafda00 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.898+0000 7fbd49172640 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587faaffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.898+0000 7fbd2d7fa640 10 client.15642 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.898+0000 7fbd49172640 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587faaffe80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14b31f3024bceac2 server_cookie=5cd3139cd80e6832 in_seq=0 out_seq=0 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/1806242066 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5587fa787720 con 0x5587faafda00 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd2d7fa640 10 client.15642 handle_command_reply: tid=0 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 client.15642 shutdown 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 msgr2=0x5587faaea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaea950 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbd40035000 tx=0x7fbd40009260 comp rx=0 tx=0).stop 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 shutdown_connections 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587faaffe80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd2d7fa640 0 client.15642 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587faaeb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaeae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/1806242066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaea950 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 >> 172.21.15.76:0/1806242066 conn(0x5587faadeb20 msgr2=0x5587faae0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 shutdown_connections 2024-03-23T15:43:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.899+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/1806242066 wait complete. 2024-03-23T15:43:57.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.900+0000 7fbd4e7b9740 20 client.15642 trim_cache size 0 max 16384 2024-03-23T15:43:57.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 Processor -- start 2024-03-23T15:43:57.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 -- start start 2024-03-23T15:43:57.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaf9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab05790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5587faafa490 con 0x5587faae43f0 2024-03-23T15:43:57.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587faada200 con 0x5587faae4d10 2024-03-23T15:43:57.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5587faaf96c0 con 0x5587faaed8f0 2024-03-23T15:43:57.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd49973640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaf9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd49973640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaf9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39186/0 (socket says 172.21.15.76:39186) 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd49973640 1 -- 172.21.15.76:0/3455216295 learned_addr learned my addr 172.21.15.76:0/3455216295 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd4a174640 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab05790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.913+0000 7fbd49172640 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd49172640 1 -- 172.21.15.76:0/3455216295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 msgr2=0x5587fab05790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd49172640 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab05790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd49172640 1 -- 172.21.15.76:0/3455216295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 msgr2=0x5587faaf9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd49172640 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaf9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd49172640 1 -- 172.21.15.76:0/3455216295 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587fa9e5c80 con 0x5587faae4d10 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd49973640 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaf9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd49172640 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaddb40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd34004ae0 tx=0x7fbd34009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554189a42e62551a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd2f7fe640 1 -- 172.21.15.76:0/3455216295 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34009780 con 0x5587faae4d10 2024-03-23T15:43:57.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd2f7fe640 1 -- 172.21.15.76:0/3455216295 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd34004bd0 con 0x5587faae4d10 2024-03-23T15:43:57.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.914+0000 7fbd2f7fe640 1 -- 172.21.15.76:0/3455216295 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34005440 con 0x5587faae4d10 2024-03-23T15:43:57.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3455216295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 msgr2=0x5587faaddb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaddb40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd34004ae0 tx=0x7fbd34009d70 comp rx=0 tx=0).stop 2024-03-23T15:43:57.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3455216295 shutdown_connections 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faaddb40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3455216295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faaf9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3455216295 >> 172.21.15.76:0/3455216295 conn(0x5587faadeb20 msgr2=0x5587faadd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3455216295 shutdown_connections 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3455216295 wait complete. 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 -- start start 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.915+0000 7fbd4e7b9740 1 Processor -- start 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faae7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faae8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4e7b9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab62dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5587fab63300 con 0x5587faae43f0 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587fab63440 con 0x5587faae4d10 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4e7b9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5587fab63580 con 0x5587faaed8f0 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4e7b9740 10 client.0 _ll_register_callbacks cb 0x7fbd40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faae8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faae8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59502/0 (socket says 172.21.15.76:59502) 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 -- 172.21.15.76:0/3323803734 learned_addr learned my addr 172.21.15.76:0/3323803734 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4a174640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab62dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49973640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faae7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 -- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 msgr2=0x5587fab62dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab62dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 -- 172.21.15.76:0/3323803734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 msgr2=0x5587faae7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:57.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faae7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 -- 172.21.15.76:0/3323803734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587faa49bd0 con 0x5587faae4d10 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd4a174640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab62dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49973640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faae7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd49172640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faae8210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5587faae4af0 tx=0x7fbd34004640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acfd39f695dea363 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.916+0000 7fbd2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/3323803734 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd340050c0 con 0x5587faae4d10 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd340043b0 con 0x5587faae4d10 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/3323803734 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd34005220 con 0x5587faae4d10 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/3323803734 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbd340053e0 con 0x5587faae4d10 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 10 client.25433 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5587fa9e0de0 con 0x5587faae4d10 2024-03-23T15:43:57.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/3323803734 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd34045080 con 0x5587faae4d10 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/3323803734 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbd34029040 con 0x5587faae4d10 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 10 client.25433 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 10 client.25433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 20 client.25433 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587fab6f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 4 client.25433 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.917+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587fa7d6890 con 0x5587faafda00 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd49973640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587fab6f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd2d7fa640 10 client.25433 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd49973640 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587fab6f250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cdbd13413991867 server_cookie=6ce6f79aab795a1c in_seq=0 out_seq=0 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd2d7fa640 1 -- 172.21.15.76:0/3323803734 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587fa7d6890 con 0x5587faafda00 2024-03-23T15:43:57.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd2d7fa640 10 client.25433 handle_command_reply: tid=0 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 client.25433 shutdown 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 msgr2=0x5587faae8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faae8210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5587faae4af0 tx=0x7fbd34004640 comp rx=0 tx=0).stop 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 shutdown_connections 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x5587faafda00 0x5587fab6f250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587faaed8f0 0x5587fab62dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd2d7fa640 0 client.25433 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587faae4d10 0x5587faae8210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 --2- 172.21.15.76:0/3323803734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5587faae43f0 0x5587faae7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 >> 172.21.15.76:0/3323803734 conn(0x5587faadeb20 msgr2=0x5587faaddcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.918+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 shutdown_connections 2024-03-23T15:43:57.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4e7b9740 1 -- 172.21.15.76:0/3323803734 wait complete. 2024-03-23T15:43:57.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4e7b9740 20 client.25433 trim_cache size 0 max 16384 2024-03-23T15:43:57.929 INFO:teuthology.orchestra.run.smithi076.stdout:[] 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fbd300a2350 msgr2=0x7fbd300a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fbd300a2350 0x7fbd300a4810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006010 tx=0x7fbd40005e40 comp rx=0 tx=0).stop 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 msgr2=0x7fbd4419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 0x7fbd4419e860 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00eb10 tx=0x7fbd3c00c2d0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 shutdown_connections 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fbd300a2350 0x7fbd300a4810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd4410df60 0x7fbd4419e860 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd440743e0 0x7fbd4419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 --2- 172.21.15.76:0/1133226442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd44073ac0 0x7fbd44199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:57.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 >> 172.21.15.76:0/1133226442 conn(0x7fbd440fd1b0 msgr2=0x7fbd440fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:57.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 shutdown_connections 2024-03-23T15:43:57.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:57.919+0000 7fbd4bbd7640 1 -- 172.21.15.76:0/1133226442 wait complete. 2024-03-23T15:43:57.933 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T15:43:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.036+0000 7fe06faf4640 1 -- 172.21.15.76:0/3346793330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe068107760 msgr2=0x7fe068107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.036+0000 7fe06faf4640 1 --2- 172.21.15.76:0/3346793330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe068107760 0x7fe068107b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0640098c0 tx=0x7fe06402bb50 comp rx=0 tx=0).stop 2024-03-23T15:43:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 -- 172.21.15.76:0/3346793330 shutdown_connections 2024-03-23T15:43:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 --2- 172.21.15.76:0/3346793330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0680692d0 0x7fe06810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 --2- 172.21.15.76:0/3346793330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe068068910 0x7fe068068d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 --2- 172.21.15.76:0/3346793330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe068107760 0x7fe068107b40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 -- 172.21.15.76:0/3346793330 >> 172.21.15.76:0/3346793330 conn(0x7fe068077710 msgr2=0x7fe068077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 -- 172.21.15.76:0/3346793330 shutdown_connections 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 -- 172.21.15.76:0/3346793330 wait complete. 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 Processor -- start 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 -- start start 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068068910 0x7fe06819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.037+0000 7fe06faf4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0680692d0 0x7fe06819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06faf4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe068107760 0x7fe068197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06faf4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681982c0 con 0x7fe068107760 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06faf4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe068198430 con 0x7fe0680692d0 2024-03-23T15:43:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06faf4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681985f0 con 0x7fe068068910 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068068910 0x7fe06819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d08f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0680692d0 0x7fe06819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068068910 0x7fe06819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41950/0 (socket says 172.21.15.76:41950) 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 -- 172.21.15.76:0/789359808 learned_addr learned my addr 172.21.15.76:0/789359808 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d08f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0680692d0 0x7fe06819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59520/0 (socket says 172.21.15.76:59520) 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06e091640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe068107760 0x7fe068197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 -- 172.21.15.76:0/789359808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0680692d0 msgr2=0x7fe06819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0680692d0 0x7fe06819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 -- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe068107760 msgr2=0x7fe068197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe068107760 0x7fe068197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 -- 172.21.15.76:0/789359808 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe058009560 con 0x7fe068068910 2024-03-23T15:43:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06d890640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068068910 0x7fe06819da70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe064009340 tx=0x7fe064004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe05effd640 1 -- 172.21.15.76:0/789359808 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06402c840 con 0x7fe068068910 2024-03-23T15:43:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.038+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe064009490 con 0x7fe068068910 2024-03-23T15:43:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.039+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe068100500 con 0x7fe068068910 2024-03-23T15:43:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.039+0000 7fe05effd640 1 -- 172.21.15.76:0/789359808 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06402c9a0 con 0x7fe068068910 2024-03-23T15:43:58.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.039+0000 7fe05effd640 1 -- 172.21.15.76:0/789359808 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064035e40 con 0x7fe068068910 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.041+0000 7fe05effd640 1 -- 172.21.15.76:0/789359808 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7fe064035650 con 0x7fe068068910 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.041+0000 7fe0727b2740 1 Processor -- start 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.041+0000 7fe0727b2740 1 -- start start 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.041+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a990d10 0x564f7a9993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9998f0 0x564f7a99bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe0727b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564f7a8fe840 con 0x564f7a9998f0 2024-03-23T15:43:58.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe0727b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564f7a8fbd70 con 0x564f7a9903f0 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe0727b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564f7a8f7550 con 0x564f7a990d10 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06d890640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a990d10 0x564f7a9993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9907d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59524/0 (socket says 172.21.15.76:59524) 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06d890640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a990d10 0x564f7a9993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41952/0 (socket says 172.21.15.76:41952) 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 -- 172.21.15.76:0/3193004819 learned_addr learned my addr 172.21.15.76:0/3193004819 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe05effd640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe0540a23d0 0x7fe0540a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06d08f640 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9998f0 0x564f7a99bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 -- 172.21.15.76:0/3193004819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a990d10 msgr2=0x564f7a9993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a990d10 0x564f7a9993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 -- 172.21.15.76:0/3193004819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9998f0 msgr2=0x564f7a99bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9998f0 0x564f7a99bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 -- 172.21.15.76:0/3193004819 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f7a88cde0 con 0x564f7a9903f0 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06d08f640 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9998f0 0x564f7a99bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe0540a23d0 0x7fe0540a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe05effd640 1 -- 172.21.15.76:0/789359808 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe064031070 con 0x7fe068068910 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.042+0000 7fe06e091640 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9907d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe060009fc0 tx=0x7fe060010920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb2c8c9086da6a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe053fff640 1 -- 172.21.15.76:0/3193004819 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600047d0 con 0x564f7a9903f0 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe053fff640 1 -- 172.21.15.76:0/3193004819 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe060026040 con 0x564f7a9903f0 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe053fff640 1 -- 172.21.15.76:0/3193004819 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06002b640 con 0x564f7a9903f0 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe06e091640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe0540a23d0 0x7fe0540a4890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe060034000 tx=0x7fe060035000 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe0727b2740 1 -- 172.21.15.76:0/3193004819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 msgr2=0x564f7a9907d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9907d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe060009fc0 tx=0x7fe060010920 comp rx=0 tx=0).stop 2024-03-23T15:43:58.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe0727b2740 1 -- 172.21.15.76:0/3193004819 shutdown_connections 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9998f0 0x564f7a99bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a990d10 0x564f7a9993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3193004819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9907d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.043+0000 7fe0727b2740 1 -- 172.21.15.76:0/3193004819 >> 172.21.15.76:0/3193004819 conn(0x564f7a98ab20 msgr2=0x564f7a98af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 -- 172.21.15.76:0/3193004819 shutdown_connections 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 -- 172.21.15.76:0/3193004819 wait complete. 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 -- start start 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 Processor -- start 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a9968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a996e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a997370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564f7a5ca430 con 0x564f7a9903f0 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564f7a586e00 con 0x564f7a990d10 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564f7a87c590 con 0x564f7a9998f0 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe0727b2740 10 client.0 _ll_register_callbacks cb 0x564f7a8ebfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.044+0000 7fe06e091640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a997370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a997370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:41960/0 (socket says 172.21.15.76:41960) 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 -- 172.21.15.76:0/3440126641 learned_addr learned my addr 172.21.15.76:0/3440126641 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06d890640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a9968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06d08f640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a996e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 -- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 msgr2=0x564f7a996e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a996e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 -- 172.21.15.76:0/3440126641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 msgr2=0x564f7a9968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a9968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 -- 172.21.15.76:0/3440126641 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe060009cd0 con 0x564f7a9998f0 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06d08f640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a996e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06d890640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a9968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe06e091640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a997370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe060003c80 tx=0x7fe06002ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44af3dda874ffdb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe0517fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe0517fa640 1 -- 172.21.15.76:0/3440126641 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06002c890 con 0x564f7a9998f0 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe0517fa640 1 -- 172.21.15.76:0/3440126641 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe06003b040 con 0x564f7a9998f0 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f7a5f67e0 con 0x564f7a9998f0 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe0517fa640 1 -- 172.21.15.76:0/3440126641 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe06002b950 con 0x564f7a9998f0 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe0727b2740 10 client.25465 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:58.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.045+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564f7a8f5bd0 con 0x564f7a9998f0 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0517fa640 1 -- 172.21.15.76:0/3440126641 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564f7a5f67e0 con 0x564f7a9998f0 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0517fa640 1 -- 172.21.15.76:0/3440126641 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe060035070 con 0x564f7a9998f0 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0727b2740 10 client.25465 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0727b2740 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0727b2740 20 client.25465 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7a9abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0727b2740 4 client.25465 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f7a633720 con 0x564f7a9a9ac0 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.046+0000 7fe06d890640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7a9abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.047+0000 7fe06d890640 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7a9abee0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f124af92fa76c2fb server_cookie=28003883ad3ba5 in_seq=0 out_seq=0 2024-03-23T15:43:58.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.047+0000 7fe0517fa640 10 client.25465 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:58.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0517fa640 1 -- 172.21.15.76:0/3440126641 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564f7a633720 con 0x564f7a9a9ac0 2024-03-23T15:43:58.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0517fa640 10 client.25465 handle_command_reply: tid=0 2024-03-23T15:43:58.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 client.25465 shutdown 2024-03-23T15:43:58.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 msgr2=0x564f7a997370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a997370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe060003c80 tx=0x7fe06002ccd0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 shutdown_connections 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a997370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7a9abee0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0517fa640 0 client.25465 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a996e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 --2- 172.21.15.76:0/3440126641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a9968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 >> 172.21.15.76:0/3440126641 conn(0x564f7a98ab20 msgr2=0x564f7a98c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 shutdown_connections 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.048+0000 7fe0727b2740 1 -- 172.21.15.76:0/3440126641 wait complete. 2024-03-23T15:43:58.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.049+0000 7fe0727b2740 20 client.25465 trim_cache size 0 max 16384 2024-03-23T15:43:58.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 Processor -- start 2024-03-23T15:43:58.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 -- start start 2024-03-23T15:43:58.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a999020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a9a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a9b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564f7a98b570 con 0x564f7a9903f0 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564f7a98a650 con 0x564f7a990d10 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe0727b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564f7a5f67e0 con 0x564f7a9998f0 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe06d890640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a999020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe06d08f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a9a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe06d890640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a999020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39230/0 (socket says 172.21.15.76:39230) 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe06d08f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a9a51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59542/0 (socket says 172.21.15.76:59542) 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe06d890640 1 -- 172.21.15.76:0/4170630398 learned_addr learned my addr 172.21.15.76:0/4170630398 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.062+0000 7fe06e091640 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a9b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06d890640 1 -- 172.21.15.76:0/4170630398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 msgr2=0x564f7a9b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06d890640 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a9b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06d890640 1 -- 172.21.15.76:0/4170630398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 msgr2=0x564f7a9a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06d890640 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a9a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06d890640 1 -- 172.21.15.76:0/4170630398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f7a891c80 con 0x564f7a9903f0 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06e091640 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a9b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06d08f640 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a9a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe06d890640 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a999020 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe064000c00 tx=0x7fe0640b9410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6290fc6e2f16ef25 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe0537fe640 1 -- 172.21.15.76:0/4170630398 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640bd040 con 0x564f7a9903f0 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe0537fe640 1 -- 172.21.15.76:0/4170630398 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0640c10e0 con 0x564f7a9903f0 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.063+0000 7fe0537fe640 1 -- 172.21.15.76:0/4170630398 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640ee300 con 0x564f7a9903f0 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 -- 172.21.15.76:0/4170630398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 msgr2=0x564f7a999020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a999020 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe064000c00 tx=0x7fe0640b9410 comp rx=0 tx=0).stop 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 -- 172.21.15.76:0/4170630398 shutdown_connections 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a9b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a990d10 0x564f7a9a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 --2- 172.21.15.76:0/4170630398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a9903f0 0x564f7a999020 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 -- 172.21.15.76:0/4170630398 >> 172.21.15.76:0/4170630398 conn(0x564f7a98ab20 msgr2=0x564f7a985bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:58.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 -- 172.21.15.76:0/4170630398 shutdown_connections 2024-03-23T15:43:58.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 -- 172.21.15.76:0/4170630398 wait complete. 2024-03-23T15:43:58.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 -- start start 2024-03-23T15:43:58.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.064+0000 7fe0727b2740 1 Processor -- start 2024-03-23T15:43:58.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a990d10 0x564f7a99ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe0727b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a99d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe0727b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564f7a99da90 con 0x564f7a990d10 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe0727b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564f7a99dc00 con 0x564f7a9903f0 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe0727b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564f7aa0edc0 con 0x564f7a9998f0 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe0727b2740 10 client.0 _ll_register_callbacks cb 0x7fe0640b9360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59544/0 (socket says 172.21.15.76:59544) 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 -- 172.21.15.76:0/650923416 learned_addr learned my addr 172.21.15.76:0/650923416 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06e091640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a99d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d08f640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a990d10 0x564f7a99ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 -- 172.21.15.76:0/650923416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 msgr2=0x564f7a99d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a99d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 -- 172.21.15.76:0/650923416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a990d10 msgr2=0x564f7a99ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a990d10 0x564f7a99ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d890640 1 -- 172.21.15.76:0/650923416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe058009d80 con 0x564f7a9903f0 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.065+0000 7fe06d08f640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a990d10 0x564f7a99ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe06d890640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9a1a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0640eded0 tx=0x7fe0640bd130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c346b4ca20f6f92f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0517fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0517fa640 1 -- 172.21.15.76:0/650923416 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640ee2a0 con 0x564f7a9903f0 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0517fa640 1 -- 172.21.15.76:0/650923416 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe0640d4030 con 0x564f7a9903f0 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f7a996720 con 0x564f7a9903f0 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0517fa640 1 -- 172.21.15.76:0/650923416 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe0640bb440 con 0x564f7a9903f0 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 10 client.25451 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:58.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x7fe068100500 con 0x564f7a9903f0 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0517fa640 1 -- 172.21.15.76:0/650923416 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564f7a996720 con 0x564f7a9903f0 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0517fa640 1 -- 172.21.15.76:0/650923416 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe06403d040 con 0x564f7a9903f0 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 10 client.25451 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 20 client.25451 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7aa12090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 4 client.25451 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.066+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe060026040 con 0x564f7a9a9ac0 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe06d08f640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7aa12090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe0517fa640 10 client.25451 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:58.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe06d08f640 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7aa12090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b2d936c0daa841 server_cookie=e8feafd02d9a0341 in_seq=0 out_seq=0 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe0517fa640 1 -- 172.21.15.76:0/650923416 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe060026040 con 0x564f7a9a9ac0 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe0517fa640 10 client.25451 handle_command_reply: tid=0 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe0727b2740 1 client.25451 shutdown 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 msgr2=0x564f7a9a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.067+0000 7fe0727b2740 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9a1a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0640eded0 tx=0x7fe0640bd130 comp rx=0 tx=0).stop 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 shutdown_connections 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564f7a9998f0 0x564f7a99d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564f7a990d10 0x564f7a99ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564f7a9a9ac0 0x564f7aa12090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 --2- 172.21.15.76:0/650923416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564f7a9903f0 0x564f7a9a1a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 >> 172.21.15.76:0/650923416 conn(0x564f7a98ab20 msgr2=0x564f7a989430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:58.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0517fa640 0 client.25451 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:58.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 shutdown_connections 2024-03-23T15:43:58.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 1 -- 172.21.15.76:0/650923416 wait complete. 2024-03-23T15:43:58.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe0727b2740 20 client.25451 trim_cache size 0 max 16384 2024-03-23T15:43:58.079 INFO:teuthology.orchestra.run.smithi076.stdout:[] 2024-03-23T15:43:58.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe0540a23d0 msgr2=0x7fe0540a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe06faf4640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe0540a23d0 0x7fe0540a4890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe060034000 tx=0x7fe060035000 comp rx=0 tx=0).stop 2024-03-23T15:43:58.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068068910 msgr2=0x7fe06819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:58.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.068+0000 7fe06faf4640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068068910 0x7fe06819da70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe064009340 tx=0x7fe064004600 comp rx=0 tx=0).stop 2024-03-23T15:43:58.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 shutdown_connections 2024-03-23T15:43:58.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe0540a23d0 0x7fe0540a4890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe068107760 0x7fe068197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0680692d0 0x7fe06819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 --2- 172.21.15.76:0/789359808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068068910 0x7fe06819da70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:58.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 >> 172.21.15.76:0/789359808 conn(0x7fe068077710 msgr2=0x7fe0680fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:58.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 shutdown_connections 2024-03-23T15:43:58.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:58.069+0000 7fe06faf4640 1 -- 172.21.15.76:0/789359808 wait complete. 2024-03-23T15:43:58.291 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:58 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3111012880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:58.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:58 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3111012880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:58.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:58 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3111012880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:43:59.084 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T15:43:59.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:43:59.084 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T15:43:59.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 -- 172.21.15.76:0/1219445632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 msgr2=0x7f5fc4107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 --2- 172.21.15.76:0/1219445632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 0x7f5fc4107a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009900 tx=0x7f5fb802bc30 comp rx=0 tx=0).stop 2024-03-23T15:43:59.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 -- 172.21.15.76:0/1219445632 shutdown_connections 2024-03-23T15:43:59.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 --2- 172.21.15.76:0/1219445632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fc40692b0 0x7f5fc410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 --2- 172.21.15.76:0/1219445632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc40688f0 0x7f5fc4068d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 --2- 172.21.15.76:0/1219445632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 0x7f5fc4107a70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 -- 172.21.15.76:0/1219445632 >> 172.21.15.76:0/1219445632 conn(0x7f5fc4077710 msgr2=0x7f5fc4077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 -- 172.21.15.76:0/1219445632 shutdown_connections 2024-03-23T15:43:59.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 -- 172.21.15.76:0/1219445632 wait complete. 2024-03-23T15:43:59.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.189+0000 7f5fc916c640 1 Processor -- start 2024-03-23T15:43:59.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc916c640 1 -- start start 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc916c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc40688f0 0x7f5fc419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc916c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fc40692b0 0x7f5fc419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc916c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 0x7f5fc4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc916c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41982c0 con 0x7f5fc40688f0 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc916c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4198430 con 0x7f5fc4107690 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc916c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41985f0 con 0x7f5fc40692b0 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc40688f0 0x7f5fc419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc40688f0 0x7f5fc419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39244/0 (socket says 172.21.15.76:39244) 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc2d9d640 1 -- 172.21.15.76:0/2909899144 learned_addr learned my addr 172.21.15.76:0/2909899144 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc259c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fc40692b0 0x7f5fc419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc359e640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 0x7f5fc4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc259c640 1 -- 172.21.15.76:0/2909899144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 msgr2=0x7f5fc4197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc259c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 0x7f5fc4197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc259c640 1 -- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc40688f0 msgr2=0x7f5fc419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc259c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc40688f0 0x7f5fc419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.190+0000 7f5fc259c640 1 -- 172.21.15.76:0/2909899144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb8009560 con 0x7f5fc40692b0 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.191+0000 7f5fc259c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fc40692b0 0x7f5fc419dfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009bc0 tx=0x7f5fb400b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.191+0000 7f5fabfff640 1 -- 172.21.15.76:0/2909899144 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4015070 con 0x7f5fc40692b0 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.191+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc4198870 con 0x7f5fc40692b0 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.191+0000 7f5fabfff640 1 -- 172.21.15.76:0/2909899144 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb400be60 con 0x7f5fc40692b0 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.191+0000 7f5fabfff640 1 -- 172.21.15.76:0/2909899144 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40187d0 con 0x7f5fc40692b0 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.191+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc41006e0 con 0x7f5fc40692b0 2024-03-23T15:43:59.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.194+0000 7f5fabfff640 1 -- 172.21.15.76:0/2909899144 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f5fb4019b10 con 0x7f5fc40692b0 2024-03-23T15:43:59.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.194+0000 7f5fcbe23740 1 Processor -- start 2024-03-23T15:43:59.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.194+0000 7f5fcbe23740 1 -- start start 2024-03-23T15:43:59.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.194+0000 7f5fabfff640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5fac0a2350 0x7f5fac0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.194+0000 7f5fc359e640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5fac0a2350 0x7f5fac0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fabfff640 1 -- 172.21.15.76:0/2909899144 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5fb40d1c40 con 0x7f5fc40692b0 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21c1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21c1c60 0x55f7c21c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7c2126530 con 0x55f7c21b8760 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c2123a60 con 0x55f7c21c1c60 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7c211f240 con 0x55f7c21b9080 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39248/0 (socket says 172.21.15.76:39248) 2024-03-23T15:43:59.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc2d9d640 1 -- 172.21.15.76:0/1332860143 learned_addr learned my addr 172.21.15.76:0/1332860143 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc259c640 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc359e640 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21c1c60 0x55f7c21c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc259c640 1 -- 172.21.15.76:0/1332860143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21c1c60 msgr2=0x55f7c21c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc259c640 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21c1c60 0x55f7c21c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc259c640 1 -- 172.21.15.76:0/1332860143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 msgr2=0x55f7c21b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc259c640 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc259c640 1 -- 172.21.15.76:0/1332860143 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c211d8c0 con 0x55f7c21b9080 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc359e640 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21c1c60 0x55f7c21c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc359e640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5fac0a2350 0x7f5fac0a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005fd0 tx=0x7f5fbc011810 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.195+0000 7f5fc259c640 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21c1720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40cf210 tx=0x7f5fb409dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5a13cf762eae3a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fa97fa640 1 -- 172.21.15.76:0/1332860143 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40a2530 con 0x55f7c21b9080 2024-03-23T15:43:59.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fa97fa640 1 -- 172.21.15.76:0/1332860143 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb40a6b80 con 0x55f7c21b9080 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fa97fa640 1 -- 172.21.15.76:0/1332860143 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40cf730 con 0x55f7c21b9080 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1332860143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 msgr2=0x55f7c21c1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21c1720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40cf210 tx=0x7f5fb409dfd0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1332860143 shutdown_connections 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21c1c60 0x55f7c21c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21c1720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1332860143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1332860143 >> 172.21.15.76:0/1332860143 conn(0x55f7c21b2e90 msgr2=0x55f7c21b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1332860143 shutdown_connections 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.196+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1332860143 wait complete. 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 -- start start 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 Processor -- start 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b9080 0x55f7c21bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7c20c3f80 con 0x55f7c21b9080 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1df24b0 con 0x55f7c21b8760 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7c20a4ad0 con 0x55f7c21c1c60 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fcbe23740 10 client.0 _ll_register_callbacks cb 0x55f7c20d7b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b9080 0x55f7c21bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b9080 0x55f7c21bf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39256/0 (socket says 172.21.15.76:39256) 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fc259c640 1 -- 172.21.15.76:0/99418609 learned_addr learned my addr 172.21.15.76:0/99418609 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.197+0000 7f5fc359e640 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fc259c640 1 -- 172.21.15.76:0/99418609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 msgr2=0x55f7c21bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fc259c640 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fc259c640 1 -- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 msgr2=0x55f7c21bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fc259c640 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fc259c640 1 -- 172.21.15.76:0/99418609 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc41006e0 con 0x55f7c21b9080 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fc259c640 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b9080 0x55f7c21bf2a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40cf270 tx=0x7f5fb409e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f691babbdb3566 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/99418609 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40a2530 con 0x55f7c21b9080 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/99418609 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5fb40bcb50 con 0x55f7c21b9080 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7c21437a0 con 0x55f7c21b9080 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/99418609 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5fb40bcd10 con 0x55f7c21b9080 2024-03-23T15:43:59.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fcbe23740 10 client.15678 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.198+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f7c20d56d0 con 0x55f7c21b9080 2024-03-23T15:43:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/99418609 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f7c21437a0 con 0x55f7c21b9080 2024-03-23T15:43:59.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/99418609 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5fb407e070 con 0x55f7c21b9080 2024-03-23T15:43:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fcbe23740 10 client.15678 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fcbe23740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fcbe23740 20 client.15678 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c21d1970 0x55f7c21d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fcbe23740 4 client.15678 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:59.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c21151f0 con 0x55f7c21d1970 2024-03-23T15:43:59.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.199+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c21d1970 0x55f7c21d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.200+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c21d1970 0x55f7c21d3d90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d890515e735e4f72 server_cookie=3b0635dbbd5be570 in_seq=0 out_seq=0 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.200+0000 7f5fa6ff5640 10 client.15678 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/99418609 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f7c21151f0 con 0x55f7c21d1970 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fa6ff5640 10 client.15678 handle_command_reply: tid=0 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 client.15678 shutdown 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b9080 msgr2=0x55f7c21bf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b9080 0x55f7c21bf2a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40cf270 tx=0x7f5fb409e100 comp rx=0 tx=0).stop 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 shutdown_connections 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b9080 0x55f7c21bf2a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c21d1970 0x55f7c21d3d90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/99418609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fa6ff5640 0 client.15678 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 >> 172.21.15.76:0/99418609 conn(0x55f7c21b2e90 msgr2=0x55f7c21b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 shutdown_connections 2024-03-23T15:43:59.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.201+0000 7f5fcbe23740 1 -- 172.21.15.76:0/99418609 wait complete. 2024-03-23T15:43:59.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.202+0000 7f5fcbe23740 20 client.15678 trim_cache size 0 max 16384 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 Processor -- start 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 -- start start 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21d9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21c1c60 0x55f7c21be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7c21d43f0 con 0x55f7c21c1c60 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c21b8ea0 con 0x55f7c21b8760 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7c21bfcc0 con 0x55f7c21b9080 2024-03-23T15:43:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fc359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21c1c60 0x55f7c21be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21d9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59590/0 (socket says 172.21.15.76:59590) 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21d9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:42020/0 (socket says 172.21.15.76:42020) 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.230+0000 7f5fc2d9d640 1 -- 172.21.15.76:0/3284734752 learned_addr learned my addr 172.21.15.76:0/3284734752 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fc2d9d640 1 -- 172.21.15.76:0/3284734752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 msgr2=0x55f7c21d9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/3284734752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21d9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fc2d9d640 1 -- 172.21.15.76:0/3284734752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21c1c60 msgr2=0x55f7c21be000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/3284734752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21c1c60 0x55f7c21be000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fc2d9d640 1 -- 172.21.15.76:0/3284734752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c20b9c60 con 0x55f7c21b8760 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/3284734752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21cd4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8005ab0 tx=0x7f5fb8005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fce7b73badd2258 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fa8ff9640 1 -- 172.21.15.76:0/3284734752 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8045040 con 0x55f7c21b8760 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fa8ff9640 1 -- 172.21.15.76:0/3284734752 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb8047040 con 0x55f7c21b8760 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.231+0000 7f5fa8ff9640 1 -- 172.21.15.76:0/3284734752 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb802dea0 con 0x55f7c21b8760 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 -- 172.21.15.76:0/3284734752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 msgr2=0x55f7c21cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/3284734752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21cd4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8005ab0 tx=0x7f5fb8005a20 comp rx=0 tx=0).stop 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 -- 172.21.15.76:0/3284734752 shutdown_connections 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/3284734752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21c1c60 0x55f7c21be000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/3284734752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21b9080 0x55f7c21d9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/3284734752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b8760 0x55f7c21cd4f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 -- 172.21.15.76:0/3284734752 >> 172.21.15.76:0/3284734752 conn(0x55f7c21b2e90 msgr2=0x55f7c21bf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 -- 172.21.15.76:0/3284734752 shutdown_connections 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 -- 172.21.15.76:0/3284734752 wait complete. 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 -- start start 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 Processor -- start 2024-03-23T15:43:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.232+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b9080 0x55f7c21c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fcbe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7c21c62a0 con 0x55f7c21b8760 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7c21c6410 con 0x55f7c21b9080 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fcbe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7c21c65d0 con 0x55f7c21c1c60 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fcbe23740 10 client.0 _ll_register_callbacks cb 0x7f5fb40cf820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b9080 0x55f7c21c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b9080 0x55f7c21c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59606/0 (socket says 172.21.15.76:59606) 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 -- 172.21.15.76:0/1165909038 learned_addr learned my addr 172.21.15.76:0/1165909038 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc359e640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 -- 172.21.15.76:0/1165909038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 msgr2=0x55f7c21c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 -- 172.21.15.76:0/1165909038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 msgr2=0x55f7c21c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 -- 172.21.15.76:0/1165909038 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb8030070 con 0x55f7c21b9080 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc359e640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fc259c640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b9080 0x55f7c21c9d00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400bb90 tx=0x7f5fb40108b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eef54d9499c3188 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/1165909038 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40aa070 con 0x55f7c21b9080 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7c21b1cc0 con 0x55f7c21b9080 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.233+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/1165909038 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5fb40a2530 con 0x55f7c21b9080 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/1165909038 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5fb409a4a0 con 0x55f7c21b9080 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 10 client.25469 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f7c20df970 con 0x55f7c21b9080 2024-03-23T15:43:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/1165909038 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f7c21b1cc0 con 0x55f7c21b9080 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/1165909038 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f5fb407e040 con 0x55f7c21b9080 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 10 client.25469 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 20 client.25469 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c2241130 0x55f7c2243520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 4 client.25469 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f5fb40a6b80 con 0x55f7c2241130 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c2241130 0x55f7c2243520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fa6ff5640 10 client.25469 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.234+0000 7f5fc2d9d640 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c2241130 0x55f7c2243520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beacf3de1ecc2a98 server_cookie=42087f4e6f9b00e0 in_seq=0 out_seq=0 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fa6ff5640 1 -- 172.21.15.76:0/1165909038 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f5fb40a6b80 con 0x55f7c2241130 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fa6ff5640 10 client.25469 handle_command_reply: tid=0 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 client.25469 shutdown 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b9080 msgr2=0x55f7c21c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b9080 0x55f7c21c9d00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400bb90 tx=0x7f5fb40108b0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 shutdown_connections 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7c21c1c60 0x55f7c21c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7c2241130 0x55f7c2243520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7c21b9080 0x55f7c21c9d00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 --2- 172.21.15.76:0/1165909038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7c21b8760 0x55f7c21c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 >> 172.21.15.76:0/1165909038 conn(0x55f7c21b2e90 msgr2=0x55f7c21bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fa6ff5640 0 client.25469 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 shutdown_connections 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 1 -- 172.21.15.76:0/1165909038 wait complete. 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fcbe23740 20 client.25469 trim_cache size 0 max 16384 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-03-23T15:43:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "39d00f11-1d07-408d-956b-783bc6dbd054", 2024-03-23T15:43:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-03-23T15:43:59.247 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:59.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5fac0a2350 msgr2=0x7f5fac0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.236+0000 7f5fc916c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5fac0a2350 0x7f5fac0a4810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005fd0 tx=0x7f5fbc011810 comp rx=0 tx=0).stop 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fc40692b0 msgr2=0x7f5fc419dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fc40692b0 0x7f5fc419dfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009bc0 tx=0x7f5fb400b690 comp rx=0 tx=0).stop 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 shutdown_connections 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5fac0a2350 0x7f5fac0a4810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc4107690 0x7f5fc4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fc40692b0 0x7f5fc419dfb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 --2- 172.21.15.76:0/2909899144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fc40688f0 0x7f5fc419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 >> 172.21.15.76:0/2909899144 conn(0x7f5fc4077710 msgr2=0x7f5fc40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 shutdown_connections 2024-03-23T15:43:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.237+0000 7f5fc916c640 1 -- 172.21.15.76:0/2909899144 wait complete. 2024-03-23T15:43:59.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T15:43:59.251 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T15:43:59.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 -- 172.21.15.76:0/1153432190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 msgr2=0x7f699c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 --2- 172.21.15.76:0/1153432190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 0x7f699c107b40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f698c009890 tx=0x7f698c02bb70 comp rx=0 tx=0).stop 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 -- 172.21.15.76:0/1153432190 shutdown_connections 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 --2- 172.21.15.76:0/1153432190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0692d0 0x7f699c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 --2- 172.21.15.76:0/1153432190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 0x7f699c068d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 --2- 172.21.15.76:0/1153432190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 0x7f699c107b40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 -- 172.21.15.76:0/1153432190 >> 172.21.15.76:0/1153432190 conn(0x7f699c077710 msgr2=0x7f699c077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 -- 172.21.15.76:0/1153432190 shutdown_connections 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 -- 172.21.15.76:0/1153432190 wait complete. 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.357+0000 7f69a1f13640 1 Processor -- start 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f69a1f13640 1 -- start start 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f69a1f13640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 0x7f699c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f69a1f13640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0692d0 0x7f699c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f69a1f13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 0x7f699c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f69a1f13640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1982a0 con 0x7f699c107760 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f69a1f13640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f699c198410 con 0x7f699c068910 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f69a1f13640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1985d0 con 0x7f699c0692d0 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 0x7f699c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 0x7f699c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 0x7f699c19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59628/0 (socket says 172.21.15.76:59628) 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 -- 172.21.15.76:0/2718361097 learned_addr learned my addr 172.21.15.76:0/2718361097 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699affd640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0692d0 0x7f699c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 -- 172.21.15.76:0/2718361097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0692d0 msgr2=0x7f699c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0692d0 0x7f699c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 -- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 msgr2=0x7f699c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 0x7f699c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699b7fe640 1 -- 172.21.15.76:0/2718361097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6990009560 con 0x7f699c068910 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699affd640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0692d0 0x7f699c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.358+0000 7f699bfff640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 0x7f699c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.359+0000 7f699b7fe640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 0x7f699c19da50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.359+0000 7f6998ff9640 1 -- 172.21.15.76:0/2718361097 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c034950 con 0x7f699c068910 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.359+0000 7f6998ff9640 1 -- 172.21.15.76:0/2718361097 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698c02c9f0 con 0x7f699c068910 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.359+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c009490 con 0x7f699c068910 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.359+0000 7f6998ff9640 1 -- 172.21.15.76:0/2718361097 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c032600 con 0x7f699c068910 2024-03-23T15:43:59.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.359+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a47f0 con 0x7f699c068910 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.361+0000 7f6998ff9640 1 -- 172.21.15.76:0/2718361097 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 136983+0+0 (secure 0 0 0) 0x7f698c03b040 con 0x7f699c068910 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.361+0000 7f6998ff9640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f69840a2350 0x7f69840a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.361+0000 7f69a4bd1740 1 Processor -- start 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f6998ff9640 1 -- 172.21.15.76:0/2718361097 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f698c0efce0 con 0x7f699c068910 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f69a4bd1740 1 -- start start 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699affd640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f69840a2350 0x7f69840a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea4d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea56190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea58ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56145e971f30 con 0x56145ea4d1d0 2024-03-23T15:43:59.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56145e9bb810 con 0x56145ea4daf0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56145e9b8d40 con 0x56145ea566d0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea4d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea4d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39292/0 (socket says 172.21.15.76:39292) 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 -- 172.21.15.76:0/3890149400 learned_addr learned my addr 172.21.15.76:0/3890149400 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699affd640 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea56190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699bfff640 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea58ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699affd640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f69840a2350 0x7f69840a4810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f699c1992f0 tx=0x7f6990009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 -- 172.21.15.76:0/3890149400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 msgr2=0x56145ea58ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea58ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 -- 172.21.15.76:0/3890149400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 msgr2=0x56145ea56190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea56190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 -- 172.21.15.76:0/3890149400 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145e975ea0 con 0x56145ea4d1d0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.362+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea4d5b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f698c0bc5d0 tx=0x7f698c034ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb04a3476ef1133 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f6981ffb640 1 -- 172.21.15.76:0/3890149400 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0c06e0 con 0x56145ea4d1d0 2024-03-23T15:43:59.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f6981ffb640 1 -- 172.21.15.76:0/3890149400 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698c0c4d30 con 0x56145ea4d1d0 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f6981ffb640 1 -- 172.21.15.76:0/3890149400 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0035f0 con 0x56145ea4d1d0 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3890149400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 msgr2=0x56145ea4d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea4d5b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f698c0bc5d0 tx=0x7f698c034ab0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3890149400 shutdown_connections 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea58ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea56190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3890149400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea4d5b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.363+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3890149400 >> 172.21.15.76:0/3890149400 conn(0x56145ea47900 msgr2=0x56145ea47ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3890149400 shutdown_connections 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3890149400 wait complete. 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 -- start start 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 Processor -- start 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea53680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea53bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea566d0 0x56145ea54100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56145e9b4520 con 0x56145ea566d0 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56145e687430 con 0x56145ea4daf0 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56145e643e00 con 0x56145ea4d1d0 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.364+0000 7f69a4bd1740 10 client.0 _ll_register_callbacks cb 0x56145e952d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea53bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea53bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:59636/0 (socket says 172.21.15.76:59636) 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 -- 172.21.15.76:0/3664791066 learned_addr learned my addr 172.21.15.76:0/3664791066 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea53680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699bfff640 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea566d0 0x56145ea54100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 -- 172.21.15.76:0/3664791066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 msgr2=0x56145ea53680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea53680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 -- 172.21.15.76:0/3664791066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea566d0 msgr2=0x56145ea54100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea566d0 0x56145ea54100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 -- 172.21.15.76:0/3664791066 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a47f0 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f699affd640 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea53bc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f699000fc60 tx=0x7f6990004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b331f9dddf51992c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f697affd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f697affd640 1 -- 172.21.15.76:0/3664791066 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990027070 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f697affd640 1 -- 172.21.15.76:0/3664791066 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6990005020 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.365+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56145e6b37e0 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f697affd640 1 -- 172.21.15.76:0/3664791066 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f699000fea0 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 10 client.25481 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56145e9b2ba0 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f697affd640 1 -- 172.21.15.76:0/3664791066 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56145e6b37e0 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f697affd640 1 -- 172.21.15.76:0/3664791066 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6990035480 con 0x56145ea4daf0 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 10 client.25481 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 10 client.25481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 20 client.25481 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:59.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145ea66720 0x56145ea68b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 4 client.25481 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145e6f0720 con 0x56145ea66720 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.366+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145ea66720 0x56145ea68b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f697affd640 10 client.25481 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145ea66720 0x56145ea68b60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53e4b4ba48c1d03 server_cookie=22d2cb9cef2097c1 in_seq=0 out_seq=0 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f697affd640 1 -- 172.21.15.76:0/3664791066 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56145e6f0720 con 0x56145ea66720 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f697affd640 10 client.25481 handle_command_reply: tid=0 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 client.25481 shutdown 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 msgr2=0x56145ea53bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea53bc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f699000fc60 tx=0x7f6990004730 comp rx=0 tx=0).stop 2024-03-23T15:43:59.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 shutdown_connections 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea566d0 0x56145ea54100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145ea66720 0x56145ea68b60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f697affd640 0 client.25481 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea53bc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3664791066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 >> 172.21.15.76:0/3664791066 conn(0x56145ea47900 msgr2=0x56145ea46b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.367+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 shutdown_connections 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.368+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3664791066 wait complete. 2024-03-23T15:43:59.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.368+0000 7f69a4bd1740 20 client.25481 trim_cache size 0 max 16384 2024-03-23T15:43:59.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 Processor -- start 2024-03-23T15:43:59.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 -- start start 2024-03-23T15:43:59.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea55db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4daf0 0x56145ea53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea566d0 0x56145ea49440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56145e939430 con 0x56145ea4daf0 2024-03-23T15:43:59.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56145ea63210 con 0x56145ea566d0 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56145ea4c970 con 0x56145ea4d1d0 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4daf0 0x56145ea53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4daf0 0x56145ea53f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39306/0 (socket says 172.21.15.76:39306) 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699affd640 1 -- 172.21.15.76:0/1484337783 learned_addr learned my addr 172.21.15.76:0/1484337783 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699b7fe640 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea55db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699bfff640 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea566d0 0x56145ea49440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699b7fe640 1 -- 172.21.15.76:0/1484337783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea566d0 msgr2=0x56145ea49440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699b7fe640 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea566d0 0x56145ea49440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699b7fe640 1 -- 172.21.15.76:0/1484337783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4daf0 msgr2=0x56145ea53f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699b7fe640 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4daf0 0x56145ea53f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699b7fe640 1 -- 172.21.15.76:0/1484337783 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145e94eb20 con 0x56145ea4d1d0 2024-03-23T15:43:59.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.385+0000 7f699affd640 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4daf0 0x56145ea53f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.386+0000 7f699b7fe640 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea55db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f698c005b60 tx=0x7f698c0a7040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed802edb3f26d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.386+0000 7f69817fa640 1 -- 172.21.15.76:0/1484337783 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0c06e0 con 0x56145ea4d1d0 2024-03-23T15:43:59.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.386+0000 7f69817fa640 1 -- 172.21.15.76:0/1484337783 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698c02cb60 con 0x56145ea4d1d0 2024-03-23T15:43:59.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.386+0000 7f69817fa640 1 -- 172.21.15.76:0/1484337783 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0c48a0 con 0x56145ea4d1d0 2024-03-23T15:43:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 -- 172.21.15.76:0/1484337783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 msgr2=0x56145ea55db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea55db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f698c005b60 tx=0x7f698c0a7040 comp rx=0 tx=0).stop 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 -- 172.21.15.76:0/1484337783 shutdown_connections 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea566d0 0x56145ea49440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4daf0 0x56145ea53f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/1484337783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea4d1d0 0x56145ea55db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 -- 172.21.15.76:0/1484337783 >> 172.21.15.76:0/1484337783 conn(0x56145ea47900 msgr2=0x56145ea48d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 -- 172.21.15.76:0/1484337783 shutdown_connections 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 -- 172.21.15.76:0/1484337783 wait complete. 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 -- start start 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.387+0000 7f69a4bd1740 1 Processor -- start 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea50350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea50890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f69a4bd1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea50dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56145eacbaa0 con 0x56145ea4d1d0 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56145eacbbe0 con 0x56145ea4daf0 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f69a4bd1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56145eacbd20 con 0x56145ea566d0 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea50350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea50890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea50350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39308/0 (socket says 172.21.15.76:39308) 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 -- 172.21.15.76:0/3083564216 learned_addr learned my addr 172.21.15.76:0/3083564216 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f69a4bd1740 10 client.0 _ll_register_callbacks cb 0x7f6990009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699bfff640 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea50dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 -- 172.21.15.76:0/3083564216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 msgr2=0x56145ea50dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea50dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 -- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 msgr2=0x56145ea50890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea50890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 -- 172.21.15.76:0/3083564216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145e9b2ba0 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699affd640 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea50890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f699b7fe640 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea50350 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f698c0bc800 tx=0x7f698c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac2476846b7640 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f697affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.388+0000 7f697affd640 1 -- 172.21.15.76:0/3083564216 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0bbfa0 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f697affd640 1 -- 172.21.15.76:0/3083564216 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f698c0c06e0 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f698c0c4d30 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f697affd640 1 -- 172.21.15.76:0/3083564216 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f698c0035f0 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 10 client.15702 fetch_fsmap learned FSMap version 22 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56145e975ea0 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f697affd640 1 -- 172.21.15.76:0/3083564216 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698c0c9020 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f697affd640 1 -- 172.21.15.76:0/3083564216 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f698c0edc10 con 0x56145ea4d1d0 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 10 client.15702 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:43:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 20 client.15702 populate_metadata read hostname 'smithi076' 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145eaccfd0 0x56145eacf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 4 client.15702 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56145e73f890 con 0x56145eaccfd0 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.389+0000 7f699affd640 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145eaccfd0 0x56145eacf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f699affd640 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145eaccfd0 0x56145eacf3c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e83ac98d38df73 server_cookie=9d62fb98255e3091 in_seq=0 out_seq=0 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f697affd640 10 client.15702 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f697affd640 1 -- 172.21.15.76:0/3083564216 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56145e73f890 con 0x56145eaccfd0 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f697affd640 10 client.15702 handle_command_reply: tid=0 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 client.15702 shutdown 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 msgr2=0x56145ea50350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea50350 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f698c0bc800 tx=0x7f698c003f70 comp rx=0 tx=0).stop 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 shutdown_connections 2024-03-23T15:43:59.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145ea566d0 0x56145ea50dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56145eaccfd0 0x56145eacf3c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145ea4daf0 0x56145ea50890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 --2- 172.21.15.76:0/3083564216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56145ea4d1d0 0x56145ea50350 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 >> 172.21.15.76:0/3083564216 conn(0x56145ea47900 msgr2=0x56145ea495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f697affd640 0 client.15702 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 shutdown_connections 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 1 -- 172.21.15.76:0/3083564216 wait complete. 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.390+0000 7f69a4bd1740 20 client.15702 trim_cache size 0 max 16384 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:43:59.400 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout: "39d00f11-1d07-408d-956b-783bc6dbd054": { 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/", 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "39d00f11-1d07-408d-956b-783bc6dbd054", 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:43:59.401 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f69840a2350 msgr2=0x7f69840a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f69840a2350 0x7f69840a4810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f699c1992f0 tx=0x7f6990009260 comp rx=0 tx=0).stop 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 msgr2=0x7f699c19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 0x7f699c19da50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c004530 comp rx=0 tx=0).stop 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 shutdown_connections 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f69840a2350 0x7f69840a4810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c107760 0x7f699c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0692d0 0x7f699c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 --2- 172.21.15.76:0/2718361097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f699c068910 0x7f699c19da50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:43:59.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 >> 172.21.15.76:0/2718361097 conn(0x7f699c077710 msgr2=0x7f699c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:43:59.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 shutdown_connections 2024-03-23T15:43:59.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:43:59.391+0000 7f69a1f13640 1 -- 172.21.15.76:0/2718361097 wait complete. 2024-03-23T15:43:59.404 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:43:59 smithi076 ceph-mon[32176]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T15:43:59.405 INFO:tasks.cephfs.filesystem:scrub status for tag:39d00f11-1d07-408d-956b-783bc6dbd054 - {'path': '/', 'tag': '39d00f11-1d07-408d-956b-783bc6dbd054', 'options': 'recursive,force'} 2024-03-23T15:43:59.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:43:59 smithi192 ceph-mon[36653]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T15:43:59.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:43:59 smithi183 ceph-mon[36296]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-23T15:44:00.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:00 smithi192 ceph-mon[36653]: scrub queued for path: / 2024-03-23T15:44:00.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:00 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:00.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:00 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:00.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:00 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:00.783 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:00 smithi183 ceph-mon[36296]: scrub queued for path: / 2024-03-23T15:44:00.783 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:00 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:00.783 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:00 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:00.783 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:00 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:00.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-mon[32176]: scrub queued for path: / 2024-03-23T15:44:00.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:00.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:00.784 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:01.078 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:44:00.596+0000 7fc614669700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:44:01.078 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:44:00.596+0000 7fc614669700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:44:01.078 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:44:00.596+0000 7fc614669700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:44:01.078 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:44:00.596+0000 7fc614669700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:44:01.078 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:44:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:44:00.596+0000 7fc614669700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-23T15:44:01.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:01 smithi192 ceph-mon[36653]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-23T15:44:01.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:01 smithi076 ceph-mon[32176]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-23T15:44:01.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:01 smithi183 ceph-mon[36296]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-23T15:44:03.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:03 smithi192 ceph-mon[36653]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-23T15:44:03.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:03 smithi192 ceph-mon[36653]: mgrmap e24: x(active, since 42s), standbys: y, z 2024-03-23T15:44:03.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:03 smithi192 ceph-mon[36653]: scrub summary: idle 2024-03-23T15:44:03.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:03 smithi076 ceph-mon[32176]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-23T15:44:03.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:03 smithi076 ceph-mon[32176]: mgrmap e24: x(active, since 42s), standbys: y, z 2024-03-23T15:44:03.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:03 smithi076 ceph-mon[32176]: scrub summary: idle 2024-03-23T15:44:03.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:03 smithi183 ceph-mon[36296]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-23T15:44:03.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:03 smithi183 ceph-mon[36296]: mgrmap e24: x(active, since 42s), standbys: y, z 2024-03-23T15:44:03.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:03 smithi183 ceph-mon[36296]: scrub summary: idle 2024-03-23T15:44:05.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:05 smithi192 ceph-mon[36653]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-23T15:44:05.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:05 smithi076 ceph-mon[32176]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-23T15:44:05.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:05 smithi183 ceph-mon[36296]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-23T15:44:07.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:07 smithi192 ceph-mon[36653]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-23T15:44:07.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:07 smithi076 ceph-mon[32176]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-23T15:44:07.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:07 smithi183 ceph-mon[36296]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-03-23T15:44:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 -- 172.21.15.76:0/3484989121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac068d90 msgr2=0x7fd8ac0691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3484989121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac068d90 0x7fd8ac0691f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009900 tx=0x7fd89c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 -- 172.21.15.76:0/3484989121 shutdown_connections 2024-03-23T15:44:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3484989121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac069730 0x7fd8ac10e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3484989121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac068d90 0x7fd8ac0691f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3484989121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ac107800 0x7fd8ac107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 -- 172.21.15.76:0/3484989121 >> 172.21.15.76:0/3484989121 conn(0x7fd8ac077460 msgr2=0x7fd8ac077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 -- 172.21.15.76:0/3484989121 shutdown_connections 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.763+0000 7fd8b3807640 1 -- 172.21.15.76:0/3484989121 wait complete. 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 Processor -- start 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 -- start start 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac068d90 0x7fd8ac19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac069730 0x7fd8ac19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ac107800 0x7fd8ac197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac198150 con 0x7fd8ac069730 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac1982c0 con 0x7fd8ac068d90 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.764+0000 7fd8b3807640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac198480 con 0x7fd8ac107800 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b0da2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac069730 0x7fd8ac19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac068d90 0x7fd8ac19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac068d90 0x7fd8ac19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:53782/0 (socket says 172.21.15.76:53782) 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 -- 172.21.15.76:0/3883363866 learned_addr learned my addr 172.21.15.76:0/3883363866 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b0da2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac069730 0x7fd8ac19de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40156/0 (socket says 172.21.15.76:40156) 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ac107800 0x7fd8ac197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 -- 172.21.15.76:0/3883363866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ac107800 msgr2=0x7fd8ac197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ac107800 0x7fd8ac197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 -- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac069730 msgr2=0x7fd8ac19de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac069730 0x7fd8ac19de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 -- 172.21.15.76:0/3883363866 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c009560 con 0x7fd8ac068d90 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac069730 0x7fd8ac19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b15a3640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac068d90 0x7fd8ac19d950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800e9f0 tx=0x7fd8a800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8a27fc640 1 -- 172.21.15.76:0/3883363866 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8009720 con 0x7fd8ac068d90 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8a27fc640 1 -- 172.21.15.76:0/3883363866 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a800cd80 con 0x7fd8ac068d90 2024-03-23T15:44:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8a27fc640 1 -- 172.21.15.76:0/3883363866 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8010690 con 0x7fd8ac068d90 2024-03-23T15:44:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.765+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac198760 con 0x7fd8ac068d90 2024-03-23T15:44:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.766+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1a49f0 con 0x7fd8ac068d90 2024-03-23T15:44:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8a27fc640 1 -- 172.21.15.76:0/3883363866 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7fd8a80046c0 con 0x7fd8ac068d90 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 Processor -- start 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 -- start start 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e7b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561298e361b0 con 0x561298e7bf00 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561298de8f80 con 0x561298e84cd0 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561298de64b0 con 0x561298e7b5e0 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b15a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b1da4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e7b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b1da4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e7b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:47826/0 (socket says 172.21.15.76:47826) 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3991473347 learned_addr learned my addr 172.21.15.76:0/3991473347 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.769+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3991473347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 msgr2=0x561298e870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3991473347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 msgr2=0x561298e84790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e84790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3991473347 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561298d77810 con 0x561298e7b5e0 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8a27fc640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8980a2480 0x7fd8980a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b15a3640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8980a2480 0x7fd8980a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e7b9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009fc0 tx=0x7fd8a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43e2c11ea89bfdf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8a27fc640 1 -- 172.21.15.76:0/3883363866 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd8a8014070 con 0x7fd8ac068d90 2024-03-23T15:44:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8977fe640 1 -- 172.21.15.76:0/3991473347 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4004d20 con 0x561298e7b5e0 2024-03-23T15:44:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8977fe640 1 -- 172.21.15.76:0/3991473347 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a4004e80 con 0x561298e7b5e0 2024-03-23T15:44:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.770+0000 7fd8977fe640 1 -- 172.21.15.76:0/3991473347 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4005280 con 0x561298e7b5e0 2024-03-23T15:44:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3991473347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 msgr2=0x561298e7b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b15a3640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8980a2480 0x7fd8980a4940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80cbc50 tx=0x7fd8a80d4060 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e7b9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009fc0 tx=0x7fd8a40118d0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3991473347 shutdown_connections 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3991473347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e7b9c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3991473347 >> 172.21.15.76:0/3991473347 conn(0x561298e75f00 msgr2=0x561298e762e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3991473347 shutdown_connections 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3991473347 wait complete. 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 -- start start 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.771+0000 7fd8b63e9740 1 Processor -- start 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e82260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561298de1c90 con 0x561298e7bf00 2024-03-23T15:44:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561298d86c50 con 0x561298e84cd0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561298ab5250 con 0x561298e7b5e0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b63e9740 10 client.0 _ll_register_callbacks cb 0x561298dd5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b0da2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e82260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e81d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:47834/0 (socket says 172.21.15.76:47834) 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 -- 172.21.15.76:0/2273632918 learned_addr learned my addr 172.21.15.76:0/2273632918 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 -- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 msgr2=0x561298e827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 -- 172.21.15.76:0/2273632918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 msgr2=0x561298e82260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e82260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.772+0000 7fd8b1da4640 1 -- 172.21.15.76:0/2273632918 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a4009c70 con 0x561298e7b5e0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e82260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e81d20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4004e50 tx=0x7fd8a40046f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b9858eb2f8bfd99 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd894ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd894ff9640 1 -- 172.21.15.76:0/2273632918 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4004490 con 0x561298e7b5e0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd894ff9640 1 -- 172.21.15.76:0/2273632918 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd8a400f4e0 con 0x561298e7b5e0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561298ae1600 con 0x561298e7b5e0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd894ff9640 1 -- 172.21.15.76:0/2273632918 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd8a400f6a0 con 0x561298e7b5e0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 10 client.25525 fetch_fsmap learned FSMap version 22 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561298de0310 con 0x561298e7b5e0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd894ff9640 1 -- 172.21.15.76:0/2273632918 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561298ae1600 con 0x561298e7b5e0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd894ff9640 1 -- 172.21.15.76:0/2273632918 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd8a40288c0 con 0x561298e7b5e0 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 10 client.25525 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 20 client.25525 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298e94b00 0x561298e96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 4 client.25525 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.773+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561298b321b0 con 0x561298e94b00 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.774+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298e94b00 0x561298e96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.774+0000 7fd894ff9640 10 client.25525 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.774+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298e94b00 0x561298e96ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=433b372fbc5c8dca server_cookie=7ff7c8f5c977c49e in_seq=0 out_seq=0 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd894ff9640 1 -- 172.21.15.76:0/2273632918 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561298b321b0 con 0x561298e94b00 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd894ff9640 10 client.25525 handle_command_reply: tid=0 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 client.25525 shutdown 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 msgr2=0x561298e81d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e81d20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4004e50 tx=0x7fd8a40046f0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 shutdown_connections 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298e94b00 0x561298e96ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd894ff9640 0 client.25525 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e82260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/2273632918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7b5e0 0x561298e81d20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 >> 172.21.15.76:0/2273632918 conn(0x561298e75f00 msgr2=0x561298e77c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 shutdown_connections 2024-03-23T15:44:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 1 -- 172.21.15.76:0/2273632918 wait complete. 2024-03-23T15:44:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.775+0000 7fd8b63e9740 20 client.25525 trim_cache size 0 max 16384 2024-03-23T15:44:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 Processor -- start 2024-03-23T15:44:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 -- start start 2024-03-23T15:44:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7b5e0 0x561298e903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7bf00 0x561298e83b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561298d67660 con 0x561298e7b5e0 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561298e76ad0 con 0x561298e84cd0 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561298e847f0 con 0x561298e7bf00 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7b5e0 0x561298e903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7b5e0 0x561298e903a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40170/0 (socket says 172.21.15.76:40170) 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3040032942 learned_addr learned my addr 172.21.15.76:0/3040032942 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7bf00 0x561298e83b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3040032942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7bf00 msgr2=0x561298e83b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7bf00 0x561298e83b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3040032942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 msgr2=0x561298e9c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e9c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.790+0000 7fd8b1da4640 1 -- 172.21.15.76:0/3040032942 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561298d7c810 con 0x561298e7b5e0 2024-03-23T15:44:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.791+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7b5e0 0x561298e903a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009730 tx=0x7fd8a4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36417c2b31abea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.791+0000 7fd896ffd640 1 -- 172.21.15.76:0/3040032942 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4005000 con 0x561298e7b5e0 2024-03-23T15:44:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.791+0000 7fd896ffd640 1 -- 172.21.15.76:0/3040032942 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a4004c60 con 0x561298e7b5e0 2024-03-23T15:44:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.791+0000 7fd896ffd640 1 -- 172.21.15.76:0/3040032942 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4005510 con 0x561298e7b5e0 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3040032942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7b5e0 msgr2=0x561298e903a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7b5e0 0x561298e903a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009730 tx=0x7fd8a4014040 comp rx=0 tx=0).stop 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3040032942 shutdown_connections 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e84cd0 0x561298e9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e7bf00 0x561298e83b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/3040032942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7b5e0 0x561298e903a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3040032942 >> 172.21.15.76:0/3040032942 conn(0x561298e75f00 msgr2=0x561298e83440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3040032942 shutdown_connections 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 -- 172.21.15.76:0/3040032942 wait complete. 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 -- start start 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.792+0000 7fd8b63e9740 1 Processor -- start 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e7b5e0 0x561298e7fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e8c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b63e9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e84cd0 0x561298ef5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561298ef62c0 con 0x561298e7bf00 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561298ef6430 con 0x561298e7b5e0 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b63e9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561298ef65f0 con 0x561298e84cd0 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b63e9740 10 client.0 _ll_register_callbacks cb 0x7fd8a80d1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e8c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b1da4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e7b5e0 0x561298e7fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e8c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40184/0 (socket says 172.21.15.76:40184) 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 -- 172.21.15.76:0/144679285 learned_addr learned my addr 172.21.15.76:0/144679285 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e84cd0 0x561298ef5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 -- 172.21.15.76:0/144679285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e84cd0 msgr2=0x561298ef5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e84cd0 0x561298ef5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 -- 172.21.15.76:0/144679285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e7b5e0 msgr2=0x561298e7fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e7b5e0 0x561298e7fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.793+0000 7fd8b0da2640 1 -- 172.21.15.76:0/144679285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561298de0310 con 0x561298e7bf00 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b0da2640 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e8c9d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd89c02c170 tx=0x7fd89c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2577b2d7951ea286 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd894ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd894ff9640 1 -- 172.21.15.76:0/144679285 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c049d20 con 0x561298e7bf00 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd894ff9640 1 -- 172.21.15.76:0/144679285 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd89c045040 con 0x561298e7bf00 2024-03-23T15:44:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561298e06790 con 0x561298e7bf00 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd894ff9640 1 -- 172.21.15.76:0/144679285 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd89c049e80 con 0x561298e7bf00 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 10 client.15726 fetch_fsmap learned FSMap version 22 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561298da25a0 con 0x561298e7bf00 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd894ff9640 1 -- 172.21.15.76:0/144679285 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561298e06790 con 0x561298e7bf00 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd894ff9640 1 -- 172.21.15.76:0/144679285 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd89c052020 con 0x561298e7bf00 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 10 client.15726 fetch_fsmap finished waiting for FSMap version 22 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 20 client.15726 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298eff1b0 0x561298f015a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 4 client.15726 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.794+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561298b1e550 con 0x561298eff1b0 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.795+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298eff1b0 0x561298f015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.795+0000 7fd894ff9640 10 client.15726 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.795+0000 7fd8b1da4640 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298eff1b0 0x561298f015a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372061b0815a4a5d server_cookie=118e5337e7ba09d9 in_seq=0 out_seq=0 2024-03-23T15:44:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.801+0000 7fd894ff9640 1 -- 172.21.15.76:0/144679285 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561298b1e550 con 0x561298eff1b0 2024-03-23T15:44:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.801+0000 7fd894ff9640 10 client.15726 handle_command_reply: tid=0 2024-03-23T15:44:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.801+0000 7fd8b63e9740 1 client.15726 shutdown 2024-03-23T15:44:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 msgr2=0x561298e8c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e8c9d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd89c02c170 tx=0x7fd89c005be0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 shutdown_connections 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561298e84cd0 0x561298ef5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561298e7bf00 0x561298e8c9d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x561298eff1b0 0x561298f015a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 --2- 172.21.15.76:0/144679285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561298e7b5e0 0x561298e7fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 >> 172.21.15.76:0/144679285 conn(0x561298e75f00 msgr2=0x561298e81ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd894ff9640 0 client.15726 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 shutdown_connections 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 1 -- 172.21.15.76:0/144679285 wait complete. 2024-03-23T15:44:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.802+0000 7fd8b63e9740 20 client.15726 trim_cache size 0 max 16384 2024-03-23T15:44:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:44:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:44:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:44:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8980a2480 msgr2=0x7fd8980a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8980a2480 0x7fd8980a4940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80cbc50 tx=0x7fd8a80d4060 comp rx=0 tx=0).stop 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac068d90 msgr2=0x7fd8ac19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac068d90 0x7fd8ac19d950 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800e9f0 tx=0x7fd8a800edf0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 shutdown_connections 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8980a2480 0x7fd8980a4940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8ac107800 0x7fd8ac197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ac069730 0x7fd8ac19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 --2- 172.21.15.76:0/3883363866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ac068d90 0x7fd8ac19d950 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 >> 172.21.15.76:0/3883363866 conn(0x7fd8ac077460 msgr2=0x7fd8ac0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 shutdown_connections 2024-03-23T15:44:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:07.803+0000 7fd8b3807640 1 -- 172.21.15.76:0/3883363866 wait complete. 2024-03-23T15:44:09.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:09 smithi192 ceph-mon[36653]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-23T15:44:09.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:09 smithi076 ceph-mon[32176]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-23T15:44:09.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:09 smithi183 ceph-mon[36296]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr:Note: switching to 'd39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b'. 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr:state without impacting any branches by switching back to a branch. 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-23T15:44:10.660 INFO:tasks.workunit.client.0.smithi076.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr: git switch -c 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr:Or undo this operation with: 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr: git switch - 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-03-23T15:44:10.661 INFO:tasks.workunit.client.0.smithi076.stderr:HEAD is now at d39ab7bb1f1 qa/suites/fs/workload: enable snap_schedule early 2024-03-23T15:44:10.665 DEBUG:teuthology.orchestra.run.smithi076:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-23T15:44:10.687 INFO:tasks.workunit.client.0.smithi076.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-23T15:44:10.689 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T15:44:10.689 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-23T15:44:11.445 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-23T15:44:11.513 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-23T15:44:11.553 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T15:44:11.554 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T15:44:11.554 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-23T15:44:11.569 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:11 smithi192 ceph-mon[36653]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-03-23T15:44:11.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:11 smithi076 ceph-mon[32176]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-03-23T15:44:11.593 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T15:44:11.596 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:44:11.596 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-23T15:44:11.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:11 smithi183 ceph-mon[36296]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-03-23T15:44:11.651 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-23T15:44:11.653 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-23T15:44:11.653 DEBUG:teuthology.orchestra.run.smithi076:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-23T15:44:11.713 INFO:tasks.workunit.client.0.smithi076.stderr:+ mkdir -p fsstress 2024-03-23T15:44:11.715 INFO:tasks.workunit.client.0.smithi076.stderr:+ pushd fsstress 2024-03-23T15:44:11.715 INFO:tasks.workunit.client.0.smithi076.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-23T15:44:11.715 INFO:tasks.workunit.client.0.smithi076.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-23T15:44:13.309 INFO:tasks.workunit.client.0.smithi076.stderr:+ tar xzf ltp-full.tgz 2024-03-23T15:44:13.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:13 smithi076 ceph-mon[32176]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-03-23T15:44:13.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:13 smithi183 ceph-mon[36296]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-03-23T15:44:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:13 smithi192 ceph-mon[36653]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-03-23T15:44:15.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:15 smithi076 ceph-mon[32176]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-23T15:44:15.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:15 smithi183 ceph-mon[36296]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-23T15:44:15.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:15 smithi192 ceph-mon[36653]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-23T15:44:17.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:17 smithi183 ceph-mon[36296]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-23T15:44:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:17 smithi192 ceph-mon[36653]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-23T15:44:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:17 smithi076 ceph-mon[32176]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-23T15:44:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:19 smithi183 ceph-mon[36296]: pgmap v32: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 280 KiB/s wr, 11 op/s 2024-03-23T15:44:19.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:19 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:19 smithi192 ceph-mon[36653]: pgmap v32: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 280 KiB/s wr, 11 op/s 2024-03-23T15:44:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:19 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:19 smithi076 ceph-mon[32176]: pgmap v32: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 280 KiB/s wr, 11 op/s 2024-03-23T15:44:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:19 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.899+0000 7fca57295640 1 -- 172.21.15.76:0/1250814173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 msgr2=0x7fca50100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.899+0000 7fca57295640 1 --2- 172.21.15.76:0/1250814173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50100ce0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fca40009900 tx=0x7fca4002bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.899+0000 7fca57295640 1 -- 172.21.15.76:0/1250814173 shutdown_connections 2024-03-23T15:44:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.899+0000 7fca57295640 1 --2- 172.21.15.76:0/1250814173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca50101220 0x7fca5010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.899+0000 7fca57295640 1 --2- 172.21.15.76:0/1250814173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50100ce0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.899+0000 7fca57295640 1 --2- 172.21.15.76:0/1250814173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca501055b0 0x7fca50105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.899+0000 7fca57295640 1 -- 172.21.15.76:0/1250814173 >> 172.21.15.76:0/1250814173 conn(0x7fca50077460 msgr2=0x7fca50077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 -- 172.21.15.76:0/1250814173 shutdown_connections 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 -- 172.21.15.76:0/1250814173 wait complete. 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 Processor -- start 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 -- start start 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca50101220 0x7fca501990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca501055b0 0x7fca501a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fca501998e0 con 0x7fca50100860 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca50199a50 con 0x7fca501055b0 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca57295640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fca501a41d0 con 0x7fca50101220 2024-03-23T15:44:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca55031640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca55031640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41566/0 (socket says 172.21.15.76:41566) 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca55832640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca501055b0 0x7fca501a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.900+0000 7fca55031640 1 -- 172.21.15.76:0/1325651363 learned_addr learned my addr 172.21.15.76:0/1325651363 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca54830640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca50101220 0x7fca501990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca55031640 1 -- 172.21.15.76:0/1325651363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca50101220 msgr2=0x7fca501990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca55031640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca50101220 0x7fca501990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca55031640 1 -- 172.21.15.76:0/1325651363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca501055b0 msgr2=0x7fca501a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca55031640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca501055b0 0x7fca501a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca55031640 1 -- 172.21.15.76:0/1325651363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca40009560 con 0x7fca50100860 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca55832640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca501055b0 0x7fca501a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca55031640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50198b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9f0 tx=0x7fca4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca467fc640 1 -- 172.21.15.76:0/1325651363 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c009700 con 0x7fca50100860 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca467fc640 1 -- 172.21.15.76:0/1325651363 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4c00cd80 con 0x7fca50100860 2024-03-23T15:44:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca501a44b0 con 0x7fca50100860 2024-03-23T15:44:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca467fc640 1 -- 172.21.15.76:0/1325651363 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c018920 con 0x7fca50100860 2024-03-23T15:44:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.901+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a4990 con 0x7fca50100860 2024-03-23T15:44:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.903+0000 7fca467fc640 1 -- 172.21.15.76:0/1325651363 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7fca4c018a80 con 0x7fca50100860 2024-03-23T15:44:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 Processor -- start 2024-03-23T15:44:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 -- start start 2024-03-23T15:44:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca467fc640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fca3c0a2480 0x7fca3c0a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f25e0 0x560ba80f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba80fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ba80ad1b0 con 0x560ba80fbcd0 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ba805ff80 con 0x560ba80f2f00 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca59e71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560ba805d4b0 con 0x560ba80f25e0 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca55832640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fca3c0a2480 0x7fca3c0a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca55031640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f25e0 0x560ba80f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca467fc640 1 -- 172.21.15.76:0/1325651363 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fca4c014070 con 0x7fca50100860 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca55832640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba80fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca55031640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f25e0 0x560ba80f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:39818/0 (socket says 172.21.15.76:39818) 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.904+0000 7fca55031640 1 -- 172.21.15.76:0/450291396 learned_addr learned my addr 172.21.15.76:0/450291396 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca54830640 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55031640 1 -- 172.21.15.76:0/450291396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 msgr2=0x560ba80fb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55031640 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80fb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55031640 1 -- 172.21.15.76:0/450291396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 msgr2=0x560ba80fe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55031640 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba80fe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55031640 1 -- 172.21.15.76:0/450291396 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba7fee810 con 0x560ba80f25e0 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55832640 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba80fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55832640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fca3c0a2480 0x7fca3c0a4940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca48011720 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca55031640 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f25e0 0x560ba80f29c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9c0 tx=0x7fca4c09a810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecae23c686a8afca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca3b7fe640 1 -- 172.21.15.76:0/450291396 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c09d030 con 0x560ba80f25e0 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca3b7fe640 1 -- 172.21.15.76:0/450291396 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4c09aeb0 con 0x560ba80f25e0 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.905+0000 7fca3b7fe640 1 -- 172.21.15.76:0/450291396 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c001aa0 con 0x560ba80f25e0 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 -- 172.21.15.76:0/450291396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f25e0 msgr2=0x560ba80f29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f25e0 0x560ba80f29c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9c0 tx=0x7fca4c09a810 comp rx=0 tx=0).stop 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 -- 172.21.15.76:0/450291396 shutdown_connections 2024-03-23T15:44:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba80fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 --2- 172.21.15.76:0/450291396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f25e0 0x560ba80f29c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 -- 172.21.15.76:0/450291396 >> 172.21.15.76:0/450291396 conn(0x560ba80ecf00 msgr2=0x560ba80ed2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 -- 172.21.15.76:0/450291396 shutdown_connections 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 -- 172.21.15.76:0/450291396 wait complete. 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 -- start start 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.906+0000 7fca59e71740 1 Processor -- start 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca59e71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba80f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca59e71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f2f00 0x560ba80f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca59e71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca59e71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ba8058c90 con 0x560ba80f2f00 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca59e71740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ba7ffdc50 con 0x560ba80f25e0 2024-03-23T15:44:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca59e71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560ba7d2c250 con 0x560ba80fbcd0 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca59e71740 10 client.0 _ll_register_callbacks cb 0x560ba804c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f2f00 0x560ba80f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f2f00 0x560ba80f92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41574/0 (socket says 172.21.15.76:41574) 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 -- 172.21.15.76:0/3914002343 learned_addr learned my addr 172.21.15.76:0/3914002343 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca55031640 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba80f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca55832640 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 -- 172.21.15.76:0/3914002343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 msgr2=0x560ba80f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 -- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 msgr2=0x560ba80f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba80f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 -- 172.21.15.76:0/3914002343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a4990 con 0x560ba80f2f00 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca54830640 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f2f00 0x560ba80f92d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fca40005ec0 tx=0x7fca400048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ce32d5a7a1af0c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.907+0000 7fca38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca38ff9640 1 -- 172.21.15.76:0/3914002343 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4002c610 con 0x560ba80f2f00 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca38ff9640 1 -- 172.21.15.76:0/3914002343 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fca40004040 con 0x560ba80f2f00 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ba807d790 con 0x560ba80f2f00 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca38ff9640 1 -- 172.21.15.76:0/3914002343 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fca40004220 con 0x560ba80f2f00 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 10 client.15738 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560ba8057310 con 0x560ba80f2f00 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca38ff9640 1 -- 172.21.15.76:0/3914002343 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560ba807d790 con 0x560ba80f2f00 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca38ff9640 1 -- 172.21.15.76:0/3914002343 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fca4003c040 con 0x560ba80f2f00 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 10 client.15738 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 20 client.15738 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8110060 0x560ba81124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 4 client.15738 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ba7d58600 con 0x560ba8110060 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.908+0000 7fca55031640 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8110060 0x560ba81124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.909+0000 7fca55031640 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8110060 0x560ba81124a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6232327912582ef2 server_cookie=bc35f07981a87dde in_seq=0 out_seq=0 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.909+0000 7fca38ff9640 10 client.15738 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.909+0000 7fca38ff9640 1 -- 172.21.15.76:0/3914002343 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560ba7d58600 con 0x560ba8110060 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.909+0000 7fca38ff9640 10 client.15738 handle_command_reply: tid=0 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.909+0000 7fca59e71740 1 client.15738 shutdown 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f2f00 msgr2=0x560ba80f92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f2f00 0x560ba80f92d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fca40005ec0 tx=0x7fca400048f0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 shutdown_connections 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f2f00 0x560ba80f92d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8110060 0x560ba81124a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 --2- 172.21.15.76:0/3914002343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba80f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca38ff9640 0 client.15738 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 >> 172.21.15.76:0/3914002343 conn(0x560ba80ecf00 msgr2=0x560ba80eec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 shutdown_connections 2024-03-23T15:44:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 1 -- 172.21.15.76:0/3914002343 wait complete. 2024-03-23T15:44:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.910+0000 7fca59e71740 20 client.15738 trim_cache size 0 max 16384 2024-03-23T15:44:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 Processor -- start 2024-03-23T15:44:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 -- start start 2024-03-23T15:44:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f25e0 0x560ba810b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ba7fde660 con 0x560ba80f25e0 2024-03-23T15:44:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ba80edd30 con 0x560ba80f2f00 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca59e71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560ba80eb3f0 con 0x560ba80fbcd0 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f25e0 0x560ba810b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f25e0 0x560ba810b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41590/0 (socket says 172.21.15.76:41590) 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 -- 172.21.15.76:0/4253288490 learned_addr learned my addr 172.21.15.76:0/4253288490 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca54830640 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55832640 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 -- 172.21.15.76:0/4253288490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 msgr2=0x560ba80ebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80ebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 -- 172.21.15.76:0/4253288490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 msgr2=0x560ba80f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80f9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 -- 172.21.15.76:0/4253288490 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba7ff3810 con 0x560ba80f25e0 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca54830640 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:44:20.933 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55832640 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca55031640 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f25e0 0x560ba810b7b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fca4c09ae40 tx=0x7fca4c097280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc99d8b0be5f0b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca3affd640 1 -- 172.21.15.76:0/4253288490 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c09d040 con 0x560ba80f25e0 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca3affd640 1 -- 172.21.15.76:0/4253288490 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4c0a3620 con 0x560ba80f25e0 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.925+0000 7fca3affd640 1 -- 172.21.15.76:0/4253288490 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c09a560 con 0x560ba80f25e0 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 -- 172.21.15.76:0/4253288490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f25e0 msgr2=0x560ba810b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f25e0 0x560ba810b7b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fca4c09ae40 tx=0x7fca4c097280 comp rx=0 tx=0).stop 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 -- 172.21.15.76:0/4253288490 shutdown_connections 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80fbcd0 0x560ba80ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f2f00 0x560ba80f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 --2- 172.21.15.76:0/4253288490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80f25e0 0x560ba810b7b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 -- 172.21.15.76:0/4253288490 >> 172.21.15.76:0/4253288490 conn(0x560ba80ecf00 msgr2=0x560ba80eb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 -- 172.21.15.76:0/4253288490 shutdown_connections 2024-03-23T15:44:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 -- 172.21.15.76:0/4253288490 wait complete. 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 -- start start 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.926+0000 7fca59e71740 1 Processor -- start 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca59e71740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba81037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca59e71740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f2f00 0x560ba8103d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca59e71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba8104270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca59e71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560ba81753e0 con 0x560ba80fbcd0 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca59e71740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ba8175520 con 0x560ba80f25e0 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca59e71740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560ba81756e0 con 0x560ba80f2f00 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca59e71740 10 client.0 _ll_register_callbacks cb 0x7fca480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba8104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55031640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba81037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca54830640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f2f00 0x560ba8103d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba8104270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41592/0 (socket says 172.21.15.76:41592) 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 -- 172.21.15.76:0/1595082515 learned_addr learned my addr 172.21.15.76:0/1595082515 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55031640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba81037f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:56792/0 (socket says 172.21.15.76:56792) 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 -- 172.21.15.76:0/1595082515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f2f00 msgr2=0x560ba8103d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f2f00 0x560ba8103d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 -- 172.21.15.76:0/1595082515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 msgr2=0x560ba81037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba81037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 -- 172.21.15.76:0/1595082515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba7fee810 con 0x560ba80fbcd0 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca55832640 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba8104270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fca48009930 tx=0x7fca48009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840f7c33190a52bf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.927+0000 7fca38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca38ff9640 1 -- 172.21.15.76:0/1595082515 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48016040 con 0x560ba80fbcd0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca38ff9640 1 -- 172.21.15.76:0/1595082515 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fca4800e620 con 0x560ba80fbcd0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c09aeb0 con 0x560ba80fbcd0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca38ff9640 1 -- 172.21.15.76:0/1595082515 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fca4800e7e0 con 0x560ba80fbcd0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 10 client.15744 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560ba80195a0 con 0x560ba80fbcd0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca38ff9640 1 -- 172.21.15.76:0/1595082515 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca48038020 con 0x560ba80fbcd0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca38ff9640 1 -- 172.21.15.76:0/1595082515 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fca4802b940 con 0x560ba80fbcd0 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 10 client.15744 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 20 client.15744 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8177020 0x560ba8179440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 4 client.15744 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:44:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ba7da91b0 con 0x560ba8177020 2024-03-23T15:44:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.928+0000 7fca55031640 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8177020 0x560ba8179440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.929+0000 7fca55031640 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8177020 0x560ba8179440 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eff4c2399c2cfdb server_cookie=8f3a3401af722b64 in_seq=0 out_seq=0 2024-03-23T15:44:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:20.929+0000 7fca38ff9640 10 client.15744 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:21.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:21 smithi183 ceph-mon[36296]: pgmap v33: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 278 KiB/s wr, 8 op/s 2024-03-23T15:44:21.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:21 smithi192 ceph-mon[36653]: pgmap v33: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 278 KiB/s wr, 8 op/s 2024-03-23T15:44:21.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:21 smithi076 ceph-mon[32176]: pgmap v33: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 278 KiB/s wr, 8 op/s 2024-03-23T15:44:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.973+0000 7fca38ff9640 1 -- 172.21.15.76:0/1595082515 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ba7da91b0 con 0x560ba8177020 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.973+0000 7fca38ff9640 10 client.15744 handle_command_reply: tid=0 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.973+0000 7fca59e71740 1 client.15744 shutdown 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 msgr2=0x560ba8104270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba8104270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fca48009930 tx=0x7fca48009500 comp rx=0 tx=0).stop 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 shutdown_connections 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560ba80fbcd0 0x560ba8104270 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560ba80f2f00 0x560ba8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x560ba8177020 0x560ba8179440 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 --2- 172.21.15.76:0/1595082515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ba80f25e0 0x560ba81037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca38ff9640 0 client.15744 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 >> 172.21.15.76:0/1595082515 conn(0x560ba80ecf00 msgr2=0x560ba80ebea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 shutdown_connections 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 1 -- 172.21.15.76:0/1595082515 wait complete. 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca59e71740 20 client.15744 trim_cache size 0 max 16384 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:44:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:44:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fca3c0a2480 msgr2=0x7fca3c0a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fca3c0a2480 0x7fca3c0a4940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca48011720 comp rx=0 tx=0).stop 2024-03-23T15:44:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 msgr2=0x7fca50198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50198b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9f0 tx=0x7fca4c00eeb0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 shutdown_connections 2024-03-23T15:44:22.980 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fca3c0a2480 0x7fca3c0a4940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca501055b0 0x7fca501a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca50101220 0x7fca501990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 --2- 172.21.15.76:0/1325651363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca50100860 0x7fca50198b90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.974+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 >> 172.21.15.76:0/1325651363 conn(0x7fca50077460 msgr2=0x7fca500fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.975+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 shutdown_connections 2024-03-23T15:44:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:22.975+0000 7fca57295640 1 -- 172.21.15.76:0/1325651363 wait complete. 2024-03-23T15:44:23.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:23 smithi183 ceph-mon[36296]: pgmap v34: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-03-23T15:44:23.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:23 smithi192 ceph-mon[36653]: pgmap v34: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-03-23T15:44:23.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:23 smithi076 ceph-mon[32176]: pgmap v34: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-03-23T15:44:25.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:25 smithi183 ceph-mon[36296]: pgmap v35: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-23T15:44:25.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:44:25.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:25.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:25 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:25.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:25 smithi192 ceph-mon[36653]: pgmap v35: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-23T15:44:25.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:44:25.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:25.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:25 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:25.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:25 smithi076 ceph-mon[32176]: pgmap v35: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-23T15:44:25.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:44:25.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:25.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:25 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:26.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:44:26.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:44:26.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:26.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:44:26.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:44:26.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:26.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:44:26.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:44:26.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:44:27.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:27 smithi183 ceph-mon[36296]: pgmap v36: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-23T15:44:27.721 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:27 smithi192 ceph-mon[36653]: pgmap v36: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-23T15:44:27.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:27 smithi076 ceph-mon[32176]: pgmap v36: 97 pgs: 97 active+clean; 29 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-03-23T15:44:29.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T15:44:29.407 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T15:44:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 -- 172.21.15.76:0/4016884456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b01016b0 msgr2=0x7f79b0101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 --2- 172.21.15.76:0/4016884456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b01016b0 0x7f79b0101b30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7998009930 tx=0x7f799802bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 -- 172.21.15.76:0/4016884456 shutdown_connections 2024-03-23T15:44:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 --2- 172.21.15.76:0/4016884456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b0102070 0x7f79b010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 --2- 172.21.15.76:0/4016884456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b01016b0 0x7f79b0101b30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 --2- 172.21.15.76:0/4016884456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79b01076c0 0x7f79b0107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 -- 172.21.15.76:0/4016884456 >> 172.21.15.76:0/4016884456 conn(0x7f79b00fd140 msgr2=0x7f79b00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 -- 172.21.15.76:0/4016884456 shutdown_connections 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.515+0000 7f79b54c0640 1 -- 172.21.15.76:0/4016884456 wait complete. 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 Processor -- start 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 -- start start 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b01016b0 0x7f79b0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b0102070 0x7f79b01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79b01076c0 0x7f79b019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f79b019cc50 con 0x7f79b0102070 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79b019cdc0 con 0x7f79b01016b0 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79b54c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f79b019cf80 con 0x7f79b01076c0 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b0102070 0x7f79b01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b0102070 0x7f79b01982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44952/0 (socket says 172.21.15.76:44952) 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 -- 172.21.15.76:0/2806468039 learned_addr learned my addr 172.21.15.76:0/2806468039 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79aeffd640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b01016b0 0x7f79b0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79af7fe640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79b01076c0 0x7f79b019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 -- 172.21.15.76:0/2806468039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79b01076c0 msgr2=0x7f79b019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79b01076c0 0x7f79b019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 -- 172.21.15.76:0/2806468039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b01016b0 msgr2=0x7f79b0197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b01016b0 0x7f79b0197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 -- 172.21.15.76:0/2806468039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7998009560 con 0x7f79b0102070 2024-03-23T15:44:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.516+0000 7f79a7fff640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b0102070 0x7f79b01982d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7998009230 tx=0x7f799802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.517+0000 7f79acff9640 1 -- 172.21.15.76:0/2806468039 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998004350 con 0x7f79b0102070 2024-03-23T15:44:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.517+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b019d200 con 0x7f79b0102070 2024-03-23T15:44:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.517+0000 7f79acff9640 1 -- 172.21.15.76:0/2806468039 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79980044b0 con 0x7f79b0102070 2024-03-23T15:44:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.517+0000 7f79acff9640 1 -- 172.21.15.76:0/2806468039 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998003d00 con 0x7f79b0102070 2024-03-23T15:44:29.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.517+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b01a4ae0 con 0x7f79b0102070 2024-03-23T15:44:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79acff9640 1 -- 172.21.15.76:0/2806468039 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7f7998004df0 con 0x7f79b0102070 2024-03-23T15:44:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79b8179740 1 Processor -- start 2024-03-23T15:44:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79b8179740 1 -- start start 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79acff9640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f799c0a23e0 0x7f799c0a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79af7fe640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f799c0a23e0 0x7f799c0a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79acff9640 1 -- 172.21.15.76:0/2806468039 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f79980f0230 con 0x7f79b0102070 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79b8179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.519+0000 7f79b8179740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb81190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79b8179740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfb83ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79b8179740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5624cfa9cf30 con 0x5624cfb816d0 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79b8179740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624cfae6810 con 0x5624cfb78af0 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79b8179740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5624cfae3d40 con 0x5624cfb781d0 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb81190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfb83ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb81190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:50420/0 (socket says 172.21.15.76:50420) 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfb83ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44954/0 (socket says 172.21.15.76:44954) 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79a7fff640 1 -- 172.21.15.76:0/2481154053 learned_addr learned my addr 172.21.15.76:0/2481154053 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79af7fe640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f799c0a23e0 0x7f799c0a48a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005fd0 tx=0x7f79a8013b60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79aeffd640 1 -- 172.21.15.76:0/2481154053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 msgr2=0x5624cfb81190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79aeffd640 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb81190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79aeffd640 1 -- 172.21.15.76:0/2481154053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 msgr2=0x5624cfb83ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79aeffd640 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfb83ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79aeffd640 1 -- 172.21.15.76:0/2481154053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cfaa0ea0 con 0x5624cfb781d0 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79af7fe640 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfb83ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79a7fff640 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb81190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79aeffd640 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb785b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009c00 tx=0x7f79a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=491e2fa3951bc15c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.520+0000 7f79a57fa640 1 -- 172.21.15.76:0/2481154053 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0004050 con 0x5624cfb781d0 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79a57fa640 1 -- 172.21.15.76:0/2481154053 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a0005500 con 0x5624cfb781d0 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79a57fa640 1 -- 172.21.15.76:0/2481154053 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0004cb0 con 0x5624cfb781d0 2024-03-23T15:44:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 -- 172.21.15.76:0/2481154053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 msgr2=0x5624cfb785b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb785b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009c00 tx=0x7f79a0011810 comp rx=0 tx=0).stop 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 -- 172.21.15.76:0/2481154053 shutdown_connections 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfb83ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb81190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 --2- 172.21.15.76:0/2481154053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb785b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 -- 172.21.15.76:0/2481154053 >> 172.21.15.76:0/2481154053 conn(0x5624cfb72900 msgr2=0x5624cfb72ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 -- 172.21.15.76:0/2481154053 shutdown_connections 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.521+0000 7f79b8179740 1 -- 172.21.15.76:0/2481154053 wait complete. 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 -- start start 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 Processor -- start 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb7e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb78af0 0x5624cfb7ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb816d0 0x5624cfb7f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5624cfadf520 con 0x5624cfb781d0 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624cf7b2430 con 0x5624cfb816d0 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5624cf76ee00 con 0x5624cfb78af0 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb7e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79b8179740 10 client.0 _ll_register_callbacks cb 0x5624cfa7dd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb7e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44958/0 (socket says 172.21.15.76:44958) 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 -- 172.21.15.76:0/3457784250 learned_addr learned my addr 172.21.15.76:0/3457784250 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79a7fff640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb78af0 0x5624cfb7ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79af7fe640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb816d0 0x5624cfb7f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 -- 172.21.15.76:0/3457784250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb78af0 msgr2=0x5624cfb7ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb78af0 0x5624cfb7ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 -- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb816d0 msgr2=0x5624cfb7f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb816d0 0x5624cfb7f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79aeffd640 1 -- 172.21.15.76:0/3457784250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a0009830 con 0x5624cfb781d0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.522+0000 7f79a7fff640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb78af0 0x5624cfb7ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79aeffd640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb7e6d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f79a0011d10 tx=0x7f79a00044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0bad1514f57aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79967fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79967fc640 1 -- 172.21.15.76:0/3457784250 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0003a00 con 0x5624cfb781d0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79967fc640 1 -- 172.21.15.76:0/3457784250 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f79a00046c0 con 0x5624cfb781d0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5624cf7de7e0 con 0x5624cfb781d0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79967fc640 1 -- 172.21.15.76:0/3457784250 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f79a0003b60 con 0x5624cfb781d0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 10 client.15756 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5624cfaddba0 con 0x5624cfb781d0 2024-03-23T15:44:29.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79967fc640 1 -- 172.21.15.76:0/3457784250 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5624cf7de7e0 con 0x5624cfb781d0 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79967fc640 1 -- 172.21.15.76:0/3457784250 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f79a0021040 con 0x5624cfb781d0 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 10 client.15756 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 20 client.15756 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfb91a10 0x5624cfb93e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 4 client.15756 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.523+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624cf81b720 con 0x5624cfb91a10 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.524+0000 7f79a7fff640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfb91a10 0x5624cfb93e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.524+0000 7f79967fc640 10 client.15756 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.524+0000 7f79a7fff640 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfb91a10 0x5624cfb93e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9321d184d60001e9 server_cookie=bf15a1db4e16ed25 in_seq=0 out_seq=0 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79967fc640 1 -- 172.21.15.76:0/3457784250 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5624cf81b720 con 0x5624cfb91a10 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79967fc640 10 client.15756 handle_command_reply: tid=0 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 client.15756 shutdown 2024-03-23T15:44:29.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 msgr2=0x5624cfb7e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb7e6d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f79a0011d10 tx=0x7f79a00044b0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 shutdown_connections 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfb91a10 0x5624cfb93e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb816d0 0x5624cfb7f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb78af0 0x5624cfb7ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 --2- 172.21.15.76:0/3457784250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb7e6d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79967fc640 0 client.15756 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 >> 172.21.15.76:0/3457784250 conn(0x5624cfb72900 msgr2=0x5624cfb71b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 shutdown_connections 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 1 -- 172.21.15.76:0/3457784250 wait complete. 2024-03-23T15:44:29.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.525+0000 7f79b8179740 20 client.15756 trim_cache size 0 max 16384 2024-03-23T15:44:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 Processor -- start 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 -- start start 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb80d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb816d0 0x5624cfb719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5624cfa64430 con 0x5624cfb781d0 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624cfb73900 con 0x5624cfb78af0 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79b8179740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5624cfb77e30 con 0x5624cfb816d0 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb80d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb80d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44968/0 (socket says 172.21.15.76:44968) 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79aeffd640 1 -- 172.21.15.76:0/3701589246 learned_addr learned my addr 172.21.15.76:0/3701589246 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79a7fff640 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.543+0000 7f79af7fe640 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb816d0 0x5624cfb719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79aeffd640 1 -- 172.21.15.76:0/3701589246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb816d0 msgr2=0x5624cfb719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79aeffd640 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb816d0 0x5624cfb719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79aeffd640 1 -- 172.21.15.76:0/3701589246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 msgr2=0x5624cfb8cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79aeffd640 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb8cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79aeffd640 1 -- 172.21.15.76:0/3701589246 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cfa79b20 con 0x5624cfb781d0 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79aeffd640 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb80d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009c00 tx=0x7f79a0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c1c52449e811b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79a4ff9640 1 -- 172.21.15.76:0/3701589246 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0023b70 con 0x5624cfb781d0 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79a4ff9640 1 -- 172.21.15.76:0/3701589246 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a0005670 con 0x5624cfb781d0 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.544+0000 7f79a4ff9640 1 -- 172.21.15.76:0/3701589246 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a002e440 con 0x5624cfb781d0 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- 172.21.15.76:0/3701589246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 msgr2=0x5624cfb80d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb80d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009c00 tx=0x7f79a0005b90 comp rx=0 tx=0).stop 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- 172.21.15.76:0/3701589246 shutdown_connections 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb816d0 0x5624cfb719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb8cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 --2- 172.21.15.76:0/3701589246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb781d0 0x5624cfb80d90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- 172.21.15.76:0/3701589246 >> 172.21.15.76:0/3701589246 conn(0x5624cfb72900 msgr2=0x5624cfb712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- 172.21.15.76:0/3701589246 shutdown_connections 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- 172.21.15.76:0/3701589246 wait complete. 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- start start 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 Processor -- start 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb85730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfbff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5624cfb85ed0 con 0x5624cfb816d0 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624cfb86040 con 0x5624cfb78af0 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5624cfbffac0 con 0x5624cfb781d0 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79b8179740 10 client.0 _ll_register_callbacks cb 0x7f79a8009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfbff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.545+0000 7f79af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfbff580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44974/0 (socket says 172.21.15.76:44974) 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79af7fe640 1 -- 172.21.15.76:0/1745168759 learned_addr learned my addr 172.21.15.76:0/1745168759 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79a7fff640 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb85730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79aeffd640 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79af7fe640 1 -- 172.21.15.76:0/1745168759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 msgr2=0x5624cfb851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79af7fe640 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb851f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79af7fe640 1 -- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 msgr2=0x5624cfb85730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79af7fe640 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb85730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79af7fe640 1 -- 172.21.15.76:0/1745168759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cfaddba0 con 0x5624cfb816d0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79af7fe640 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfbff580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009e40 tx=0x7f79a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45db247e550654ac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79967fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79967fc640 1 -- 172.21.15.76:0/1745168759 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8025070 con 0x5624cfb816d0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79967fc640 1 -- 172.21.15.76:0/1745168759 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f79a8003d40 con 0x5624cfb816d0 2024-03-23T15:44:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a0005500 con 0x5624cfb816d0 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79967fc640 1 -- 172.21.15.76:0/1745168759 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f79a8004b20 con 0x5624cfb816d0 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79b8179740 10 client.15762 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5624cfa9f700 con 0x5624cfb816d0 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.546+0000 7f79967fc640 1 -- 172.21.15.76:0/1745168759 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a8038020 con 0x5624cfb816d0 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79967fc640 1 -- 172.21.15.76:0/1745168759 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f79a8034830 con 0x5624cfb816d0 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79b8179740 10 client.15762 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79b8179740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79b8179740 20 client.15762 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79b8179740 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfc01450 0x5624cfc03870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79b8179740 4 client.15762 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624cf86a890 con 0x5624cfc01450 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79aeffd640 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfc01450 0x5624cfc03870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79967fc640 10 client.15762 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.547+0000 7f79aeffd640 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfc01450 0x5624cfc03870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb51a2e0ec06f8d server_cookie=d93f4ccbb09c064d in_seq=0 out_seq=0 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79967fc640 1 -- 172.21.15.76:0/1745168759 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5624cf86a890 con 0x5624cfc01450 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79967fc640 10 client.15762 handle_command_reply: tid=0 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 client.15762 shutdown 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 msgr2=0x5624cfbff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfbff580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009e40 tx=0x7f79a8004ab0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 shutdown_connections 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5624cfb816d0 0x5624cfbff580 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5624cfc01450 0x5624cfc03870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624cfb78af0 0x5624cfb85730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 --2- 172.21.15.76:0/1745168759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5624cfb781d0 0x5624cfb851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79967fc640 0 client.15762 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 >> 172.21.15.76:0/1745168759 conn(0x5624cfb72900 msgr2=0x5624cfb71b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 shutdown_connections 2024-03-23T15:44:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 1 -- 172.21.15.76:0/1745168759 wait complete. 2024-03-23T15:44:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b8179740 20 client.15762 trim_cache size 0 max 16384 2024-03-23T15:44:29.559 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:44:29.559 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-03-23T15:44:29.559 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-03-23T15:44:29.559 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:44:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f799c0a23e0 msgr2=0x7f799c0a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f799c0a23e0 0x7f799c0a48a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005fd0 tx=0x7f79a8013b60 comp rx=0 tx=0).stop 2024-03-23T15:44:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b0102070 msgr2=0x7f79b01982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b0102070 0x7f79b01982d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7998009230 tx=0x7f799802c650 comp rx=0 tx=0).stop 2024-03-23T15:44:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 shutdown_connections 2024-03-23T15:44:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f799c0a23e0 0x7f799c0a48a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79b01076c0 0x7f79b019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79b0102070 0x7f79b01982d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 --2- 172.21.15.76:0/2806468039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79b01016b0 0x7f79b0197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.548+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 >> 172.21.15.76:0/2806468039 conn(0x7f79b00fd140 msgr2=0x7f79b00fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.549+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 shutdown_connections 2024-03-23T15:44:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.549+0000 7f79b54c0640 1 -- 172.21.15.76:0/2806468039 wait complete. 2024-03-23T15:44:29.562 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T15:44:29.563 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:44:29.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:29 smithi183 ceph-mon[36296]: pgmap v37: 97 pgs: 97 active+clean; 27 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 828 op/s 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.666+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2115264724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a201076c0 msgr2=0x7f3a20107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.666+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2115264724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a201076c0 0x7f3a20107aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009860 tx=0x7f3a1002bb70 comp rx=0 tx=0).stop 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2115264724 shutdown_connections 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2115264724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2006a6a0 0x7f3a2010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2115264724 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20069ce0 0x7f3a2006a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2115264724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a201076c0 0x7f3a20107aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2115264724 >> 172.21.15.76:0/2115264724 conn(0x7f3a200775f0 msgr2=0x7f3a20077a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2115264724 shutdown_connections 2024-03-23T15:44:29.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2115264724 wait complete. 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 Processor -- start 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.667+0000 7f3a24c6b640 1 -- start start 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a24c6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a20069ce0 0x7f3a20197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a24c6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2006a6a0 0x7f3a201982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a24c6b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a201076c0 0x7f3a2019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a24c6b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2019cc20 con 0x7f3a2006a6a0 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a24c6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2019cd90 con 0x7f3a20069ce0 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a24c6b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2019cf50 con 0x7f3a201076c0 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2006a6a0 0x7f3a201982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a20069ce0 0x7f3a20197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2006a6a0 0x7f3a201982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44996/0 (socket says 172.21.15.76:44996) 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a20069ce0 0x7f3a20197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:50454/0 (socket says 172.21.15.76:50454) 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1dd9b640 1 -- 172.21.15.76:0/2304727196 learned_addr learned my addr 172.21.15.76:0/2304727196 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a201076c0 0x7f3a2019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 -- 172.21.15.76:0/2304727196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a201076c0 msgr2=0x7f3a2019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a201076c0 0x7f3a2019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 -- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2006a6a0 msgr2=0x7f3a201982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2006a6a0 0x7f3a201982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 -- 172.21.15.76:0/2304727196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a14009560 con 0x7f3a20069ce0 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1ed9d640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a201076c0 0x7f3a2019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.668+0000 7f3a1e59c640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a20069ce0 0x7f3a20197db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a10037040 tx=0x7f3a10004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.669+0000 7f3a0f7fe640 1 -- 172.21.15.76:0/2304727196 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10034950 con 0x7f3a20069ce0 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.669+0000 7f3a0f7fe640 1 -- 172.21.15.76:0/2304727196 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1002c9f0 con 0x7f3a20069ce0 2024-03-23T15:44:29.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.669+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a10009490 con 0x7f3a20069ce0 2024-03-23T15:44:29.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.669+0000 7f3a0f7fe640 1 -- 172.21.15.76:0/2304727196 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10003e60 con 0x7f3a20069ce0 2024-03-23T15:44:29.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.669+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2019d3b0 con 0x7f3a20069ce0 2024-03-23T15:44:29.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.670+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a20102df0 con 0x7f3a20069ce0 2024-03-23T15:44:29.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.671+0000 7f3a0f7fe640 1 -- 172.21.15.76:0/2304727196 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7f3a1003b040 con 0x7f3a20069ce0 2024-03-23T15:44:29.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.671+0000 7f3a0f7fe640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3a080a23e0 0x7f3a080a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:29.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.672+0000 7f3a1dd9b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3a080a23e0 0x7f3a080a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:29.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.672+0000 7f3a0f7fe640 1 -- 172.21.15.76:0/2304727196 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3a100f0130 con 0x7f3a20069ce0 2024-03-23T15:44:29.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.672+0000 7f3a1dd9b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3a080a23e0 0x7f3a080a48a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a20100490 tx=0x7f3a14009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:29.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.676+0000 7f3a0f7fe640 1 -- 172.21.15.76:0/2304727196 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f3a100b8d40 con 0x7f3a20069ce0 2024-03-23T15:44:29.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:29 smithi192 ceph-mon[36653]: pgmap v37: 97 pgs: 97 active+clean; 27 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 828 op/s 2024-03-23T15:44:29.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:29 smithi076 ceph-mon[32176]: pgmap v37: 97 pgs: 97 active+clean; 27 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 828 op/s 2024-03-23T15:44:29.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.898+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a20068950 con 0x7f3a20069ce0 2024-03-23T15:44:29.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.899+0000 7f3a0f7fe640 1 -- 172.21.15.76:0/2304727196 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11875 (secure 0 0 0) 0x7f3a1004b090 con 0x7f3a20069ce0 2024-03-23T15:44:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:44:29.903 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:44:17.475196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:44:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 23 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3a080a23e0 msgr2=0x7f3a080a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3a080a23e0 0x7f3a080a48a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a20100490 tx=0x7f3a14009260 comp rx=0 tx=0).stop 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a20069ce0 msgr2=0x7f3a20197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a20069ce0 0x7f3a20197db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a10037040 tx=0x7f3a10004540 comp rx=0 tx=0).stop 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 shutdown_connections 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3a080a23e0 0x7f3a080a48a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a201076c0 0x7f3a2019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a2006a6a0 0x7f3a201982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 --2- 172.21.15.76:0/2304727196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a20069ce0 0x7f3a20197db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.901+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 >> 172.21.15.76:0/2304727196 conn(0x7f3a200775f0 msgr2=0x7f3a200fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.902+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 shutdown_connections 2024-03-23T15:44:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:29.902+0000 7f3a24c6b640 1 -- 172.21.15.76:0/2304727196 wait complete. 2024-03-23T15:44:29.916 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T15:44:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 -- 172.21.15.76:0/2485780444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 msgr2=0x7f1bb810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 --2- 172.21.15.76:0/2485780444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb810e580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009cd0 tx=0x7f1bb002c0f0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 -- 172.21.15.76:0/2485780444 shutdown_connections 2024-03-23T15:44:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 --2- 172.21.15.76:0/2485780444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb810e580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 --2- 172.21.15.76:0/2485780444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bb81016b0 0x7f1bb8101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 --2- 172.21.15.76:0/2485780444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb81076c0 0x7f1bb8107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 -- 172.21.15.76:0/2485780444 >> 172.21.15.76:0/2485780444 conn(0x7f1bb80fd140 msgr2=0x7f1bb80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 -- 172.21.15.76:0/2485780444 shutdown_connections 2024-03-23T15:44:30.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.024+0000 7f1bbe442640 1 -- 172.21.15.76:0/2485780444 wait complete. 2024-03-23T15:44:30.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 Processor -- start 2024-03-23T15:44:30.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 -- start start 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bb81016b0 0x7f1bb8073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb8073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb81076c0 0x7f1bb8074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb81a61c0 con 0x7f1bb8102070 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb81a6330 con 0x7f1bb81076c0 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.025+0000 7f1bbe442640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb81a64f0 con 0x7f1bb81016b0 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb8073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb8073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45022/0 (socket says 172.21.15.76:45022) 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 -- 172.21.15.76:0/3733785869 learned_addr learned my addr 172.21.15.76:0/3733785869 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bb81016b0 0x7f1bb8073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb81076c0 0x7f1bb8074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 -- 172.21.15.76:0/3733785869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bb81016b0 msgr2=0x7f1bb8073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bb81016b0 0x7f1bb8073a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 -- 172.21.15.76:0/3733785869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb81076c0 msgr2=0x7f1bb8074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb81076c0 0x7f1bb8074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 -- 172.21.15.76:0/3733785869 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb0009900 con 0x7f1bb8102070 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bb81016b0 0x7f1bb8073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb8073f50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00e9f0 tx=0x7f1bac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb57fa640 1 -- 172.21.15.76:0/3733785869 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac009720 con 0x7f1bb8102070 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb57fa640 1 -- 172.21.15.76:0/3733785869 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac00cd80 con 0x7f1bb8102070 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb81a67d0 con 0x7f1bb8102070 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bb57fa640 1 -- 172.21.15.76:0/3733785869 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac010690 con 0x7f1bb8102070 2024-03-23T15:44:30.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.026+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb81a6c90 con 0x7f1bb8102070 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.028+0000 7f1bb57fa640 1 -- 172.21.15.76:0/3733785869 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7f1bac0046c0 con 0x7f1bb8102070 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bb57fa640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1ba00a23e0 0x7f1ba00a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1ba00a23e0 0x7f1ba00a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 Processor -- start 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bb57fa640 1 -- 172.21.15.76:0/3733785869 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1bac014070 con 0x7f1bb8102070 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 -- start start 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d4a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4ad10 0x561058d533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d538f0 0x561058d55ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561058cb8840 con 0x561058d4a3f0 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561058cb5d70 con 0x561058d538f0 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bc101a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561058cb1550 con 0x561058d4ad10 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d4a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d4a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45034/0 (socket says 172.21.15.76:45034) 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.029+0000 7f1bb7fff640 1 -- 172.21.15.76:0/1910436495 learned_addr learned my addr 172.21.15.76:0/1910436495 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d538f0 0x561058d55ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4ad10 0x561058d533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1ba00a23e0 0x7f1ba00a48a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009f40 tx=0x7f1ba8005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb7fff640 1 -- 172.21.15.76:0/1910436495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4ad10 msgr2=0x561058d533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4ad10 0x561058d533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb7fff640 1 -- 172.21.15.76:0/1910436495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d538f0 msgr2=0x561058d55ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d538f0 0x561058d55ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb7fff640 1 -- 172.21.15.76:0/1910436495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561058c46de0 con 0x561058d4a3f0 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d4a7d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8004250 tx=0x7f1ba8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d9d17369280ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1b9e7fc640 1 -- 172.21.15.76:0/1910436495 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800f480 con 0x561058d4a3f0 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1b9e7fc640 1 -- 172.21.15.76:0/1910436495 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba800fac0 con 0x561058d4a3f0 2024-03-23T15:44:30.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.030+0000 7f1b9e7fc640 1 -- 172.21.15.76:0/1910436495 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8026b10 con 0x561058d4a3f0 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 -- 172.21.15.76:0/1910436495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 msgr2=0x561058d4a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d4a7d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8004250 tx=0x7f1ba8014000 comp rx=0 tx=0).stop 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 -- 172.21.15.76:0/1910436495 shutdown_connections 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d538f0 0x561058d55ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4ad10 0x561058d533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1910436495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d4a7d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 -- 172.21.15.76:0/1910436495 >> 172.21.15.76:0/1910436495 conn(0x561058d44b20 msgr2=0x561058d44f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 -- 172.21.15.76:0/1910436495 shutdown_connections 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 -- 172.21.15.76:0/1910436495 wait complete. 2024-03-23T15:44:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 -- start start 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.031+0000 7f1bc101a740 1 Processor -- start 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d50ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d51430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561058984430 con 0x561058d538f0 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561058940e00 con 0x561058d4ad10 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561058c36590 con 0x561058d4a3f0 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 10 client.0 _ll_register_callbacks cb 0x561058ca5fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d51430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d51430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45036/0 (socket says 172.21.15.76:45036) 2024-03-23T15:44:30.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 -- 172.21.15.76:0/103911871 learned_addr learned my addr 172.21.15.76:0/103911871 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d50ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 -- 172.21.15.76:0/103911871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 msgr2=0x561058d509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 -- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 msgr2=0x561058d50ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d50ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 -- 172.21.15.76:0/103911871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba8017070 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d50ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d51430 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1bb002c700 tx=0x7f1bb0034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf992a66232f852 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1b977fe640 1 -- 172.21.15.76:0/103911871 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00328b0 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1b977fe640 1 -- 172.21.15.76:0/103911871 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1bb0046070 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.032+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610589b07e0 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1b977fe640 1 -- 172.21.15.76:0/103911871 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1bb0032a10 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 10 client.15780 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561058cafbd0 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1b977fe640 1 -- 172.21.15.76:0/103911871 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5610589b07e0 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1b977fe640 1 -- 172.21.15.76:0/103911871 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1bb0004600 con 0x561058d538f0 2024-03-23T15:44:30.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 10 client.15780 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 10 client.15780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 20 client.15780 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058d639e0 0x561058d65dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 4 client.15780 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610589ed720 con 0x561058d639e0 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058d639e0 0x561058d65dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058d639e0 0x561058d65dd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b8a338d6e56165e server_cookie=158e151b6fdf8436 in_seq=0 out_seq=0 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.033+0000 7f1b977fe640 10 client.15780 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1b977fe640 1 -- 172.21.15.76:0/103911871 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5610589ed720 con 0x561058d639e0 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1b977fe640 10 client.15780 handle_command_reply: tid=0 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 client.15780 shutdown 2024-03-23T15:44:30.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 msgr2=0x561058d51430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d51430 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1bb002c700 tx=0x7f1bb0034670 comp rx=0 tx=0).stop 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 shutdown_connections 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058d639e0 0x561058d65dd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d51430 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d50ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 --2- 172.21.15.76:0/103911871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1b977fe640 0 client.15780 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 >> 172.21.15.76:0/103911871 conn(0x561058d44b20 msgr2=0x561058d46870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 shutdown_connections 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.034+0000 7f1bc101a740 1 -- 172.21.15.76:0/103911871 wait complete. 2024-03-23T15:44:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.035+0000 7f1bc101a740 20 client.15780 trim_cache size 0 max 16384 2024-03-23T15:44:30.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 Processor -- start 2024-03-23T15:44:30.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 -- start start 2024-03-23T15:44:30.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d5f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d6b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d50f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5610589b07e0 con 0x561058d538f0 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561058d403e0 con 0x561058d4ad10 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bc101a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561058d5f6c0 con 0x561058d4a3f0 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bbc9df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d50f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bbc9df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d50f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45042/0 (socket says 172.21.15.76:45042) 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bbc9df640 1 -- 172.21.15.76:0/1118893999 learned_addr learned my addr 172.21.15.76:0/1118893999 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d5f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d6b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.048+0000 7f1bbc9df640 1 -- 172.21.15.76:0/1118893999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 msgr2=0x561058d5f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d5f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bbc9df640 1 -- 172.21.15.76:0/1118893999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 msgr2=0x561058d6b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d6b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bbc9df640 1 -- 172.21.15.76:0/1118893999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561058c4bc80 con 0x561058d538f0 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d50f00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0037040 tx=0x7f1bb00537b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385717b15a4d6ae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1b9dffb640 1 -- 172.21.15.76:0/1118893999 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00346e0 con 0x561058d538f0 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1b9dffb640 1 -- 172.21.15.76:0/1118893999 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb0044070 con 0x561058d538f0 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1b9dffb640 1 -- 172.21.15.76:0/1118893999 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb004a680 con 0x561058d538f0 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 -- 172.21.15.76:0/1118893999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 msgr2=0x561058d50f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d50f00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0037040 tx=0x7f1bb00537b0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 -- 172.21.15.76:0/1118893999 shutdown_connections 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d538f0 0x561058d50f00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d6b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 --2- 172.21.15.76:0/1118893999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d4a3f0 0x561058d5f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 -- 172.21.15.76:0/1118893999 >> 172.21.15.76:0/1118893999 conn(0x561058d44b20 msgr2=0x561058d43cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 -- 172.21.15.76:0/1118893999 shutdown_connections 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.049+0000 7f1bc101a740 1 -- 172.21.15.76:0/1118893999 wait complete. 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 -- start start 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 Processor -- start 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d5ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d4d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d538f0 0x561058d4d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561058d4de70 con 0x561058d4a3f0 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561058d4dfe0 con 0x561058d4ad10 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561058dc8cd0 con 0x561058d538f0 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bc101a740 10 client.0 _ll_register_callbacks cb 0x7f1bac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:30.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d5ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d5ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45052/0 (socket says 172.21.15.76:45052) 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb7fff640 1 -- 172.21.15.76:0/2277258156 learned_addr learned my addr 172.21.15.76:0/2277258156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d4d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bbc9df640 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d538f0 0x561058d4d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb7fff640 1 -- 172.21.15.76:0/2277258156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d538f0 msgr2=0x561058d4d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d538f0 0x561058d4d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb7fff640 1 -- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 msgr2=0x561058d4d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.050+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d4d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bb7fff640 1 -- 172.21.15.76:0/2277258156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561058cafbd0 con 0x561058d4a3f0 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bb7fff640 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d5ba80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80041f0 tx=0x7f1ba8005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3e7f50113719b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1b977fe640 1 -- 172.21.15.76:0/2277258156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba802b050 con 0x561058d4a3f0 2024-03-23T15:44:30.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1b977fe640 1 -- 172.21.15.76:0/2277258156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1ba800fe30 con 0x561058d4a3f0 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba800fac0 con 0x561058d4a3f0 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1b977fe640 1 -- 172.21.15.76:0/2277258156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1ba8003f00 con 0x561058d4a3f0 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 10 client.15786 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561058c71920 con 0x561058d4a3f0 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1b977fe640 1 -- 172.21.15.76:0/2277258156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba8040020 con 0x561058d4a3f0 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1b977fe640 1 -- 172.21.15.76:0/2277258156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1ba8024020 con 0x561058d4a3f0 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 10 client.15786 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 10 client.15786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 20 client.15786 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058dca310 0x561058dcc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 4 client.15786 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.051+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561058a3c890 con 0x561058dca310 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058dca310 0x561058dcc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1b977fe640 10 client.15786 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bb77fe640 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058dca310 0x561058dcc700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7af248ee055ca527 server_cookie=a2ec32c976467f0e in_seq=0 out_seq=0 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1b977fe640 1 -- 172.21.15.76:0/2277258156 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561058a3c890 con 0x561058dca310 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1b977fe640 10 client.15786 handle_command_reply: tid=0 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 client.15786 shutdown 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 msgr2=0x561058d5ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d5ba80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80041f0 tx=0x7f1ba8005d70 comp rx=0 tx=0).stop 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 shutdown_connections 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561058d538f0 0x561058d4d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561058d4ad10 0x561058d4d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561058dca310 0x561058dcc700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 --2- 172.21.15.76:0/2277258156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561058d4a3f0 0x561058d5ba80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 >> 172.21.15.76:0/2277258156 conn(0x561058d44b20 msgr2=0x561058d45b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1b977fe640 0 client.15786 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:44:30.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 shutdown_connections 2024-03-23T15:44:30.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 1 -- 172.21.15.76:0/2277258156 wait complete. 2024-03-23T15:44:30.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.052+0000 7f1bc101a740 20 client.15786 trim_cache size 0 max 16384 2024-03-23T15:44:30.067 INFO:teuthology.orchestra.run.smithi076.stdout:[] 2024-03-23T15:44:30.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1ba00a23e0 msgr2=0x7f1ba00a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1ba00a23e0 0x7f1ba00a48a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009f40 tx=0x7f1ba8005f90 comp rx=0 tx=0).stop 2024-03-23T15:44:30.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 msgr2=0x7f1bb8073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb8073f50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00e9f0 tx=0x7f1bac00edf0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 shutdown_connections 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f1ba00a23e0 0x7f1ba00a48a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb81076c0 0x7f1bb8074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1bb8102070 0x7f1bb8073f50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 --2- 172.21.15.76:0/3733785869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bb81016b0 0x7f1bb8073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 >> 172.21.15.76:0/3733785869 conn(0x7f1bb80fd140 msgr2=0x7f1bb80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 shutdown_connections 2024-03-23T15:44:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.053+0000 7f1bbe442640 1 -- 172.21.15.76:0/3733785869 wait complete. 2024-03-23T15:44:30.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2954699170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8106660 msgr2=0x7fb5b8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2954699170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8106660 0x7fb5b8106a40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009900 tx=0x7fb5a802bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2954699170 shutdown_connections 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2954699170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5b8100f60 0x7fb5b810d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2954699170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5b81005a0 0x7fb5b8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2954699170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8106660 0x7fb5b8106a40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2954699170 >> 172.21.15.76:0/2954699170 conn(0x7fb5b8077710 msgr2=0x7fb5b8077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2954699170 shutdown_connections 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.174+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2954699170 wait complete. 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 Processor -- start 2024-03-23T15:44:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 -- start start 2024-03-23T15:44:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5b81005a0 0x7fb5b8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8100f60 0x7fb5b8199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5b8106660 0x7fb5b81a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81998a0 con 0x7fb5b8100f60 2024-03-23T15:44:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81a40a0 con 0x7fb5b8106660 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5be9c7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b81a41e0 con 0x7fb5b81005a0 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8100f60 0x7fb5b8199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8100f60 0x7fb5b8199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45062/0 (socket says 172.21.15.76:45062) 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5b77fe640 1 -- 172.21.15.76:0/2202434991 learned_addr learned my addr 172.21.15.76:0/2202434991 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5b81005a0 0x7fb5b8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.175+0000 7fb5bcf64640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5b8106660 0x7fb5b81a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b77fe640 1 -- 172.21.15.76:0/2202434991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5b81005a0 msgr2=0x7fb5b8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5b81005a0 0x7fb5b8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b77fe640 1 -- 172.21.15.76:0/2202434991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5b8106660 msgr2=0x7fb5b81a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5b8106660 0x7fb5b81a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b77fe640 1 -- 172.21.15.76:0/2202434991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a8009560 con 0x7fb5b8100f60 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8100f60 0x7fb5b8199020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00ea20 tx=0x7fb5ac00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b57fa640 1 -- 172.21.15.76:0/2202434991 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac009700 con 0x7fb5b8100f60 2024-03-23T15:44:30.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b57fa640 1 -- 172.21.15.76:0/2202434991 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ac00cd20 con 0x7fb5b8100f60 2024-03-23T15:44:30.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b81a4440 con 0x7fb5b8100f60 2024-03-23T15:44:30.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5b57fa640 1 -- 172.21.15.76:0/2202434991 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac018840 con 0x7fb5b8100f60 2024-03-23T15:44:30.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.176+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b81a49a0 con 0x7fb5b8100f60 2024-03-23T15:44:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.178+0000 7fb5b57fa640 1 -- 172.21.15.76:0/2202434991 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7fb5ac010a80 con 0x7fb5b8100f60 2024-03-23T15:44:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.178+0000 7fb5c15a1740 1 Processor -- start 2024-03-23T15:44:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.178+0000 7fb5c15a1740 1 -- start start 2024-03-23T15:44:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047d47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047d4d10 0x557b047dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047dd8f0 0x557b047dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557b04742840 con 0x557b047d43f0 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b57fa640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fb5a00a2460 0x7fb5a00a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b0473fd70 con 0x557b047d4d10 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557b0473b550 con 0x557b047dd8f0 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5bcf64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047d47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5bcf64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047d47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45066/0 (socket says 172.21.15.76:45066) 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047d4d10 0x557b047dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5bcf64640 1 -- 172.21.15.76:0/328068238 learned_addr learned my addr 172.21.15.76:0/328068238 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047dd8f0 0x557b047dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5bcf64640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fb5a00a2460 0x7fb5a00a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b57fa640 1 -- 172.21.15.76:0/2202434991 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb5ac014070 con 0x7fb5b8100f60 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b7fff640 1 -- 172.21.15.76:0/328068238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047dd8f0 msgr2=0x557b047dfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047dd8f0 0x557b047dfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b7fff640 1 -- 172.21.15.76:0/328068238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 msgr2=0x557b047d47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047d47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b7fff640 1 -- 172.21.15.76:0/328068238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b046d0de0 con 0x557b047d4d10 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5bcf64640 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047d4d10 0x557b047dd3b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a802c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=124e43aed7427204 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb5bcf64640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fb5a00a2460 0x7fb5a00a4920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00099e0 tx=0x7fb5b0009500 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.179+0000 7fb59e7fc640 1 -- 172.21.15.76:0/328068238 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a802c910 con 0x557b047d4d10 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb59e7fc640 1 -- 172.21.15.76:0/328068238 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a802ca70 con 0x557b047d4d10 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb59e7fc640 1 -- 172.21.15.76:0/328068238 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8005050 con 0x557b047d4d10 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 -- 172.21.15.76:0/328068238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047d4d10 msgr2=0x557b047dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047d4d10 0x557b047dd3b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a802c6a0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 -- 172.21.15.76:0/328068238 shutdown_connections 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047dd8f0 0x557b047dfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047d4d10 0x557b047dd3b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/328068238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 -- 172.21.15.76:0/328068238 >> 172.21.15.76:0/328068238 conn(0x557b047ceb20 msgr2=0x557b047cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 -- 172.21.15.76:0/328068238 shutdown_connections 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.180+0000 7fb5c15a1740 1 -- 172.21.15.76:0/328068238 wait complete. 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 -- start start 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 Processor -- start 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047dae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557b0440e430 con 0x557b047d43f0 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b043cae00 con 0x557b047dd8f0 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557b046c0590 con 0x557b047d4d10 2024-03-23T15:44:30.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5c15a1740 10 client.0 _ll_register_callbacks cb 0x557b0472ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047da950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45072/0 (socket says 172.21.15.76:45072) 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1831970323 learned_addr learned my addr 172.21.15.76:0/1831970323 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047dae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.181+0000 7fb5bcf64640 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 msgr2=0x557b047dae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047dae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1831970323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 msgr2=0x557b047db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047db3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1831970323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b810dd20 con 0x557b047d43f0 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047dae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047da950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5a802c170 tx=0x7fb5a802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee76dd99c1f2007 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5977fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:30.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5977fe640 1 -- 172.21.15.76:0/1831970323 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8005000 con 0x557b047d43f0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5977fe640 1 -- 172.21.15.76:0/1831970323 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb5a8005160 con 0x557b047d43f0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b0443a7e0 con 0x557b047d43f0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5977fe640 1 -- 172.21.15.76:0/1831970323 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb5a8005320 con 0x557b047d43f0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5c15a1740 10 client.15798 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557b04739bd0 con 0x557b047d43f0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5977fe640 1 -- 172.21.15.76:0/1831970323 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557b0443a7e0 con 0x557b047d43f0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.182+0000 7fb5977fe640 1 -- 172.21.15.76:0/1831970323 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb5a803c050 con 0x557b047d43f0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5c15a1740 10 client.15798 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5c15a1740 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5c15a1740 20 client.15798 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b047ed630 0x557b047efa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5c15a1740 4 client.15798 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b04477720 con 0x557b047ed630 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b047ed630 0x557b047efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5977fe640 10 client.15798 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.183+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b047ed630 0x557b047efa40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cbbecb0d8725379 server_cookie=9ccb727f6e68c3bb in_seq=0 out_seq=0 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5977fe640 1 -- 172.21.15.76:0/1831970323 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557b04477720 con 0x557b047ed630 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5977fe640 10 client.15798 handle_command_reply: tid=0 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 client.15798 shutdown 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 msgr2=0x557b047da950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047da950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5a802c170 tx=0x7fb5a802bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 shutdown_connections 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b047ed630 0x557b047efa40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047dae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5977fe640 0 client.15798 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1831970323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047da950 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 >> 172.21.15.76:0/1831970323 conn(0x557b047ceb20 msgr2=0x557b047d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 shutdown_connections 2024-03-23T15:44:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1831970323 wait complete. 2024-03-23T15:44:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.184+0000 7fb5c15a1740 20 client.15798 trim_cache size 0 max 16384 2024-03-23T15:44:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 Processor -- start 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 -- start start 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557b047cfb20 con 0x557b047d43f0 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b047d3b90 con 0x557b047dd8f0 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557b047ca1c0 con 0x557b047d4d10 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45084/0 (socket says 172.21.15.76:45084) 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1301591709 learned_addr learned my addr 172.21.15.76:0/1301591709 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5bcf64640 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1301591709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 msgr2=0x557b047e9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1301591709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 msgr2=0x557b047f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 -- 172.21.15.76:0/1301591709 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b046d5c80 con 0x557b047d43f0 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.198+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047dd080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8d7e9bdeb2d832 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb59dffb640 1 -- 172.21.15.76:0/1301591709 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a802cbe0 con 0x557b047d43f0 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb59dffb640 1 -- 172.21.15.76:0/1301591709 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a803fdf0 con 0x557b047d43f0 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb59dffb640 1 -- 172.21.15.76:0/1301591709 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8005370 con 0x557b047d43f0 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1301591709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 msgr2=0x557b047dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047dd080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a8005bf0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1301591709 shutdown_connections 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/1301591709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047dd080 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1301591709 >> 172.21.15.76:0/1301591709 conn(0x557b047ceb20 msgr2=0x557b047d9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1301591709 shutdown_connections 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.199+0000 7fb5c15a1740 1 -- 172.21.15.76:0/1301591709 wait complete. 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 -- start start 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 Processor -- start 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557b047e1790 con 0x557b047d43f0 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b047e1900 con 0x557b047dd8f0 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557b047e1ac0 con 0x557b047d4d10 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5c15a1740 10 client.0 _ll_register_callbacks cb 0x7fb5b0011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:30.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45096/0 (socket says 172.21.15.76:45096) 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b7fff640 1 -- 172.21.15.76:0/239867475 learned_addr learned my addr 172.21.15.76:0/239867475 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5bcf64640 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b7fff640 1 -- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 msgr2=0x557b047e0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b7fff640 1 -- 172.21.15.76:0/239867475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 msgr2=0x557b047e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.200+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047e1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5b7fff640 1 -- 172.21.15.76:0/239867475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b04739bd0 con 0x557b047d43f0 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5b7fff640 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047e5760 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009800 tx=0x7fb5a8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502b07ceb1704605 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5977fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5977fe640 1 -- 172.21.15.76:0/239867475 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a803f780 con 0x557b047d43f0 2024-03-23T15:44:30.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a802ca70 con 0x557b047d43f0 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5977fe640 1 -- 172.21.15.76:0/239867475 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb5a8000e40 con 0x557b047d43f0 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5977fe640 1 -- 172.21.15.76:0/239867475 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb5a803f8e0 con 0x557b047d43f0 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 10 client.15804 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557b046fb920 con 0x557b047d43f0 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5977fe640 1 -- 172.21.15.76:0/239867475 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a802ca70 con 0x557b047d43f0 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5977fe640 1 -- 172.21.15.76:0/239867475 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb5a803c050 con 0x557b047d43f0 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 10 client.15804 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 10 client.15804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 20 client.15804 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b04853a60 0x557b04855e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 4 client.15804 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.201+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b044c6890 con 0x557b04853a60 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b04853a60 0x557b04855e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5977fe640 10 client.15804 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:44:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5b77fe640 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b04853a60 0x557b04855e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59acdbf760889ea0 server_cookie=cb82a980b08521e2 in_seq=0 out_seq=0 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5977fe640 1 -- 172.21.15.76:0/239867475 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b044c6890 con 0x557b04853a60 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5977fe640 10 client.15804 handle_command_reply: tid=0 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 client.15804 shutdown 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 msgr2=0x557b047e5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047e5760 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009800 tx=0x7fb5a8005e40 comp rx=0 tx=0).stop 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 shutdown_connections 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x557b04853a60 0x557b04855e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b047dd8f0 0x557b047e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b047d4d10 0x557b047e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5977fe640 0 client.15804 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 --2- 172.21.15.76:0/239867475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557b047d43f0 0x557b047e5760 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 >> 172.21.15.76:0/239867475 conn(0x557b047ceb20 msgr2=0x557b047cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 shutdown_connections 2024-03-23T15:44:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.202+0000 7fb5c15a1740 1 -- 172.21.15.76:0/239867475 wait complete. 2024-03-23T15:44:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5c15a1740 20 client.15804 trim_cache size 0 max 16384 2024-03-23T15:44:30.213 INFO:teuthology.orchestra.run.smithi076.stdout:[] 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fb5a00a2460 msgr2=0x7fb5a00a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fb5a00a2460 0x7fb5a00a4920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00099e0 tx=0x7fb5b0009500 comp rx=0 tx=0).stop 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8100f60 msgr2=0x7fb5b8199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8100f60 0x7fb5b8199020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00ea20 tx=0x7fb5ac00eee0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 shutdown_connections 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fb5a00a2460 0x7fb5a00a4920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5b8106660 0x7fb5b81a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b8100f60 0x7fb5b8199020 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 --2- 172.21.15.76:0/2202434991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5b81005a0 0x7fb5b8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 >> 172.21.15.76:0/2202434991 conn(0x7fb5b8077710 msgr2=0x7fb5b80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 shutdown_connections 2024-03-23T15:44:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.203+0000 7fb5be9c7640 1 -- 172.21.15.76:0/2202434991 wait complete. 2024-03-23T15:44:30.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 -- 172.21.15.76:0/2477695732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 msgr2=0x7f63241011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 --2- 172.21.15.76:0/2477695732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f63241011e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f630c009900 tx=0x7f630c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 -- 172.21.15.76:0/2477695732 shutdown_connections 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 --2- 172.21.15.76:0/2477695732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6324101720 0x7f632410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 --2- 172.21.15.76:0/2477695732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f63241011e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 --2- 172.21.15.76:0/2477695732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63241055a0 0x7f6324105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 -- 172.21.15.76:0/2477695732 >> 172.21.15.76:0/2477695732 conn(0x7f6324076bd0 msgr2=0x7f63240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 -- 172.21.15.76:0/2477695732 shutdown_connections 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.321+0000 7f6328d5e640 1 -- 172.21.15.76:0/2477695732 wait complete. 2024-03-23T15:44:30.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 Processor -- start 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 -- start start 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f632410db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6324101720 0x7f632410e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63241055a0 0x7f632410e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63241a6080 con 0x7f6324100d60 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63241a61f0 con 0x7f6324101720 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.322+0000 7f6328d5e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f63241a63b0 con 0x7f63241055a0 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f632410db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f632410db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45110/0 (socket says 172.21.15.76:45110) 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 -- 172.21.15.76:0/1218611415 learned_addr learned my addr 172.21.15.76:0/1218611415 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f6319d9b640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6324101720 0x7f632410e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f6322d9d640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63241055a0 0x7f632410e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 -- 172.21.15.76:0/1218611415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63241055a0 msgr2=0x7f632410e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63241055a0 0x7f632410e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 -- 172.21.15.76:0/1218611415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6324101720 msgr2=0x7f632410e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6324101720 0x7f632410e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 -- 172.21.15.76:0/1218611415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c009560 con 0x7f6324100d60 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f632259c640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f632410db80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f631400e9f0 tx=0x7f631400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f631bfff640 1 -- 172.21.15.76:0/1218611415 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314009700 con 0x7f6324100d60 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f631bfff640 1 -- 172.21.15.76:0/1218611415 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631400cd80 con 0x7f6324100d60 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63241a6690 con 0x7f6324100d60 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f631bfff640 1 -- 172.21.15.76:0/1218611415 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314018920 con 0x7f6324100d60 2024-03-23T15:44:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.323+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63241a6b70 con 0x7f6324100d60 2024-03-23T15:44:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.325+0000 7f631bfff640 1 -- 172.21.15.76:0/1218611415 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7f6314018a80 con 0x7f6324100d60 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 Processor -- start 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 -- start start 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f631bfff640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f63100a23e0 0x7f63100a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6f3f0 0x55c594c6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c7ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c594bdd840 con 0x55c594c6fd10 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c594bdad70 con 0x55c594c6f3f0 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.326+0000 7f632ba18740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c594bd6550 con 0x55c594c788f0 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6319d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6322d9d640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f63100a23e0 0x7f63100a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f632259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6f3f0 0x55c594c6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6319d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c783b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45112/0 (socket says 172.21.15.76:45112) 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6319d9b640 1 -- 172.21.15.76:0/1021520199 learned_addr learned my addr 172.21.15.76:0/1021520199 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6322d9d640 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c7ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f631bfff640 1 -- 172.21.15.76:0/1218611415 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6314014070 con 0x7f6324100d60 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f632259c640 1 -- 172.21.15.76:0/1021520199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 msgr2=0x55c594c7ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f632259c640 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c7ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f632259c640 1 -- 172.21.15.76:0/1021520199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 msgr2=0x55c594c783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f632259c640 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f632259c640 1 -- 172.21.15.76:0/1021520199 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c594b6bde0 con 0x55c594c6f3f0 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6319d9b640 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6322d9d640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f63100a23e0 0x7f63100a48a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f631c005fd0 tx=0x7f631c009e60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f632259c640 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6f3f0 0x55c594c6f7d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63140cbb60 tx=0x7f6314096b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a521388fbba136f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6318d99640 1 -- 172.21.15.76:0/1021520199 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631409d070 con 0x55c594c6f3f0 2024-03-23T15:44:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6318d99640 1 -- 172.21.15.76:0/1021520199 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63140a0470 con 0x55c594c6f3f0 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.327+0000 7f6318d99640 1 -- 172.21.15.76:0/1021520199 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63140a1280 con 0x55c594c6f3f0 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 -- 172.21.15.76:0/1021520199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6f3f0 msgr2=0x55c594c6f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6f3f0 0x55c594c6f7d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63140cbb60 tx=0x7f6314096b50 comp rx=0 tx=0).stop 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 -- 172.21.15.76:0/1021520199 shutdown_connections 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c7ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 --2- 172.21.15.76:0/1021520199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6f3f0 0x55c594c6f7d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 -- 172.21.15.76:0/1021520199 >> 172.21.15.76:0/1021520199 conn(0x55c594c69b20 msgr2=0x55c594c69f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 -- 172.21.15.76:0/1021520199 shutdown_connections 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.328+0000 7f632ba18740 1 -- 172.21.15.76:0/1021520199 wait complete. 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 -- start start 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 Processor -- start 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c75f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c788f0 0x55c594c76470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c5948a9430 con 0x55c594c6fd10 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c594865e00 con 0x55c594c788f0 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c594b5b590 con 0x55c594c6f3f0 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632ba18740 10 client.0 _ll_register_callbacks cb 0x55c594bcafc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c75f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c75f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45118/0 (socket says 172.21.15.76:45118) 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 -- 172.21.15.76:0/726975055 learned_addr learned my addr 172.21.15.76:0/726975055 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6322d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c788f0 0x55c594c76470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f632259c640 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 -- 172.21.15.76:0/726975055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 msgr2=0x55c594c759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 -- 172.21.15.76:0/726975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c788f0 msgr2=0x55c594c76470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c788f0 0x55c594c76470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.329+0000 7f6319d9b640 1 -- 172.21.15.76:0/726975055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63241a6b70 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f6319d9b640 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c75f30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f630c036040 tx=0x7f630c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5038b25033de4be server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f6309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f6309ffb640 1 -- 172.21.15.76:0/726975055 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c004a90 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f6309ffb640 1 -- 172.21.15.76:0/726975055 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f630c004250 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5948d57e0 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f6309ffb640 1 -- 172.21.15.76:0/726975055 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f630c004410 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 10 client.15816 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c594bd4bd0 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f6309ffb640 1 -- 172.21.15.76:0/726975055 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c5948d57e0 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f6309ffb640 1 -- 172.21.15.76:0/726975055 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f630c03c040 con 0x55c594c6fd10 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 10 client.15816 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 20 client.15816 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:30.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594c88be0 0x55c594c8b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 4 client.15816 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c594912720 con 0x55c594c88be0 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.330+0000 7f632259c640 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594c88be0 0x55c594c8b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.331+0000 7f6309ffb640 10 client.15816 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.331+0000 7f632259c640 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594c88be0 0x55c594c8b000 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfea68aa6d5452d5 server_cookie=fdcd6b9afb754247 in_seq=0 out_seq=0 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.331+0000 7f6309ffb640 1 -- 172.21.15.76:0/726975055 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c594912720 con 0x55c594c88be0 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.331+0000 7f6309ffb640 10 client.15816 handle_command_reply: tid=0 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.331+0000 7f632ba18740 1 client.15816 shutdown 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 msgr2=0x55c594c75f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c75f30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f630c036040 tx=0x7f630c0048f0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 shutdown_connections 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594c88be0 0x55c594c8b000 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c788f0 0x55c594c76470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6fd10 0x55c594c75f30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f6309ffb640 0 client.15816 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:30.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 --2- 172.21.15.76:0/726975055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 >> 172.21.15.76:0/726975055 conn(0x55c594c69b20 msgr2=0x55c594c6b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 shutdown_connections 2024-03-23T15:44:30.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 1 -- 172.21.15.76:0/726975055 wait complete. 2024-03-23T15:44:30.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.332+0000 7f632ba18740 20 client.15816 trim_cache size 0 max 16384 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 Processor -- start 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 -- start start 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6f3f0 0x55c594c78120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c76290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c90910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c594c6f050 con 0x55c594c6f3f0 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c594c653e0 con 0x55c594c6fd10 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632ba18740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c594c73f40 con 0x55c594c788f0 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6f3f0 0x55c594c78120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6f3f0 0x55c594c78120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45122/0 (socket says 172.21.15.76:45122) 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 -- 172.21.15.76:0/2945896048 learned_addr learned my addr 172.21.15.76:0/2945896048 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f6319d9b640 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c76290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f6322d9d640 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c90910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 -- 172.21.15.76:0/2945896048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 msgr2=0x55c594c90910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c90910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 -- 172.21.15.76:0/2945896048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 msgr2=0x55c594c76290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c76290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.345+0000 7f632259c640 1 -- 172.21.15.76:0/2945896048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c594b70c80 con 0x55c594c6f3f0 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.346+0000 7f632259c640 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6f3f0 0x55c594c78120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f63140cbca0 tx=0x7f631401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4062ef3c3d0623fc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.346+0000 7f630bfff640 1 -- 172.21.15.76:0/2945896048 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631409d040 con 0x55c594c6f3f0 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.346+0000 7f630bfff640 1 -- 172.21.15.76:0/2945896048 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63140a35a0 con 0x55c594c6f3f0 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.346+0000 7f630bfff640 1 -- 172.21.15.76:0/2945896048 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314010b80 con 0x55c594c6f3f0 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.346+0000 7f632ba18740 1 -- 172.21.15.76:0/2945896048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6f3f0 msgr2=0x55c594c78120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6f3f0 0x55c594c78120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f63140cbca0 tx=0x7f631401a040 comp rx=0 tx=0).stop 2024-03-23T15:44:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- 172.21.15.76:0/2945896048 shutdown_connections 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c788f0 0x55c594c90910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c76290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 --2- 172.21.15.76:0/2945896048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c6f3f0 0x55c594c78120 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- 172.21.15.76:0/2945896048 >> 172.21.15.76:0/2945896048 conn(0x55c594c69b20 msgr2=0x55c594c6ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- 172.21.15.76:0/2945896048 shutdown_connections 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- 172.21.15.76:0/2945896048 wait complete. 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- start start 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 Processor -- start 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c80810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c788f0 0x55c594c7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c594c7ce90 con 0x55c594c788f0 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c594c7d000 con 0x55c594c6fd10 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c594c7d1c0 con 0x55c594c6f3f0 2024-03-23T15:44:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632ba18740 10 client.0 _ll_register_callbacks cb 0x7f630c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f6322d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c788f0 0x55c594c7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632259c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:60324/0 (socket says 172.21.15.76:60324) 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f6322d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c788f0 0x55c594c7c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45124/0 (socket says 172.21.15.76:45124) 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f632259c640 1 -- 172.21.15.76:0/4275434706 learned_addr learned my addr 172.21.15.76:0/4275434706 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.347+0000 7f6319d9b640 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c80810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6322d9d640 1 -- 172.21.15.76:0/4275434706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 msgr2=0x55c594c802d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6322d9d640 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c802d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6322d9d640 1 -- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 msgr2=0x55c594c80810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6322d9d640 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c80810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6322d9d640 1 -- 172.21.15.76:0/4275434706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c594b6bde0 con 0x55c594c788f0 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f632259c640 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6322d9d640 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c788f0 0x55c594c7c7c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55c594c6eda0 tx=0x7f631c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448e4fd9a400ae4a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6309ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6309ffb640 1 -- 172.21.15.76:0/4275434706 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c003970 con 0x55c594c788f0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f6309ffb640 1 -- 172.21.15.76:0/4275434706 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f631c026040 con 0x55c594c788f0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.348+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63140a0470 con 0x55c594c788f0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f6309ffb640 1 -- 172.21.15.76:0/4275434706 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f631c003ad0 con 0x55c594c788f0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 10 client.15822 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c594b96920 con 0x55c594c788f0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f6309ffb640 1 -- 172.21.15.76:0/4275434706 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63140a0470 con 0x55c594c788f0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f6309ffb640 1 -- 172.21.15.76:0/4275434706 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f631c0046c0 con 0x55c594c788f0 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 10 client.15822 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 20 client.15822 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594cf0150 0x55c594cf2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 4 client.15822 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c594961890 con 0x55c594cf0150 2024-03-23T15:44:30.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632259c640 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594cf0150 0x55c594cf2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f6309ffb640 10 client.15822 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.349+0000 7f632259c640 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594cf0150 0x55c594cf2570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e92cb72348c30ad server_cookie=86f1d42c417c2eae in_seq=0 out_seq=0 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.350+0000 7f6309ffb640 1 -- 172.21.15.76:0/4275434706 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c594961890 con 0x55c594cf0150 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.350+0000 7f6309ffb640 10 client.15822 handle_command_reply: tid=0 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.350+0000 7f632ba18740 1 client.15822 shutdown 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.350+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c788f0 msgr2=0x55c594c7c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.350+0000 7f632ba18740 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c788f0 0x55c594c7c7c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55c594c6eda0 tx=0x7f631c004ab0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 shutdown_connections 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c594c788f0 0x55c594c7c7c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55c594cf0150 0x55c594cf2570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c594c6fd10 0x55c594c80810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 --2- 172.21.15.76:0/4275434706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c594c6f3f0 0x55c594c802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6309ffb640 0 client.15822 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:30.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 >> 172.21.15.76:0/4275434706 conn(0x55c594c69b20 msgr2=0x55c594c6b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 shutdown_connections 2024-03-23T15:44:30.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 1 -- 172.21.15.76:0/4275434706 wait complete. 2024-03-23T15:44:30.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f632ba18740 20 client.15822 trim_cache size 0 max 16384 2024-03-23T15:44:30.360 INFO:teuthology.orchestra.run.smithi076.stdout:[] 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f63100a23e0 msgr2=0x7f63100a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f63100a23e0 0x7f63100a48a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f631c005fd0 tx=0x7f631c009e60 comp rx=0 tx=0).stop 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 msgr2=0x7f632410db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f632410db80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f631400e9f0 tx=0x7f631400eeb0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 shutdown_connections 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f63100a23e0 0x7f63100a48a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63241055a0 0x7f632410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6324101720 0x7f632410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 --2- 172.21.15.76:0/1218611415 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6324100d60 0x7f632410db80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 >> 172.21.15.76:0/1218611415 conn(0x7f6324076bd0 msgr2=0x7f6324103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 shutdown_connections 2024-03-23T15:44:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:30.351+0000 7f6328d5e640 1 -- 172.21.15.76:0/1218611415 wait complete. 2024-03-23T15:44:30.575 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:30 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2304727196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:44:30.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:30 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2304727196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:44:30.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:30 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2304727196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:44:31.365 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-23T15:44:31.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:44:31.366 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-23T15:44:31.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 -- 172.21.15.76:0/2190892575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a78106660 msgr2=0x7f9a78106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/2190892575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a78106660 0x7f9a78106a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009890 tx=0x7f9a7402bb70 comp rx=0 tx=0).stop 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 -- 172.21.15.76:0/2190892575 shutdown_connections 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/2190892575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a78100f60 0x7f9a7810d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/2190892575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a781005a0 0x7f9a78100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/2190892575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a78106660 0x7f9a78106a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 -- 172.21.15.76:0/2190892575 >> 172.21.15.76:0/2190892575 conn(0x7f9a78077710 msgr2=0x7f9a78077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 -- 172.21.15.76:0/2190892575 shutdown_connections 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 -- 172.21.15.76:0/2190892575 wait complete. 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 Processor -- start 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 -- start start 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.474+0000 7f9a7f52e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a781005a0 0x7f9a78197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7f52e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a78100f60 0x7f9a781982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7f52e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a78106660 0x7f9a7819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7f52e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7819cc70 con 0x7f9a78106660 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7f52e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7819cde0 con 0x7f9a78100f60 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7f52e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7819cfa0 con 0x7f9a781005a0 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7dacb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a78106660 0x7f9a7819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a781005a0 0x7f9a78197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7cac9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a78100f60 0x7f9a781982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a781005a0 0x7f9a78197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:60340/0 (socket says 172.21.15.76:60340) 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/3684915424 learned_addr learned my addr 172.21.15.76:0/3684915424 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/3684915424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a78100f60 msgr2=0x7f9a781982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a78100f60 0x7f9a781982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a78106660 msgr2=0x7f9a7819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a78106660 0x7f9a7819c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/3684915424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a70009900 con 0x7f9a781005a0 2024-03-23T15:44:31.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a78106660 0x7f9a7819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a781005a0 0x7f9a78197db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9a740045e0 tx=0x7f9a74004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a6e7fc640 1 -- 172.21.15.76:0/3684915424 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74034990 con 0x7f9a781005a0 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a74009490 con 0x7f9a781005a0 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a6e7fc640 1 -- 172.21.15.76:0/3684915424 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a7402c9f0 con 0x7f9a781005a0 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a6e7fc640 1 -- 172.21.15.76:0/3684915424 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74032eb0 con 0x7f9a781005a0 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.475+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7819d400 con 0x7f9a781005a0 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.477+0000 7f9a6e7fc640 1 -- 172.21.15.76:0/3684915424 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7f9a7403b040 con 0x7f9a781005a0 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 Processor -- start 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 -- start start 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a6e7fc640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a640a23e0 0x7f9a640a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38841b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c3884a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c3884d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c387af530 con 0x562c3884ac60 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562c387aca60 con 0x562c38842080 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a821df740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562c387a8240 con 0x562c38841760 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a640a23e0 0x7f9a640a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a6e7fc640 1 -- 172.21.15.76:0/3684915424 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9a740efdf0 con 0x7f9a781005a0 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a7dacb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c3884d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a7cac9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c3884a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a7dacb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c3884d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45156/0 (socket says 172.21.15.76:45156) 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a7dacb640 1 -- 172.21.15.76:0/1059728608 learned_addr learned my addr 172.21.15.76:0/1059728608 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.478+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38841b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7cac9640 1 -- 172.21.15.76:0/1059728608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 msgr2=0x562c38841b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7cac9640 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38841b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7cac9640 1 -- 172.21.15.76:0/1059728608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 msgr2=0x562c3884d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7cac9640 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c3884d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7cac9640 1 -- 172.21.15.76:0/1059728608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c387a68c0 con 0x562c38842080 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38841b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c3884d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a640a23e0 0x7f9a640a48a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c510 tx=0x7f9a7000e7a0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a7cac9640 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c3884a720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009bd0 tx=0x7f9a68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e69778902a7e484a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a637fe640 1 -- 172.21.15.76:0/1059728608 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68004050 con 0x562c38842080 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a637fe640 1 -- 172.21.15.76:0/1059728608 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a68005500 con 0x562c38842080 2024-03-23T15:44:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.479+0000 7f9a637fe640 1 -- 172.21.15.76:0/1059728608 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68004c00 con 0x562c38842080 2024-03-23T15:44:31.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 -- 172.21.15.76:0/1059728608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 msgr2=0x562c3884a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c3884a720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009bd0 tx=0x7f9a68011810 comp rx=0 tx=0).stop 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 -- 172.21.15.76:0/1059728608 shutdown_connections 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c3884d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c3884a720 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 --2- 172.21.15.76:0/1059728608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38841b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 -- 172.21.15.76:0/1059728608 >> 172.21.15.76:0/1059728608 conn(0x562c3883be90 msgr2=0x562c3883c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 -- 172.21.15.76:0/1059728608 shutdown_connections 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 -- 172.21.15.76:0/1059728608 wait complete. 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 -- start start 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.480+0000 7f9a821df740 1 Processor -- start 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a821df740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38847c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a821df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c388481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a821df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c388486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a821df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c3874cf80 con 0x562c3884ac60 2024-03-23T15:44:31.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a821df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562c3847b4b0 con 0x562c38842080 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a821df740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562c3872dad0 con 0x562c38841760 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a821df740 10 client.0 _ll_register_callbacks cb 0x562c38760b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c388486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7cac9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c388481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c388486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45166/0 (socket says 172.21.15.76:45166) 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 -- 172.21.15.76:0/2916360909 learned_addr learned my addr 172.21.15.76:0/2916360909 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38847c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 -- 172.21.15.76:0/2916360909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 msgr2=0x562c38847c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38847c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 -- 172.21.15.76:0/2916360909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 msgr2=0x562c388481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c388481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 -- 172.21.15.76:0/2916360909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a68009830 con 0x562c3884ac60 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c388486f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000d010 tx=0x7f9a700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e7151588986fa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:31.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a60ff9640 1 -- 172.21.15.76:0/2916360909 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70013070 con 0x562c3884ac60 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.481+0000 7f9a60ff9640 1 -- 172.21.15.76:0/2916360909 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9a7001d8d0 con 0x562c3884ac60 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c387cc7a0 con 0x562c3884ac60 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a60ff9640 1 -- 172.21.15.76:0/2916360909 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a7001da90 con 0x562c3884ac60 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 10 client.15834 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562c3875e6d0 con 0x562c3884ac60 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a60ff9640 1 -- 172.21.15.76:0/2916360909 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a70034020 con 0x562c3884ac60 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a60ff9640 1 -- 172.21.15.76:0/2916360909 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a7000bc60 con 0x562c3884ac60 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 10 client.15834 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 20 client.15834 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c3885add0 0x562c3885d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 4 client.15834 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:31.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c3879e1f0 con 0x562c3885add0 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.482+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c3885add0 0x562c3885d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.483+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c3885add0 0x562c3885d1f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4037e23228e840ec server_cookie=28781761a6ae537e in_seq=0 out_seq=0 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.483+0000 7f9a60ff9640 10 client.15834 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.484+0000 7f9a60ff9640 1 -- 172.21.15.76:0/2916360909 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562c3879e1f0 con 0x562c3885add0 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.484+0000 7f9a60ff9640 10 client.15834 handle_command_reply: tid=0 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.484+0000 7f9a821df740 1 client.15834 shutdown 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 msgr2=0x562c388486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c388486f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000d010 tx=0x7f9a700095c0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 shutdown_connections 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c3884ac60 0x562c388486f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c3885add0 0x562c3885d1f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c388481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 --2- 172.21.15.76:0/2916360909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38841760 0x562c38847c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 >> 172.21.15.76:0/2916360909 conn(0x562c3883be90 msgr2=0x562c3883dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:31.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a60ff9640 0 client.15834 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:31.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 shutdown_connections 2024-03-23T15:44:31.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 1 -- 172.21.15.76:0/2916360909 wait complete. 2024-03-23T15:44:31.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.485+0000 7f9a821df740 20 client.15834 trim_cache size 0 max 16384 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.502+0000 7f9a821df740 1 Processor -- start 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.502+0000 7f9a821df740 1 -- start start 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.502+0000 7f9a821df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c38852f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.502+0000 7f9a821df740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38842080 0x562c388564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.502+0000 7f9a821df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c3884ac60 0x562c38848510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.502+0000 7f9a821df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c388413c0 con 0x562c38841760 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a821df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562c3883ca60 con 0x562c3884ac60 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a821df740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562c3883ce90 con 0x562c38842080 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c38852f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c38852f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45172/0 (socket says 172.21.15.76:45172) 2024-03-23T15:44:31.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/1606990039 learned_addr learned my addr 172.21.15.76:0/1606990039 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c3884ac60 0x562c38848510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7cac9640 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38842080 0x562c388564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/1606990039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38842080 msgr2=0x562c388564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38842080 0x562c388564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/1606990039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c3884ac60 msgr2=0x562c38848510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c3884ac60 0x562c38848510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/1606990039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c38742c60 con 0x562c38841760 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c38852f00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a7402c180 tx=0x7f9a740a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ae17ac92737485 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a62ffd640 1 -- 172.21.15.76:0/1606990039 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a740bc760 con 0x562c38841760 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a62ffd640 1 -- 172.21.15.76:0/1606990039 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a740c0800 con 0x562c38841760 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.503+0000 7f9a62ffd640 1 -- 172.21.15.76:0/1606990039 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74032600 con 0x562c38841760 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 -- 172.21.15.76:0/1606990039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 msgr2=0x562c38852f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c38852f00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a7402c180 tx=0x7f9a740a7040 comp rx=0 tx=0).stop 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 -- 172.21.15.76:0/1606990039 shutdown_connections 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c3884ac60 0x562c38848510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c38842080 0x562c388564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 --2- 172.21.15.76:0/1606990039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c38852f00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 -- 172.21.15.76:0/1606990039 >> 172.21.15.76:0/1606990039 conn(0x562c3883be90 msgr2=0x562c3883aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 -- 172.21.15.76:0/1606990039 shutdown_connections 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 -- 172.21.15.76:0/1606990039 wait complete. 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 -- start start 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.504+0000 7f9a821df740 1 Processor -- start 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c388453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c38845910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c3884ac60 0x562c388440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c388447e0 con 0x562c38841760 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562c38844950 con 0x562c38842080 2024-03-23T15:44:31.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562c38844b10 con 0x562c3884ac60 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 10 client.0 _ll_register_callbacks cb 0x7f9a70009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c388453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c388453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45182/0 (socket says 172.21.15.76:45182) 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/772954156 learned_addr learned my addr 172.21.15.76:0/772954156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7cac9640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c38845910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c3884ac60 0x562c388440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/772954156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c3884ac60 msgr2=0x562c388440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c3884ac60 0x562c388440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 msgr2=0x562c38845910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c38845910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 -- 172.21.15.76:0/772954156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a74009490 con 0x562c38841760 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7dacb640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c3884ac60 0x562c388440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a7d2ca640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c388453d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a740c4920 tx=0x7f9a74005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c11e09fe349170 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:31.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a60ff9640 1 -- 172.21.15.76:0/772954156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a740bc760 con 0x562c38841760 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a60ff9640 1 -- 172.21.15.76:0/772954156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9a740baa60 con 0x562c38841760 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a68005500 con 0x562c38841760 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a60ff9640 1 -- 172.21.15.76:0/772954156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a740b8e40 con 0x562c38841760 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 10 client.15840 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.505+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562c38768970 con 0x562c38841760 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a60ff9640 1 -- 172.21.15.76:0/772954156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a68005500 con 0x562c38841760 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a60ff9640 1 -- 172.21.15.76:0/772954156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a74090200 con 0x562c38841760 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a821df740 10 client.15840 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a821df740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a821df740 20 client.15840 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a821df740 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c388c60c0 0x562c388c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a821df740 4 client.15840 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-23T15:44:31.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c384a7860 con 0x562c388c60c0 2024-03-23T15:44:31.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a7cac9640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c388c60c0 0x562c388c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:31.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a60ff9640 10 client.15840 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:31.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.506+0000 7f9a7cac9640 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c388c60c0 0x562c388c84e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbbdfd36fd822b42 server_cookie=37fa45fd402a758 in_seq=0 out_seq=0 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a60ff9640 1 -- 172.21.15.76:0/772954156 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c384a7860 con 0x562c388c60c0 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a60ff9640 10 client.15840 handle_command_reply: tid=0 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 client.15840 shutdown 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 msgr2=0x562c388453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c388453d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a740c4920 tx=0x7f9a74005f90 comp rx=0 tx=0).stop 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 shutdown_connections 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562c3884ac60 0x562c388440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562c388c60c0 0x562c388c84e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562c38842080 0x562c38845910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a60ff9640 0 client.15840 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 --2- 172.21.15.76:0/772954156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c38841760 0x562c388453d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 >> 172.21.15.76:0/772954156 conn(0x562c3883be90 msgr2=0x562c3883d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 shutdown_connections 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 1 -- 172.21.15.76:0/772954156 wait complete. 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.510+0000 7f9a821df740 20 client.15840 trim_cache size 0 max 16384 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "38295c3f-aac4-467d-afc6-2aae6db5e9b0", 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-03-23T15:44:31.517 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:44:31.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a640a23e0 msgr2=0x7f9a640a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a640a23e0 0x7f9a640a48a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c510 tx=0x7f9a7000e7a0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a781005a0 msgr2=0x7f9a78197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:31.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a781005a0 0x7f9a78197db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9a740045e0 tx=0x7f9a74004610 comp rx=0 tx=0).stop 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 shutdown_connections 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a640a23e0 0x7f9a640a48a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a78106660 0x7f9a7819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a78100f60 0x7f9a781982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 --2- 172.21.15.76:0/3684915424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a781005a0 0x7f9a78197db0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 >> 172.21.15.76:0/3684915424 conn(0x7f9a78077710 msgr2=0x7f9a780ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 shutdown_connections 2024-03-23T15:44:31.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:31.511+0000 7f9a7f52e640 1 -- 172.21.15.76:0/3684915424 wait complete. 2024-03-23T15:44:31.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T15:44:32.148 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T15:44:32.149 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:31 smithi192 ceph-mon[36653]: pgmap v38: 97 pgs: 97 active+clean; 27 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 825 op/s 2024-03-23T15:44:32.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:31 smithi183 ceph-mon[36296]: pgmap v38: 97 pgs: 97 active+clean; 27 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 825 op/s 2024-03-23T15:44:32.149 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:31 smithi076 ceph-mon[32176]: pgmap v38: 97 pgs: 97 active+clean; 27 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 825 op/s 2024-03-23T15:44:32.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/3896241702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84106660 msgr2=0x7f5f84106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/3896241702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84106660 0x7f5f84106a40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009900 tx=0x7f5f8002bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:32.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/3896241702 shutdown_connections 2024-03-23T15:44:32.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/3896241702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f84100f60 0x7f5f8410d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/3896241702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f841005a0 0x7f5f84100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/3896241702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84106660 0x7f5f84106a40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/3896241702 >> 172.21.15.76:0/3896241702 conn(0x7f5f84077710 msgr2=0x7f5f84077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/3896241702 shutdown_connections 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/3896241702 wait complete. 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.253+0000 7f5f8bd5e640 1 Processor -- start 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 -- start start 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f841005a0 0x7f5f8419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84100f60 0x7f5f8419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f84106660 0x7f5f84197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f84198190 con 0x7f5f84100f60 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f84198300 con 0x7f5f84106660 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f841984c0 con 0x7f5f841005a0 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84100f60 0x7f5f8419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84100f60 0x7f5f8419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45204/0 (socket says 172.21.15.76:45204) 2024-03-23T15:44:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 -- 172.21.15.76:0/2792152621 learned_addr learned my addr 172.21.15.76:0/2792152621 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f89afa640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f841005a0 0x7f5f8419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f84106660 0x7f5f84197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 -- 172.21.15.76:0/2792152621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f841005a0 msgr2=0x7f5f8419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f841005a0 0x7f5f8419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 -- 172.21.15.76:0/2792152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f84106660 msgr2=0x7f5f84197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f84106660 0x7f5f84197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 -- 172.21.15.76:0/2792152621 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f74009490 con 0x7f5f84100f60 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f892f9640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84100f60 0x7f5f8419de80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400ea60 tx=0x7f5f7400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f7affd640 1 -- 172.21.15.76:0/2792152621 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400cde0 con 0x7f5f84100f60 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f80009560 con 0x7f5f84100f60 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.254+0000 7f5f7affd640 1 -- 172.21.15.76:0/2792152621 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f740035f0 con 0x7f5f84100f60 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.255+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f84198990 con 0x7f5f84100f60 2024-03-23T15:44:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.255+0000 7f5f7affd640 1 -- 172.21.15.76:0/2792152621 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74005220 con 0x7f5f84100f60 2024-03-23T15:44:32.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.256+0000 7f5f7affd640 1 -- 172.21.15.76:0/2792152621 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7f5f74003af0 con 0x7f5f84100f60 2024-03-23T15:44:32.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.257+0000 7f5f8ea1e740 1 Processor -- start 2024-03-23T15:44:32.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.257+0000 7f5f7affd640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f700a23e0 0x7f5f700a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.257+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f700a23e0 0x7f5f700a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.257+0000 7f5f7affd640 1 -- 172.21.15.76:0/2792152621 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5f74014070 con 0x7f5f84100f60 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.257+0000 7f5f8ea1e740 1 -- start start 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50a1d0 0x5629bf50a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf513190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf515ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5629bf42ef30 con 0x5629bf50a1d0 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5629bf478810 con 0x5629bf50aaf0 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5629bf475d40 con 0x5629bf5136d0 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50a1d0 0x5629bf50a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f892f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf513190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f700a23e0 0x7f5f700a48a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005fd0 tx=0x7f5f7c009500 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50a1d0 0x5629bf50a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45214/0 (socket says 172.21.15.76:45214) 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 -- 172.21.15.76:0/3720986099 learned_addr learned my addr 172.21.15.76:0/3720986099 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:32.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf515ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 -- 172.21.15.76:0/3720986099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 msgr2=0x5629bf515ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf515ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 -- 172.21.15.76:0/3720986099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 msgr2=0x5629bf513190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf513190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 -- 172.21.15.76:0/3720986099 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bf432ea0 con 0x5629bf50a1d0 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f892f9640 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf513190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f89afa640 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50a1d0 0x5629bf50a5b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009900 tx=0x7f5f8002c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d53984dc11b56e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f6ffff640 1 -- 172.21.15.76:0/3720986099 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8002c910 con 0x5629bf50a1d0 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f6ffff640 1 -- 172.21.15.76:0/3720986099 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f8002ca70 con 0x5629bf50a1d0 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.258+0000 7f5f6ffff640 1 -- 172.21.15.76:0/3720986099 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80005170 con 0x5629bf50a1d0 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/3720986099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50a1d0 msgr2=0x5629bf50a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50a1d0 0x5629bf50a5b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009900 tx=0x7f5f8002c6a0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/3720986099 shutdown_connections 2024-03-23T15:44:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf515ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf513190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/3720986099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50a1d0 0x5629bf50a5b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/3720986099 >> 172.21.15.76:0/3720986099 conn(0x5629bf504900 msgr2=0x5629bf504ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/3720986099 shutdown_connections 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.259+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/3720986099 wait complete. 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 -- start start 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 Processor -- start 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf510730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50aaf0 0x5629bf510c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf5111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5629bf471520 con 0x5629bf50aaf0 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5629bf144430 con 0x5629bf50a1d0 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5629bf100e00 con 0x5629bf5136d0 2024-03-23T15:44:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8ea1e740 10 client.0 _ll_register_callbacks cb 0x5629bf40fd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50aaf0 0x5629bf510c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50aaf0 0x5629bf510c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45226/0 (socket says 172.21.15.76:45226) 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 -- 172.21.15.76:0/2471476535 learned_addr learned my addr 172.21.15.76:0/2471476535 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf5111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f89afa640 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf510730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 -- 172.21.15.76:0/2471476535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 msgr2=0x5629bf5111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf5111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 -- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 msgr2=0x5629bf510730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf510730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.260+0000 7f5f892f9640 1 -- 172.21.15.76:0/2471476535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8410dd20 con 0x5629bf50aaf0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f892f9640 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50aaf0 0x5629bf510c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5f740cbd60 tx=0x7f5f7409b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63356186fe93b42 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/2471476535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f740b94a0 con 0x5629bf50aaf0 2024-03-23T15:44:32.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/2471476535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5f74003760 con 0x5629bf50aaf0 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629bf1707e0 con 0x5629bf50aaf0 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/2471476535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5f74003920 con 0x5629bf50aaf0 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 10 client.15852 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5629bf46fba0 con 0x5629bf50aaf0 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/2471476535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5629bf1707e0 con 0x5629bf50aaf0 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/2471476535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5f7409e040 con 0x5629bf50aaf0 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 10 client.15852 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 20 client.15852 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf523b60 0x5629bf525fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 4 client.15852 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629bf1ad720 con 0x5629bf523b60 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.261+0000 7f5f89afa640 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf523b60 0x5629bf525fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.262+0000 7f5f6d7fa640 10 client.15852 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:32.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.262+0000 7f5f89afa640 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf523b60 0x5629bf525fa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bae020f2b5bfc08 server_cookie=bce795d89ca22ab0 in_seq=0 out_seq=0 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/2471476535 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5629bf1ad720 con 0x5629bf523b60 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f6d7fa640 10 client.15852 handle_command_reply: tid=0 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 client.15852 shutdown 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50aaf0 msgr2=0x5629bf510c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50aaf0 0x5629bf510c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5f740cbd60 tx=0x7f5f7409b050 comp rx=0 tx=0).stop 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 shutdown_connections 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf5136d0 0x5629bf5111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf50aaf0 0x5629bf510c70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf523b60 0x5629bf525fa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/2471476535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf510730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 >> 172.21.15.76:0/2471476535 conn(0x5629bf504900 msgr2=0x5629bf503b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:32.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f6d7fa640 0 client.15852 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:32.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 shutdown_connections 2024-03-23T15:44:32.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/2471476535 wait complete. 2024-03-23T15:44:32.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.265+0000 7f5f8ea1e740 20 client.15852 trim_cache size 0 max 16384 2024-03-23T15:44:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 Processor -- start 2024-03-23T15:44:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 -- start start 2024-03-23T15:44:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf51ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50aaf0 0x5629bf510fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf52b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5629bf3f6430 con 0x5629bf5136d0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5629bf5205b0 con 0x5629bf50a1d0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5629bf5095a0 con 0x5629bf50aaf0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf52b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf52b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45234/0 (socket says 172.21.15.76:45234) 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/4182578331 learned_addr learned my addr 172.21.15.76:0/4182578331 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f89afa640 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf51ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f892f9640 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50aaf0 0x5629bf510fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/4182578331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50aaf0 msgr2=0x5629bf510fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50aaf0 0x5629bf510fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/4182578331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 msgr2=0x5629bf51ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf51ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/4182578331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629bf40bb20 con 0x5629bf5136d0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf52b8b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00e790 tx=0x7f5f7c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3da1eb81ac7cedc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:32.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.283+0000 7f5f6f7fe640 1 -- 172.21.15.76:0/4182578331 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c024070 con 0x5629bf5136d0 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f6f7fe640 1 -- 172.21.15.76:0/4182578331 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c003e50 con 0x5629bf5136d0 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f6f7fe640 1 -- 172.21.15.76:0/4182578331 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c004050 con 0x5629bf5136d0 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/4182578331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 msgr2=0x5629bf52b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf52b8b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00e790 tx=0x7f5f7c014000 comp rx=0 tx=0).stop 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/4182578331 shutdown_connections 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf52b8b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50aaf0 0x5629bf510fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/4182578331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50a1d0 0x5629bf51ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/4182578331 >> 172.21.15.76:0/4182578331 conn(0x5629bf504900 msgr2=0x5629bf503b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/4182578331 shutdown_connections 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/4182578331 wait complete. 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 -- start start 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.284+0000 7f5f8ea1e740 1 Processor -- start 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50a1d0 0x5629bf50d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf50dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8ea1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf588ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5629bf50e570 con 0x5629bf5136d0 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5629bf589420 con 0x5629bf50aaf0 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8ea1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5629bf589560 con 0x5629bf50a1d0 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8ea1e740 10 client.0 _ll_register_callbacks cb 0x7f5f7409ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf588ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f89afa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50a1d0 0x5629bf50d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf588ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45248/0 (socket says 172.21.15.76:45248) 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/144180982 learned_addr learned my addr 172.21.15.76:0/144180982 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f892f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf50dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/144180982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50a1d0 msgr2=0x5629bf50d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50a1d0 0x5629bf50d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/144180982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 msgr2=0x5629bf50dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf50dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 -- 172.21.15.76:0/144180982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c017070 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f8a2fb640 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf588ee0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00e760 tx=0x7f5f7c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05d7333ccfab1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.285+0000 7f5f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/144180982 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c024040 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/144180982 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f5f7c003b80 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8002ca70 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/144180982 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5f7c004a40 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 10 client.15858 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5629bf431700 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/144180982 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c03d080 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/144180982 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5f7c005040 con 0x5629bf5136d0 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 10 client.15858 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:32.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:32.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 20 client.15858 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:32.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf58ae10 0x5629bf58d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:32.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 4 client.15858 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T15:44:32.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629bf1fc890 con 0x5629bf58ae10 2024-03-23T15:44:32.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.286+0000 7f5f89afa640 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf58ae10 0x5629bf58d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:32.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.287+0000 7f5f89afa640 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf58ae10 0x5629bf58d230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3824424f486a965e server_cookie=bcbb17544b07cab5 in_seq=0 out_seq=0 2024-03-23T15:44:32.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.287+0000 7f5f6d7fa640 10 client.15858 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:32.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f6d7fa640 1 -- 172.21.15.76:0/144180982 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5629bf1fc890 con 0x5629bf58ae10 2024-03-23T15:44:32.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f6d7fa640 10 client.15858 handle_command_reply: tid=0 2024-03-23T15:44:32.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 client.15858 shutdown 2024-03-23T15:44:32.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 msgr2=0x5629bf588ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf588ee0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00e760 tx=0x7f5f7c00edd0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 shutdown_connections 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629bf5136d0 0x5629bf588ee0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5629bf58ae10 0x5629bf58d230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5629bf50aaf0 0x5629bf50dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 --2- 172.21.15.76:0/144180982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5629bf50a1d0 0x5629bf50d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f6d7fa640 0 client.15858 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 >> 172.21.15.76:0/144180982 conn(0x5629bf504900 msgr2=0x5629bf505ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 shutdown_connections 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 1 -- 172.21.15.76:0/144180982 wait complete. 2024-03-23T15:44:32.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8ea1e740 20 client.15858 trim_cache size 0 max 16384 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "scrub active (101 inodes in the stack)", 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: "38295c3f-aac4-467d-afc6-2aae6db5e9b0": { 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/", 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "38295c3f-aac4-467d-afc6-2aae6db5e9b0", 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:44:32.298 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:44:32.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f700a23e0 msgr2=0x7f5f700a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f700a23e0 0x7f5f700a48a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005fd0 tx=0x7f5f7c009500 comp rx=0 tx=0).stop 2024-03-23T15:44:32.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84100f60 msgr2=0x7f5f8419de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:32.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.292+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84100f60 0x7f5f8419de80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400ea60 tx=0x7f5f7400ef20 comp rx=0 tx=0).stop 2024-03-23T15:44:32.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 shutdown_connections 2024-03-23T15:44:32.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5f700a23e0 0x7f5f700a48a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f84106660 0x7f5f84197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f84100f60 0x7f5f8419de80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 --2- 172.21.15.76:0/2792152621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f841005a0 0x7f5f8419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:32.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 >> 172.21.15.76:0/2792152621 conn(0x7f5f84077710 msgr2=0x7f5f840ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:32.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 shutdown_connections 2024-03-23T15:44:32.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:44:32.293+0000 7f5f8bd5e640 1 -- 172.21.15.76:0/2792152621 wait complete. 2024-03-23T15:44:32.306 INFO:tasks.cephfs.filesystem:scrub status for tag:38295c3f-aac4-467d-afc6-2aae6db5e9b0 - {'path': '/', 'tag': '38295c3f-aac4-467d-afc6-2aae6db5e9b0', 'options': 'recursive,force'} 2024-03-23T15:44:33.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:33 smithi183 ceph-mon[36296]: scrub queued for path: / 2024-03-23T15:44:33.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:33 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:33.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:33 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:33.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:33 smithi183 ceph-mon[36296]: pgmap v39: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 838 op/s 2024-03-23T15:44:33.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:33 smithi192 ceph-mon[36653]: scrub queued for path: / 2024-03-23T15:44:33.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:33 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:33.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:33 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:33.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:33 smithi192 ceph-mon[36653]: pgmap v39: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 838 op/s 2024-03-23T15:44:33.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:33 smithi076 ceph-mon[32176]: scrub queued for path: / 2024-03-23T15:44:33.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:33 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:33.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:33 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:33.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:33 smithi076 ceph-mon[32176]: pgmap v39: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 838 op/s 2024-03-23T15:44:35.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:35 smithi183 ceph-mon[36296]: pgmap v40: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-03-23T15:44:35.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:35 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:35.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:35 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:35 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:35.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:35 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:35 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:35.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:35 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:35 smithi192 ceph-mon[36653]: pgmap v40: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-03-23T15:44:35.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:35 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:35.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:35 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:35 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:35.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:35 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:35 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:35.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:35 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:35 smithi076 ceph-mon[32176]: pgmap v40: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-03-23T15:44:35.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:35 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:35.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:35 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:35 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:35.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:35 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:35.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:35 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:35.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:35 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3428243041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 msgr2=0x7f8c48105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3428243041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48105970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009900 tx=0x7f8c3c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3428243041 shutdown_connections 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3428243041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c48101200 0x7f8c4810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3428243041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100860 0x7f8c48100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3428243041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48105970 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3428243041 >> 172.21.15.76:0/3428243041 conn(0x7f8c480773e0 msgr2=0x7f8c480777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3428243041 shutdown_connections 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.072+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3428243041 wait complete. 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 Processor -- start 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 -- start start 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100860 0x7f8c4819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c48101200 0x7f8c4819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481982a0 con 0x7f8c48105590 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48198410 con 0x7f8c48100860 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ecd0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c481985d0 con 0x7f8c48101200 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4d26d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ca6c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100860 0x7f8c4819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4d26d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39142/0 (socket says 172.21.15.76:39142) 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3010113489 learned_addr learned my addr 172.21.15.76:0/3010113489 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c4ca6c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100860 0x7f8c4819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:40864/0 (socket says 172.21.15.76:40864) 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.073+0000 7f8c47fff640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c48101200 0x7f8c4819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3010113489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c48101200 msgr2=0x7f8c4819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c48101200 0x7f8c4819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3010113489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100860 msgr2=0x7f8c4819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100860 0x7f8c4819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3010113489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3c009560 con 0x7f8c48105590 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48197b10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000eb10 tx=0x7f8c4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c45ffb640 1 -- 172.21.15.76:0/3010113489 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004b20 con 0x7f8c48105590 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c45ffb640 1 -- 172.21.15.76:0/3010113489 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c40004c80 con 0x7f8c48105590 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c45ffb640 1 -- 172.21.15.76:0/3010113489 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40010650 con 0x7f8c48105590 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c481988b0 con 0x7f8c48105590 2024-03-23T15:44:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.074+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481a49d0 con 0x7f8c48105590 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.076+0000 7f8c45ffb640 1 -- 172.21.15.76:0/3010113489 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 24) ==== 137040+0+0 (secure 0 0 0) 0x7f8c40005020 con 0x7f8c48105590 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c45ffb640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8c380a2480 0x7f8c380a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 Processor -- start 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8c380a2480 0x7f8c380a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c45ffb640 1 -- 172.21.15.76:0/3010113489 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8c40014070 con 0x7f8c48105590 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 -- start start 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f824905e0 0x558f824909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82490f00 0x558f82499790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558f8244b1b0 con 0x558f82499cd0 2024-03-23T15:44:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f823fdf80 con 0x558f824905e0 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.077+0000 7f8c51982740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558f823fb4b0 con 0x558f82490f00 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39150/0 (socket says 172.21.15.76:39150) 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3197471188 learned_addr learned my addr 172.21.15.76:0/3197471188 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8c380a2480 0x7f8c380a4940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c006010 tx=0x7f8c3c005e40 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f824905e0 0x558f824909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c47fff640 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82490f00 0x558f82499790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3197471188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82490f00 msgr2=0x558f82499790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82490f00 0x558f82499790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3197471188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f824905e0 msgr2=0x558f824909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f824905e0 0x558f824909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 -- 172.21.15.76:0/3197471188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f8238c810 con 0x558f82499cd0 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249c0c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8c400cbee0 tx=0x7f8c40000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a4ceb81f6f4883 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c32ffd640 1 -- 172.21.15.76:0/3197471188 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c400b1030 con 0x558f82499cd0 2024-03-23T15:44:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c32ffd640 1 -- 172.21.15.76:0/3197471188 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c400a37e0 con 0x558f82499cd0 2024-03-23T15:44:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.078+0000 7f8c32ffd640 1 -- 172.21.15.76:0/3197471188 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40095bb0 con 0x558f82499cd0 2024-03-23T15:44:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 -- 172.21.15.76:0/3197471188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 msgr2=0x558f8249c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249c0c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8c400cbee0 tx=0x7f8c40000f40 comp rx=0 tx=0).stop 2024-03-23T15:44:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 -- 172.21.15.76:0/3197471188 shutdown_connections 2024-03-23T15:44:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249c0c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82490f00 0x558f82499790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 --2- 172.21.15.76:0/3197471188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f824905e0 0x558f824909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 -- 172.21.15.76:0/3197471188 >> 172.21.15.76:0/3197471188 conn(0x558f8248af00 msgr2=0x558f8248b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 -- 172.21.15.76:0/3197471188 shutdown_connections 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 -- 172.21.15.76:0/3197471188 wait complete. 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.079+0000 7f8c51982740 1 -- start start 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 1 Processor -- start 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f82496dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f82497310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f82497850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558f823f6c90 con 0x558f82499cd0 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f8239bc50 con 0x558f82490f00 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558f820ca250 con 0x558f824905e0 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c51982740 10 client.0 _ll_register_callbacks cb 0x558f823ea290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f82497850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f82497850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39152/0 (socket says 172.21.15.76:39152) 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 -- 172.21.15.76:0/1898366598 learned_addr learned my addr 172.21.15.76:0/1898366598 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f82496dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c47fff640 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f82497310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 -- 172.21.15.76:0/1898366598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 msgr2=0x558f82496dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f82496dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 -- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 msgr2=0x558f82497310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f82497310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.080+0000 7f8c4d26d640 1 -- 172.21.15.76:0/1898366598 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481a49d0 con 0x558f82499cd0 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f82497850 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c400049d0 tx=0x7f8c40097410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1634f3c58444442 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c27fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c27fff640 1 -- 172.21.15.76:0/1898366598 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c400b1070 con 0x558f82499cd0 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c27fff640 1 -- 172.21.15.76:0/1898366598 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8c400b2070 con 0x558f82499cd0 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f8241b790 con 0x558f82499cd0 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c27fff640 1 -- 172.21.15.76:0/1898366598 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c40095bb0 con 0x558f82499cd0 2024-03-23T15:44:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 10 client.15870 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558f823f5310 con 0x558f82499cd0 2024-03-23T15:44:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c27fff640 1 -- 172.21.15.76:0/1898366598 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558f8241b790 con 0x558f82499cd0 2024-03-23T15:44:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c27fff640 1 -- 172.21.15.76:0/1898366598 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8c4001f060 con 0x558f82499cd0 2024-03-23T15:44:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 10 client.15870 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 20 client.15870 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f824a9a10 0x558f824abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 4 client.15870 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f820f6600 con 0x558f824a9a10 2024-03-23T15:44:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.081+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f824a9a10 0x558f824abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.082+0000 7f8c27fff640 10 client.15870 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.082+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f824a9a10 0x558f824abe30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd06b30fb196e5f server_cookie=b32f510582260114 in_seq=0 out_seq=0 2024-03-23T15:44:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c27fff640 1 -- 172.21.15.76:0/1898366598 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558f820f6600 con 0x558f824a9a10 2024-03-23T15:44:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c27fff640 10 client.15870 handle_command_reply: tid=0 2024-03-23T15:44:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 client.15870 shutdown 2024-03-23T15:44:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 msgr2=0x558f82497850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f82497850 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c400049d0 tx=0x7f8c40097410 comp rx=0 tx=0).stop 2024-03-23T15:44:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 shutdown_connections 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f82497850 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f824a9a10 0x558f824abe30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f82497310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c27fff640 0 client.15870 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 --2- 172.21.15.76:0/1898366598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f82496dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 >> 172.21.15.76:0/1898366598 conn(0x558f8248af00 msgr2=0x558f8248cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 shutdown_connections 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 1 -- 172.21.15.76:0/1898366598 wait complete. 2024-03-23T15:44:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.085+0000 7f8c51982740 20 client.15870 trim_cache size 0 max 16384 2024-03-23T15:44:36.106 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.099+0000 7f8c51982740 1 Processor -- start 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.099+0000 7f8c51982740 1 -- start start 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c51982740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f824905e0 0x558f82499500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c51982740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f824a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c51982740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82499cd0 0x558f82497670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c51982740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558f8237c660 con 0x558f824905e0 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c51982740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f824a6730 con 0x558f82490f00 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c51982740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558f824866c0 con 0x558f82499cd0 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f824905e0 0x558f82499500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f824905e0 0x558f82499500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39168/0 (socket says 172.21.15.76:39168) 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/746475227 learned_addr learned my addr 172.21.15.76:0/746475227 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c47fff640 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f824a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82499cd0 0x558f82497670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/746475227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82499cd0 msgr2=0x558f82497670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82499cd0 0x558f82497670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/746475227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 msgr2=0x558f824a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f824a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/746475227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f82391810 con 0x558f824905e0 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f824905e0 0x558f82499500 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02c170 tx=0x7f8c3c044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7194082094f4b7a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c327fc640 1 -- 172.21.15.76:0/746475227 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c053040 con 0x558f824905e0 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c327fc640 1 -- 172.21.15.76:0/746475227 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c3c02cb70 con 0x558f824905e0 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.100+0000 7f8c327fc640 1 -- 172.21.15.76:0/746475227 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0567e0 con 0x558f824905e0 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 -- 172.21.15.76:0/746475227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f824905e0 msgr2=0x558f82499500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f824905e0 0x558f82499500 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02c170 tx=0x7f8c3c044000 comp rx=0 tx=0).stop 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 -- 172.21.15.76:0/746475227 shutdown_connections 2024-03-23T15:44:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f82499cd0 0x558f82497670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f824a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 --2- 172.21.15.76:0/746475227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f824905e0 0x558f82499500 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 -- 172.21.15.76:0/746475227 >> 172.21.15.76:0/746475227 conn(0x558f8248af00 msgr2=0x558f824956a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 -- 172.21.15.76:0/746475227 shutdown_connections 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 -- 172.21.15.76:0/746475227 wait complete. 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 -- start start 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.101+0000 7f8c51982740 1 Processor -- start 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c51982740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f824a1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c51982740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f8249cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c51982740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c51982740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558f824a2230 con 0x558f82499cd0 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c51982740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f8249d840 con 0x558f82490f00 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c51982740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558f8249da00 con 0x558f824905e0 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c51982740 10 client.0 _ll_register_callbacks cb 0x7f8c400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f824a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4d26d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f8249cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f824a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:59846/0 (socket says 172.21.15.76:59846) 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/3403925605 learned_addr learned my addr 172.21.15.76:0/3403925605 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/3403925605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 msgr2=0x558f8249cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f8249cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/3403925605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 msgr2=0x558f8249d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 -- 172.21.15.76:0/3403925605 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4810ddc0 con 0x558f824905e0 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4d26d640 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c4ca6c640 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f824a1930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02cce0 tx=0x7f8c3c02cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6844a02ec1cfcc23 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.102+0000 7f8c27fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-23T15:44:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c27fff640 1 -- 172.21.15.76:0/3403925605 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c053040 con 0x558f824905e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c27fff640 1 -- 172.21.15.76:0/3403925605 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8c3c047040 con 0x558f824905e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c400a37e0 con 0x558f824905e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c27fff640 1 -- 172.21.15.76:0/3403925605 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c3c0044c0 con 0x558f824905e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 10 client.25681 fetch_fsmap learned FSMap version 23 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558f823b75a0 con 0x558f824905e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c27fff640 1 -- 172.21.15.76:0/3403925605 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c400a37e0 con 0x558f824905e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c27fff640 1 -- 172.21.15.76:0/3403925605 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8c3c005590 con 0x558f824905e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 10 client.25681 fetch_fsmap finished waiting for FSMap version 23 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 10 client.25681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 20 client.25681 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f825100e0 0x558f82512500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 4 client.25681 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.103+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f821471b0 con 0x558f825100e0 2024-03-23T15:44:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.104+0000 7f8c47fff640 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f825100e0 0x558f82512500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.104+0000 7f8c47fff640 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f825100e0 0x558f82512500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218ecc18c5b82e4d server_cookie=72bd2f6f79f782ab in_seq=0 out_seq=0 2024-03-23T15:44:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:36.104+0000 7f8c27fff640 10 client.25681 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:36.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:36 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:36.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:36.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:36.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:36.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:36 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:36.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:36.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:36 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: pgmap v41: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-03-23T15:44:37.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:37.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:37 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: pgmap v41: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:37.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:37 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: pgmap v41: 97 pgs: 97 active+clean; 34 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-03-23T15:44:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:37.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:37.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:37.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:37.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:37 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:39.627 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:39 smithi076 ceph-mon[32176]: pgmap v42: 97 pgs: 97 active+clean; 74 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 1.21k op/s 2024-03-23T15:44:39.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:39 smithi183 ceph-mon[36296]: pgmap v42: 97 pgs: 97 active+clean; 74 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 1.21k op/s 2024-03-23T15:44:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:39 smithi192 ceph-mon[36653]: pgmap v42: 97 pgs: 97 active+clean; 74 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 1.21k op/s 2024-03-23T15:44:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c27fff640 1 -- 172.21.15.76:0/3403925605 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f821471b0 con 0x558f825100e0 2024-03-23T15:44:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c27fff640 10 client.25681 handle_command_reply: tid=0 2024-03-23T15:44:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 client.25681 shutdown 2024-03-23T15:44:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 msgr2=0x558f824a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f824a1930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c02cce0 tx=0x7f8c3c02cd10 comp rx=0 tx=0).stop 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 shutdown_connections 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558f82499cd0 0x558f8249d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558f825100e0 0x558f82512500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f82490f00 0x558f8249cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 --2- 172.21.15.76:0/3403925605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f824905e0 0x558f824a1930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c27fff640 0 client.25681 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 >> 172.21.15.76:0/3403925605 conn(0x558f8248af00 msgr2=0x558f82489610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 shutdown_connections 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.595+0000 7f8c51982740 1 -- 172.21.15.76:0/3403925605 wait complete. 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c51982740 20 client.25681 trim_cache size 0 max 16384 2024-03-23T15:44:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:44:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:44:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:44:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:44:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8c380a2480 msgr2=0x7f8c380a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8c380a2480 0x7f8c380a4940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c006010 tx=0x7f8c3c005e40 comp rx=0 tx=0).stop 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 msgr2=0x7f8c48197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48197b10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000eb10 tx=0x7f8c4000c2d0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 shutdown_connections 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8c380a2480 0x7f8c380a4940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c48105590 0x7f8c48197b10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c48101200 0x7f8c4819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 --2- 172.21.15.76:0/3010113489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100860 0x7f8c4819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 >> 172.21.15.76:0/3010113489 conn(0x7f8c480773e0 msgr2=0x7f8c4806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 shutdown_connections 2024-03-23T15:44:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:40.596+0000 7f8c4ecd0640 1 -- 172.21.15.76:0/3010113489 wait complete. 2024-03-23T15:44:41.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:41 smithi183 ceph-mon[36296]: pgmap v43: 97 pgs: 97 active+clean; 74 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 572 op/s 2024-03-23T15:44:41.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:41 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:41.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:41 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:41.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:41 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:41.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:41 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:41.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-mon[36653]: pgmap v43: 97 pgs: 97 active+clean; 74 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 572 op/s 2024-03-23T15:44:41.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:41.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:41.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:41.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:41.820 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.326+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021b5 [...3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/netns/ auth{0=1} v194 ap=1 f(v0 m2024-03-23T15:44:30.296576+0000 34=34+0) n(v0 rc2024-03-23T15:44:30.297576+0000 35=34+1) (inest mix dirty) (ifile excl) caps={15327=pAsLsXsFsx/pAsLsXsFsx@2},l=15327 | dirtyscattered=1 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff8140680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000021b5:[<0x10000002181/netns v179>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(2)0x100000021b5:[<0x10000002181/netns v194>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-23T15:44:30.296576+0000 34=34+0)","ondisk_value.rstat":"n(v0 rc2024-03-23T15:44:30.296576+0000 35=34+1)","memory_value.dirstat":"f(v0 m2024-03-23T15:44:30.296576+0000 34=34+0)","memory_value.rstat":"n(v0 rc2024-03-23T15:44:30.297576+0000 35=34+1)","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.327+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021d8 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile.inc auth{0=1} v202 ap=1 s=1391 n(v0 rc2024-03-23T15:44:30.298576+0000 b1391 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 dirtyrstat=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff82f7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000021d8:[<0x10000002181/Makefile.inc v174>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000021d8:[<0x10000002181/Makefile.inc v202>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.327+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000218b [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/container_test.sh auth{0=1} v204 ap=1 s=1701 n(v0 rc2024-03-23T15:44:30.199579+0000 b1701 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 dirtyrstat=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff82f7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000218b:[<0x10000002181/container_test.sh v20>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000218b:[<0x10000002181/container_test.sh v204>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.327+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000218a [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile auth{0=1} v191 ap=1 s=1379 n(v0 rc2024-03-23T15:44:30.198579+0000 b1379 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x562ff80ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000218a:[<0x10000002181/Makefile v191>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000218a:[<0x10000002181/Makefile v191>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.328+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000218c [...3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/sysvipc/ auth{0=1} v195 ap=1 f(v0 m2024-03-23T15:44:30.207579+0000 9=9+0) n(v0 rc2024-03-23T15:44:30.208579+0000 10=9+1) (inest mix dirty) (ifile excl) caps={15327=pAsLsXsFsx/pAsLsXsFsx@2},l=15327 | dirtyscattered=1 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff8140c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000218c:[<0x10000002181/sysvipc v181>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(2)0x1000000218c:[<0x10000002181/sysvipc v195>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-23T15:44:30.207579+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-03-23T15:44:30.207579+0000 10=9+1)","memory_value.dirstat":"f(v0 m2024-03-23T15:44:30.207579+0000 9=9+0)","memory_value.rstat":"n(v0 rc2024-03-23T15:44:30.208579+0000 10=9+1)","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.329+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021b1 [...3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/libclone/ auth{0=1} v196 ap=1 f(v0 m2024-03-23T15:44:30.245578+0000 3=3+0) n(v0 rc2024-03-23T15:44:30.246578+0000 4=3+1) (inest mix dirty) (ifile excl) caps={15327=pAsLsXsFsx/pAsLsXsFsx@2},l=15327 | dirtyscattered=1 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff8141180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000021b1:[<0x10000002181/libclone v183>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(2)0x100000021b1:[<0x10000002181/libclone v196>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-23T15:44:30.245578+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-03-23T15:44:30.245578+0000 4=3+1)","memory_value.dirstat":"f(v0 m2024-03-23T15:44:30.245578+0000 3=3+0)","memory_value.rstat":"n(v0 rc2024-03-23T15:44:30.246578+0000 4=3+1)","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.330+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002197 [...3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/mqns/ auth{0=1} v197 ap=1 f(v0 m2024-03-23T15:44:30.216579+0000 7=7+0) n(v0 rc2024-03-23T15:44:30.216579+0000 8=7+1) (inest mix dirty) (ifile excl) caps={15327=pAsLsXsFsx/pAsLsXsFsx@2},l=15327 | dirtyscattered=1 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff8141700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002197:[<0x10000002181/mqns v185>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(2)0x10000002197:[<0x10000002181/mqns v197>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-23T15:44:30.216579+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-03-23T15:44:30.216579+0000 8=7+1)","memory_value.dirstat":"f(v0 m2024-03-23T15:44:30.216579+0000 7=7+0)","memory_value.rstat":"n(v0 rc2024-03-23T15:44:30.216579+0000 8=7+1)","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.334+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002189 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/TEST_PLAN.txt auth{0=1} v200 ap=1 s=1451 n(v0 rc2024-03-23T15:44:30.197579+0000 b1451 1=1+0)/n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 dirtyrstat=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff80ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002189:[<0x10000002181/TEST_PLAN.txt v16>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000002189:[<0x10000002181/TEST_PLAN.txt v200>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.336+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002182 [...3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/utsname/ auth{0=1} v198 ap=1 f(v0 m2024-03-23T15:44:30.195579+0000 6=6+0) n(v0 rc2024-03-23T15:44:30.196579+0000 7=6+1) (inest mix dirty) (ifile excl) caps={15327=pAsLsXsFsx/pAsLsXsFsx@2},l=15327 | dirtyscattered=1 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x562ff835c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000002182:[<0x10000002181/utsname v187>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(2)0x10000002182:[<0x10000002181/utsname v198>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-23T15:44:30.195579+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-03-23T15:44:30.195579+0000 7=6+1)","memory_value.dirstat":"f(v0 m2024-03-23T15:44:30.195579+0000 6=6+0)","memory_value.rstat":"n(v0 rc2024-03-23T15:44:30.196579+0000 7=6+1)","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.337+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002196 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/README auth{0=1} v189 ap=1 s=3286 n(v0 rc2024-03-23T15:44:30.209579+0000 b3286 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=1 authpin=1 scrubqueue=0 0x562ff80ceb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000002196:[<0x10000002181/README v189>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000002196:[<0x10000002181/README v189>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:41.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:41.349+0000 7fdcf2e06700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000219f [...3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/pidns/ auth{0=1,2=1} v206 ap=2 RANDEPHEMERALPIN f(v0 m2024-03-23T15:44:30.239578+0000 17=17+0) n(v1 rc2024-03-23T15:44:30.240578+0000 b81826 18=17+1) (inest sync->mix g=0,2) caps={15327=pAsLsXsFs/pAsLsXsFsx@2} | dirtyscattered=0 lock=0 importingcaps=0 dirfrag=1 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x562ff82f6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000219f:[<0x10000002181/pidns v177>,<0x10000001f2c/containers v458>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(2)0x1000000219f:[<0x10000002181/pidns v206>,<0x10000001f2c/containers v490>,<0x10000000204/kernel v643>,<0x10000000202/testcases v87>,<0x10000000007/ltp-full-20091231 v62>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v52>,<0x100000001fb/client.0 v50>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-23T15:44:30.239578+0000 17=17+0)","ondisk_value.rstat":"n(v0 rc2024-03-23T15:44:30.240578+0000 b81826 18=17+1)","memory_value.dirstat":"f(v0 m2024-03-23T15:44:30.239578+0000 17=17+0)","memory_value.rstat":"n(v1 rc2024-03-23T15:44:30.240578+0000 b81826 18=17+1)","error_str":""},"return_code":0} 2024-03-23T15:44:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:41 smithi076 ceph-mon[32176]: pgmap v43: 97 pgs: 97 active+clean; 74 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 572 op/s 2024-03-23T15:44:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:41 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:41 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:41 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:41 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000021b5 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/netns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000021d8 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile.inc) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x1000000218b (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/container_test.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x1000000218a (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x1000000218c (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/sysvipc) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000021b1 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/libclone) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x10000002197 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/mqns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x10000002189 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/TEST_PLAN.txt) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x10000002182 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/utsname) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x10000002196 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/README) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: Scrub error on inode 0x1000000219f (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/pidns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:42.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:42 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:42.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000021b5 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/netns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000021d8 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile.inc) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x1000000218b (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/container_test.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x1000000218a (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x1000000218c (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/sysvipc) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000021b1 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/libclone) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x10000002197 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/mqns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x10000002189 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/TEST_PLAN.txt) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x10000002182 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/utsname) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x10000002196 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/README) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: Scrub error on inode 0x1000000219f (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/pidns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:42 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000021b5 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/netns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000021d8 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile.inc) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x1000000218b (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/container_test.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x1000000218a (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x1000000218c (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/sysvipc) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000021b1 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/libclone) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x10000002197 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/mqns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x10000002189 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/TEST_PLAN.txt) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x10000002182 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/utsname) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x10000002196 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/README) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: Scrub error on inode 0x1000000219f (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/pidns) see mds.i log and `damage ls` output for details 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:42.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:42.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:42.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:42 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: pgmap v44: 97 pgs: 97 active+clean; 64 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 1.68k op/s 2024-03-23T15:44:43.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:43.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:43.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:43.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:43.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:43.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:43 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: pgmap v44: 97 pgs: 97 active+clean; 64 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 1.68k op/s 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:43.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:43 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: pgmap v44: 97 pgs: 97 active+clean; 64 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 1.68k op/s 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:43 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.020 INFO:tasks.workunit.client.0.smithi076.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-23T15:44:44.020 INFO:tasks.workunit.client.0.smithi076.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-23T15:44:44.021 INFO:tasks.workunit.client.0.smithi076.stderr:+ make 2024-03-23T15:44:44.120 INFO:tasks.workunit.client.0.smithi076.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-23T15:44:44.378 INFO:tasks.workunit.client.0.smithi076.stderr:++ readlink -f fsstress 2024-03-23T15:44:44.379 INFO:tasks.workunit.client.0.smithi076.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-23T15:44:44.379 INFO:tasks.workunit.client.0.smithi076.stderr:+ popd 2024-03-23T15:44:44.379 INFO:tasks.workunit.client.0.smithi076.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-23T15:44:44.380 INFO:tasks.workunit.client.0.smithi076.stderr:+ popd 2024-03-23T15:44:44.380 INFO:tasks.workunit.client.0.smithi076.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-23T15:44:44.380 INFO:tasks.workunit.client.0.smithi076.stderr:++ mktemp -d -p . 2024-03-23T15:44:44.381 INFO:tasks.workunit.client.0.smithi076.stderr:+ T=./tmp.qNXWRIvOo3 2024-03-23T15:44:44.381 INFO:tasks.workunit.client.0.smithi076.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.qNXWRIvOo3 -l 1 -n 1000 -p 10 -v 2024-03-23T15:44:44.382 INFO:tasks.workunit.client.0.smithi076.stdout:seed = 1710959105 2024-03-23T15:44:44.386 INFO:tasks.workunit.client.0.smithi076.stdout:0/0: stat - no entries 2024-03-23T15:44:44.386 INFO:tasks.workunit.client.0.smithi076.stdout:0/1: rename - no filename 2024-03-23T15:44:44.386 INFO:tasks.workunit.client.0.smithi076.stdout:0/2: dwrite - no filename 2024-03-23T15:44:44.386 INFO:tasks.workunit.client.0.smithi076.stdout:0/3: truncate - no filename 2024-03-23T15:44:44.386 INFO:tasks.workunit.client.0.smithi076.stdout:0/4: dread - no filename 2024-03-23T15:44:44.386 INFO:tasks.workunit.client.0.smithi076.stdout:1/0: dwrite - no filename 2024-03-23T15:44:44.387 INFO:tasks.workunit.client.0.smithi076.stdout:0/5: mknod c0 0 2024-03-23T15:44:44.387 INFO:tasks.workunit.client.0.smithi076.stdout:0/6: write - no filename 2024-03-23T15:44:44.389 INFO:tasks.workunit.client.0.smithi076.stdout:2/0: dread - no filename 2024-03-23T15:44:44.389 INFO:tasks.workunit.client.0.smithi076.stdout:2/1: write - no filename 2024-03-23T15:44:44.389 INFO:tasks.workunit.client.0.smithi076.stdout:2/2: write - no filename 2024-03-23T15:44:44.389 INFO:tasks.workunit.client.0.smithi076.stdout:2/3: write - no filename 2024-03-23T15:44:44.390 INFO:tasks.workunit.client.0.smithi076.stdout:1/1: mkdir d0 0 2024-03-23T15:44:44.390 INFO:tasks.workunit.client.0.smithi076.stdout:1/2: dread - no filename 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/0: chown . 5562586 1 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/1: dread - no filename 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/2: rename - no filename 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/3: read - no filename 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/4: rename - no filename 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/5: link - no file 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/6: fdatasync - no filename 2024-03-23T15:44:44.392 INFO:tasks.workunit.client.0.smithi076.stdout:3/7: link - no file 2024-03-23T15:44:44.393 INFO:tasks.workunit.client.0.smithi076.stdout:2/4: symlink l0 0 2024-03-23T15:44:44.393 INFO:tasks.workunit.client.0.smithi076.stdout:2/5: rmdir - no directory 2024-03-23T15:44:44.393 INFO:tasks.workunit.client.0.smithi076.stdout:2/6: fdatasync - no filename 2024-03-23T15:44:44.394 INFO:tasks.workunit.client.0.smithi076.stdout:4/0: stat - no entries 2024-03-23T15:44:44.394 INFO:tasks.workunit.client.0.smithi076.stdout:4/1: write - no filename 2024-03-23T15:44:44.394 INFO:tasks.workunit.client.0.smithi076.stdout:4/2: chown . 154786 1 2024-03-23T15:44:44.394 INFO:tasks.workunit.client.0.smithi076.stdout:3/8: mkdir d0 0 2024-03-23T15:44:44.394 INFO:tasks.workunit.client.0.smithi076.stdout:3/9: chown d0 196 1 2024-03-23T15:44:44.394 INFO:tasks.workunit.client.0.smithi076.stdout:3/10: dwrite - no filename 2024-03-23T15:44:44.395 INFO:tasks.workunit.client.0.smithi076.stdout:3/11: stat d0 0 2024-03-23T15:44:44.396 INFO:tasks.workunit.client.0.smithi076.stdout:2/7: mkdir d1 0 2024-03-23T15:44:44.396 INFO:tasks.workunit.client.0.smithi076.stdout:2/8: truncate - no filename 2024-03-23T15:44:44.396 INFO:tasks.workunit.client.0.smithi076.stdout:2/9: fdatasync - no filename 2024-03-23T15:44:44.396 INFO:tasks.workunit.client.0.smithi076.stdout:2/10: read - no filename 2024-03-23T15:44:44.396 INFO:tasks.workunit.client.0.smithi076.stdout:2/11: write - no filename 2024-03-23T15:44:44.396 INFO:tasks.workunit.client.0.smithi076.stdout:2/12: dwrite - no filename 2024-03-23T15:44:44.398 INFO:tasks.workunit.client.0.smithi076.stdout:4/3: symlink l0 0 2024-03-23T15:44:44.402 INFO:tasks.workunit.client.0.smithi076.stdout:3/12: creat d0/f1 x:0 0 0 2024-03-23T15:44:44.404 INFO:tasks.workunit.client.0.smithi076.stdout:2/13: link l0 d1/l2 0 2024-03-23T15:44:44.411 INFO:tasks.workunit.client.0.smithi076.stdout:5/0: creat f0 x:0 0 0 2024-03-23T15:44:44.425 INFO:tasks.workunit.client.0.smithi076.stdout:0/7: getdents . 0 2024-03-23T15:44:44.425 INFO:tasks.workunit.client.0.smithi076.stdout:0/8: write - no filename 2024-03-23T15:44:44.427 INFO:tasks.workunit.client.0.smithi076.stdout:4/4: rename l0 to l1 0 2024-03-23T15:44:44.429 INFO:tasks.workunit.client.0.smithi076.stdout:3/13: unlink d0/f1 0 2024-03-23T15:44:44.430 INFO:tasks.workunit.client.0.smithi076.stdout:3/14: write - no filename 2024-03-23T15:44:44.430 INFO:tasks.workunit.client.0.smithi076.stdout:3/15: dread - no filename 2024-03-23T15:44:44.432 INFO:tasks.workunit.client.0.smithi076.stdout:2/14: mknod d1/c3 0 2024-03-23T15:44:44.435 INFO:tasks.workunit.client.0.smithi076.stdout:5/1: unlink f0 0 2024-03-23T15:44:44.435 INFO:tasks.workunit.client.0.smithi076.stdout:5/2: dread - no filename 2024-03-23T15:44:44.447 INFO:tasks.workunit.client.0.smithi076.stdout:3/16: mknod d0/c2 0 2024-03-23T15:44:44.447 INFO:tasks.workunit.client.0.smithi076.stdout:4/5: creat f2 x:0 0 0 2024-03-23T15:44:44.447 INFO:tasks.workunit.client.0.smithi076.stdout:0/9: creat f1 x:0 0 0 2024-03-23T15:44:44.447 INFO:tasks.workunit.client.0.smithi076.stdout:0/10: read - f1 zero size 2024-03-23T15:44:44.448 INFO:tasks.workunit.client.0.smithi076.stdout:4/6: write f2 [135391,96434] 0 2024-03-23T15:44:44.448 INFO:tasks.workunit.client.0.smithi076.stdout:4/7: write f2 [734251,69123] 0 2024-03-23T15:44:44.448 INFO:tasks.workunit.client.0.smithi076.stdout:2/15: mkdir d1/d4 0 2024-03-23T15:44:44.448 INFO:tasks.workunit.client.0.smithi076.stdout:2/16: readlink l0 0 2024-03-23T15:44:44.448 INFO:tasks.workunit.client.0.smithi076.stdout:2/17: dread - no filename 2024-03-23T15:44:44.448 INFO:tasks.workunit.client.0.smithi076.stdout:2/18: read - no filename 2024-03-23T15:44:44.448 INFO:tasks.workunit.client.0.smithi076.stdout:2/19: write - no filename 2024-03-23T15:44:44.460 INFO:tasks.workunit.client.0.smithi076.stdout:3/17: unlink d0/c2 0 2024-03-23T15:44:44.460 INFO:tasks.workunit.client.0.smithi076.stdout:3/18: dwrite - no filename 2024-03-23T15:44:44.460 INFO:tasks.workunit.client.0.smithi076.stdout:0/11: mkdir d2 0 2024-03-23T15:44:44.460 INFO:tasks.workunit.client.0.smithi076.stdout:2/20: creat d1/f5 x:0 0 0 2024-03-23T15:44:44.464 INFO:tasks.workunit.client.0.smithi076.stdout:0/12: symlink d2/l3 0 2024-03-23T15:44:44.465 INFO:tasks.workunit.client.0.smithi076.stdout:2/21: creat d1/d4/f6 x:0 0 0 2024-03-23T15:44:44.465 INFO:tasks.workunit.client.0.smithi076.stdout:2/22: dread - d1/d4/f6 zero size 2024-03-23T15:44:44.471 INFO:tasks.workunit.client.0.smithi076.stdout:0/13: rename f1 to d2/f4 0 2024-03-23T15:44:44.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:44.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:44 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.642 INFO:tasks.workunit.client.0.smithi076.stdout:0/14: dwrite d2/f4 [0,4194304] 0 2024-03-23T15:44:44.643 INFO:tasks.workunit.client.0.smithi076.stdout:0/15: chown c0 2065813197 1 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:5/3: write - no filename 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:5/4: dread - no filename 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:5/5: unlink - no file 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:5/6: stat - no entries 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:6/0: dwrite - no filename 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:6/1: dread - no filename 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:6/2: chown . 155971644 1 2024-03-23T15:44:44.699 INFO:tasks.workunit.client.0.smithi076.stdout:6/3: dread - no filename 2024-03-23T15:44:44.700 INFO:tasks.workunit.client.0.smithi076.stdout:6/4: dwrite - no filename 2024-03-23T15:44:44.700 INFO:tasks.workunit.client.0.smithi076.stdout:6/5: read - no filename 2024-03-23T15:44:44.700 INFO:tasks.workunit.client.0.smithi076.stdout:6/6: chown . 46136523 1 2024-03-23T15:44:44.700 INFO:tasks.workunit.client.0.smithi076.stdout:6/7: chown . 3 1 2024-03-23T15:44:44.700 INFO:tasks.workunit.client.0.smithi076.stdout:7/0: truncate - no filename 2024-03-23T15:44:44.745 INFO:tasks.workunit.client.0.smithi076.stdout:4/8: dwrite f2 [0,4194304] 0 2024-03-23T15:44:44.813 INFO:tasks.workunit.client.0.smithi076.stdout:2/23: fsync d1/d4/f6 0 2024-03-23T15:44:44.814 INFO:tasks.workunit.client.0.smithi076.stdout:2/24: read - d1/f5 zero size 2024-03-23T15:44:44.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:44 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:44.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:44.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:44 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.349 INFO:tasks.workunit.client.0.smithi076.stdout:0/16: link c0 d2/c5 0 2024-03-23T15:44:45.353 INFO:tasks.workunit.client.0.smithi076.stdout:0/17: write d2/f4 [1136262,76122] 0 2024-03-23T15:44:45.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: pgmap v45: 97 pgs: 97 active+clean; 64 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 1.66k op/s 2024-03-23T15:44:45.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:45.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:45 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.757 INFO:tasks.workunit.client.0.smithi076.stdout:5/7: creat f1 x:0 0 0 2024-03-23T15:44:45.759 INFO:tasks.workunit.client.0.smithi076.stdout:6/8: mknod c0 0 2024-03-23T15:44:45.760 INFO:tasks.workunit.client.0.smithi076.stdout:9/0: mknod c0 0 2024-03-23T15:44:45.760 INFO:tasks.workunit.client.0.smithi076.stdout:9/1: dwrite - no filename 2024-03-23T15:44:45.760 INFO:tasks.workunit.client.0.smithi076.stdout:9/2: read - no filename 2024-03-23T15:44:45.761 INFO:tasks.workunit.client.0.smithi076.stdout:9/3: write - no filename 2024-03-23T15:44:45.761 INFO:tasks.workunit.client.0.smithi076.stdout:9/4: truncate - no filename 2024-03-23T15:44:45.761 INFO:tasks.workunit.client.0.smithi076.stdout:9/5: write - no filename 2024-03-23T15:44:45.761 INFO:tasks.workunit.client.0.smithi076.stdout:9/6: dwrite - no filename 2024-03-23T15:44:45.761 INFO:tasks.workunit.client.0.smithi076.stdout:9/7: readlink - no filename 2024-03-23T15:44:45.761 INFO:tasks.workunit.client.0.smithi076.stdout:9/8: truncate - no filename 2024-03-23T15:44:45.768 INFO:tasks.workunit.client.0.smithi076.stdout:7/1: creat f0 x:0 0 0 2024-03-23T15:44:45.768 INFO:tasks.workunit.client.0.smithi076.stdout:7/2: truncate f0 548200 0 2024-03-23T15:44:45.768 INFO:tasks.workunit.client.0.smithi076.stdout:7/3: chown f0 1 1 2024-03-23T15:44:45.769 INFO:tasks.workunit.client.0.smithi076.stdout:7/4: fdatasync f0 0 2024-03-23T15:44:45.769 INFO:tasks.workunit.client.0.smithi076.stdout:7/5: read f0 [163196,36295] 0 2024-03-23T15:44:45.769 INFO:tasks.workunit.client.0.smithi076.stdout:7/6: write f0 [792295,106884] 0 2024-03-23T15:44:45.769 INFO:tasks.workunit.client.0.smithi076.stdout:7/7: truncate f0 1300703 0 2024-03-23T15:44:45.774 INFO:tasks.workunit.client.0.smithi076.stdout:7/8: dread f0 [0,4194304] 0 2024-03-23T15:44:45.774 INFO:tasks.workunit.client.0.smithi076.stdout:7/9: rmdir - no directory 2024-03-23T15:44:45.776 INFO:tasks.workunit.client.0.smithi076.stdout:7/10: dread f0 [0,4194304] 0 2024-03-23T15:44:45.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: pgmap v45: 97 pgs: 97 active+clean; 64 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 1.66k op/s 2024-03-23T15:44:45.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:45.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:45 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:44:45.375+0000 7fa1f8f14700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: pgmap v45: 97 pgs: 97 active+clean; 64 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 1.66k op/s 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:45.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:45 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:45.933 INFO:tasks.workunit.client.0.smithi076.stdout:4/9: rename l1 to l3 0 2024-03-23T15:44:46.421 INFO:tasks.workunit.client.0.smithi076.stdout:5/8: fsync f1 0 2024-03-23T15:44:46.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:46 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:46.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:46 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:46.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:46 smithi183 ceph-mon[36296]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:46.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:46 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:active 2024-03-23T15:44:46.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:46 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:46 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:46 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:46.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:46 smithi192 ceph-mon[36653]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:46.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:46 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:active 2024-03-23T15:44:46.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:46 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:46 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:46 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:46 smithi076 ceph-mon[32176]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:46.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:46 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] up:active 2024-03-23T15:44:46.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:46 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:47.062 INFO:tasks.workunit.client.0.smithi076.stdout:6/9: mkdir d1 0 2024-03-23T15:44:47.063 INFO:tasks.workunit.client.0.smithi076.stdout:9/9: mkdir d1 0 2024-03-23T15:44:47.065 INFO:tasks.workunit.client.0.smithi076.stdout:4/10: creat f4 x:0 0 0 2024-03-23T15:44:47.065 INFO:tasks.workunit.client.0.smithi076.stdout:4/11: stat f4 0 2024-03-23T15:44:47.066 INFO:tasks.workunit.client.0.smithi076.stdout:0/18: truncate d2/f4 2673007 0 2024-03-23T15:44:47.068 INFO:tasks.workunit.client.0.smithi076.stdout:5/9: rename f1 to f2 0 2024-03-23T15:44:47.069 INFO:tasks.workunit.client.0.smithi076.stdout:6/10: mkdir d1/d2 0 2024-03-23T15:44:47.070 INFO:tasks.workunit.client.0.smithi076.stdout:9/10: mknod d1/c2 0 2024-03-23T15:44:47.071 INFO:tasks.workunit.client.0.smithi076.stdout:7/11: link f0 f1 0 2024-03-23T15:44:47.072 INFO:tasks.workunit.client.0.smithi076.stdout:6/11: symlink d1/l3 0 2024-03-23T15:44:47.074 INFO:tasks.workunit.client.0.smithi076.stdout:9/11: creat d1/f3 x:0 0 0 2024-03-23T15:44:47.074 INFO:tasks.workunit.client.0.smithi076.stdout:9/12: write d1/f3 [89576,88489] 0 2024-03-23T15:44:47.074 INFO:tasks.workunit.client.0.smithi076.stdout:4/12: write f2 [4184047,102443] 0 2024-03-23T15:44:47.076 INFO:tasks.workunit.client.0.smithi076.stdout:9/13: unlink d1/c2 0 2024-03-23T15:44:47.076 INFO:tasks.workunit.client.0.smithi076.stdout:9/14: truncate d1/f3 979094 0 2024-03-23T15:44:47.078 INFO:tasks.workunit.client.0.smithi076.stdout:4/13: creat f5 x:0 0 0 2024-03-23T15:44:47.079 INFO:tasks.workunit.client.0.smithi076.stdout:4/14: write f5 [570028,24900] 0 2024-03-23T15:44:47.079 INFO:tasks.workunit.client.0.smithi076.stdout:9/15: fdatasync d1/f3 0 2024-03-23T15:44:47.079 INFO:tasks.workunit.client.0.smithi076.stdout:9/16: truncate d1/f3 1123013 0 2024-03-23T15:44:47.084 INFO:tasks.workunit.client.0.smithi076.stdout:5/10: dwrite f2 [0,4194304] 0 2024-03-23T15:44:47.092 INFO:tasks.workunit.client.0.smithi076.stdout:9/17: dwrite d1/f3 [0,4194304] 0 2024-03-23T15:44:47.103 INFO:tasks.workunit.client.0.smithi076.stdout:4/15: dwrite f5 [0,4194304] 0 2024-03-23T15:44:47.105 INFO:tasks.workunit.client.0.smithi076.stdout:4/16: rename f4 to f6 0 2024-03-23T15:44:47.108 INFO:tasks.workunit.client.0.smithi076.stdout:4/17: symlink l7 0 2024-03-23T15:44:47.110 INFO:tasks.workunit.client.0.smithi076.stdout:4/18: mkdir d8 0 2024-03-23T15:44:47.111 INFO:tasks.workunit.client.0.smithi076.stdout:4/19: mkdir d8/d9 0 2024-03-23T15:44:47.113 INFO:tasks.workunit.client.0.smithi076.stdout:4/20: mknod d8/d9/ca 0 2024-03-23T15:44:47.113 INFO:tasks.workunit.client.0.smithi076.stdout:4/21: chown l3 1414 1 2024-03-23T15:44:47.113 INFO:tasks.workunit.client.0.smithi076.stdout:4/22: rmdir d8/d9 39 2024-03-23T15:44:47.130 INFO:tasks.workunit.client.0.smithi076.stdout:4/23: dwrite f2 [0,4194304] 0 2024-03-23T15:44:47.131 INFO:tasks.workunit.client.0.smithi076.stdout:4/24: chown f6 116770831 1 2024-03-23T15:44:47.131 INFO:tasks.workunit.client.0.smithi076.stdout:4/25: stat l3 0 2024-03-23T15:44:47.131 INFO:tasks.workunit.client.0.smithi076.stdout:4/26: dread - f6 zero size 2024-03-23T15:44:47.305 INFO:tasks.workunit.client.0.smithi076.stdout:7/12: truncate f1 172694 0 2024-03-23T15:44:47.306 INFO:tasks.workunit.client.0.smithi076.stdout:7/13: creat f2 x:0 0 0 2024-03-23T15:44:47.307 INFO:tasks.workunit.client.0.smithi076.stdout:1/3: sync 2024-03-23T15:44:47.307 INFO:tasks.workunit.client.0.smithi076.stdout:3/19: sync 2024-03-23T15:44:47.307 INFO:tasks.workunit.client.0.smithi076.stdout:2/25: sync 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:3/20: chown d0 169 1 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:8/0: sync 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:8/1: dread - no filename 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:8/2: chown . 3801 1 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:8/3: readlink - no filename 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:8/4: write - no filename 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:2/26: write d1/d4/f6 [73883,75594] 0 2024-03-23T15:44:47.308 INFO:tasks.workunit.client.0.smithi076.stdout:7/14: rename f2 to f3 0 2024-03-23T15:44:47.311 INFO:tasks.workunit.client.0.smithi076.stdout:8/5: creat f0 x:0 0 0 2024-03-23T15:44:47.311 INFO:tasks.workunit.client.0.smithi076.stdout:8/6: dread - f0 zero size 2024-03-23T15:44:47.313 INFO:tasks.workunit.client.0.smithi076.stdout:2/27: creat d1/d4/f7 x:0 0 0 2024-03-23T15:44:47.313 INFO:tasks.workunit.client.0.smithi076.stdout:2/28: write d1/d4/f7 [816184,108375] 0 2024-03-23T15:44:47.315 INFO:tasks.workunit.client.0.smithi076.stdout:0/19: dwrite d2/f4 [0,4194304] 0 2024-03-23T15:44:47.315 INFO:tasks.workunit.client.0.smithi076.stdout:5/11: truncate f2 3162177 0 2024-03-23T15:44:47.315 INFO:tasks.workunit.client.0.smithi076.stdout:5/12: rmdir - no directory 2024-03-23T15:44:47.319 INFO:tasks.workunit.client.0.smithi076.stdout:0/20: creat d2/f6 x:0 0 0 2024-03-23T15:44:47.321 INFO:tasks.workunit.client.0.smithi076.stdout:3/21: rmdir d0 0 2024-03-23T15:44:47.324 INFO:tasks.workunit.client.0.smithi076.stdout:0/21: creat d2/f7 x:0 0 0 2024-03-23T15:44:47.324 INFO:tasks.workunit.client.0.smithi076.stdout:0/22: fdatasync d2/f4 0 2024-03-23T15:44:47.324 INFO:tasks.workunit.client.0.smithi076.stdout:0/23: write d2/f6 [792693,6036] 0 2024-03-23T15:44:47.325 INFO:tasks.workunit.client.0.smithi076.stdout:9/18: truncate d1/f3 3042404 0 2024-03-23T15:44:47.325 INFO:tasks.workunit.client.0.smithi076.stdout:4/27: write f5 [4346334,124226] 0 2024-03-23T15:44:47.330 INFO:tasks.workunit.client.0.smithi076.stdout:0/24: dread d2/f4 [0,4194304] 0 2024-03-23T15:44:47.332 INFO:tasks.workunit.client.0.smithi076.stdout:4/28: truncate f2 3377808 0 2024-03-23T15:44:47.334 INFO:tasks.workunit.client.0.smithi076.stdout:7/15: link f0 f4 0 2024-03-23T15:44:47.335 INFO:tasks.workunit.client.0.smithi076.stdout:8/7: getdents . 0 2024-03-23T15:44:47.336 INFO:tasks.workunit.client.0.smithi076.stdout:8/8: symlink l1 0 2024-03-23T15:44:47.338 INFO:tasks.workunit.client.0.smithi076.stdout:8/9: link f0 f2 0 2024-03-23T15:44:47.341 INFO:tasks.workunit.client.0.smithi076.stdout:8/10: creat f3 x:0 0 0 2024-03-23T15:44:47.341 INFO:tasks.workunit.client.0.smithi076.stdout:8/11: dread - f2 zero size 2024-03-23T15:44:47.343 INFO:tasks.workunit.client.0.smithi076.stdout:2/29: truncate d1/d4/f7 180491 0 2024-03-23T15:44:47.344 INFO:tasks.workunit.client.0.smithi076.stdout:8/12: symlink l4 0 2024-03-23T15:44:47.346 INFO:tasks.workunit.client.0.smithi076.stdout:5/13: dread f2 [0,4194304] 0 2024-03-23T15:44:47.346 INFO:tasks.workunit.client.0.smithi076.stdout:2/30: link l0 d1/d4/l8 0 2024-03-23T15:44:47.348 INFO:tasks.workunit.client.0.smithi076.stdout:5/14: symlink l3 0 2024-03-23T15:44:47.349 INFO:tasks.workunit.client.0.smithi076.stdout:3/22: getdents . 0 2024-03-23T15:44:47.349 INFO:tasks.workunit.client.0.smithi076.stdout:3/23: dread - no filename 2024-03-23T15:44:47.350 INFO:tasks.workunit.client.0.smithi076.stdout:3/24: write - no filename 2024-03-23T15:44:47.350 INFO:tasks.workunit.client.0.smithi076.stdout:3/25: dread - no filename 2024-03-23T15:44:47.350 INFO:tasks.workunit.client.0.smithi076.stdout:3/26: chown . 434207724 1 2024-03-23T15:44:47.351 INFO:tasks.workunit.client.0.smithi076.stdout:5/15: creat f4 x:0 0 0 2024-03-23T15:44:47.353 INFO:tasks.workunit.client.0.smithi076.stdout:7/16: dwrite f4 [0,4194304] 0 2024-03-23T15:44:47.354 INFO:tasks.workunit.client.0.smithi076.stdout:3/27: creat f3 x:0 0 0 2024-03-23T15:44:47.354 INFO:tasks.workunit.client.0.smithi076.stdout:3/28: chown f3 100157407 1 2024-03-23T15:44:47.354 INFO:tasks.workunit.client.0.smithi076.stdout:3/29: read - f3 zero size 2024-03-23T15:44:47.354 INFO:tasks.workunit.client.0.smithi076.stdout:3/30: fdatasync f3 0 2024-03-23T15:44:47.355 INFO:tasks.workunit.client.0.smithi076.stdout:5/16: creat f5 x:0 0 0 2024-03-23T15:44:47.360 INFO:tasks.workunit.client.0.smithi076.stdout:6/12: sync 2024-03-23T15:44:47.360 INFO:tasks.workunit.client.0.smithi076.stdout:0/25: dwrite d2/f6 [0,4194304] 0 2024-03-23T15:44:47.360 INFO:tasks.workunit.client.0.smithi076.stdout:6/13: dread - no filename 2024-03-23T15:44:47.362 INFO:tasks.workunit.client.0.smithi076.stdout:5/17: link f2 f6 0 2024-03-23T15:44:47.362 INFO:tasks.workunit.client.0.smithi076.stdout:5/18: read - f4 zero size 2024-03-23T15:44:47.363 INFO:tasks.workunit.client.0.smithi076.stdout:8/13: dwrite f0 [0,4194304] 0 2024-03-23T15:44:47.365 INFO:tasks.workunit.client.0.smithi076.stdout:0/26: mkdir d2/d8 0 2024-03-23T15:44:47.366 INFO:tasks.workunit.client.0.smithi076.stdout:6/14: mkdir d1/d4 0 2024-03-23T15:44:47.368 INFO:tasks.workunit.client.0.smithi076.stdout:3/31: link f3 f4 0 2024-03-23T15:44:47.368 INFO:tasks.workunit.client.0.smithi076.stdout:3/32: dread - f3 zero size 2024-03-23T15:44:47.368 INFO:tasks.workunit.client.0.smithi076.stdout:3/33: write f4 [476558,80032] 0 2024-03-23T15:44:47.369 INFO:tasks.workunit.client.0.smithi076.stdout:2/31: dwrite d1/d4/f6 [0,4194304] 0 2024-03-23T15:44:47.371 INFO:tasks.workunit.client.0.smithi076.stdout:5/19: rename f2 to f7 0 2024-03-23T15:44:47.371 INFO:tasks.workunit.client.0.smithi076.stdout:5/20: chown l3 4 1 2024-03-23T15:44:47.372 INFO:tasks.workunit.client.0.smithi076.stdout:3/34: dread f3 [0,4194304] 0 2024-03-23T15:44:47.372 INFO:tasks.workunit.client.0.smithi076.stdout:3/35: chown f4 224 1 2024-03-23T15:44:47.377 INFO:tasks.workunit.client.0.smithi076.stdout:7/17: dwrite f3 [0,4194304] 0 2024-03-23T15:44:47.377 INFO:tasks.workunit.client.0.smithi076.stdout:7/18: stat f3 0 2024-03-23T15:44:47.378 INFO:tasks.workunit.client.0.smithi076.stdout:4/29: dread f2 [0,4194304] 0 2024-03-23T15:44:47.378 INFO:tasks.workunit.client.0.smithi076.stdout:4/30: rename d8 to d8/d9/db 22 2024-03-23T15:44:47.378 INFO:tasks.workunit.client.0.smithi076.stdout:9/19: dwrite d1/f3 [0,4194304] 0 2024-03-23T15:44:47.378 INFO:tasks.workunit.client.0.smithi076.stdout:0/27: creat d2/f9 x:0 0 0 2024-03-23T15:44:47.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/15: creat d1/d4/f5 x:0 0 0 2024-03-23T15:44:47.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/16: write d1/d4/f5 [960080,117012] 0 2024-03-23T15:44:47.383 INFO:tasks.workunit.client.0.smithi076.stdout:6/17: fdatasync d1/d4/f5 0 2024-03-23T15:44:47.384 INFO:tasks.workunit.client.0.smithi076.stdout:2/32: creat d1/d4/f9 x:0 0 0 2024-03-23T15:44:47.385 INFO:tasks.workunit.client.0.smithi076.stdout:5/21: creat f8 x:0 0 0 2024-03-23T15:44:47.386 INFO:tasks.workunit.client.0.smithi076.stdout:4/31: write f5 [447928,13214] 0 2024-03-23T15:44:47.386 INFO:tasks.workunit.client.0.smithi076.stdout:4/32: dread - f6 zero size 2024-03-23T15:44:47.387 INFO:tasks.workunit.client.0.smithi076.stdout:6/18: write d1/d4/f5 [936458,88166] 0 2024-03-23T15:44:47.387 INFO:tasks.workunit.client.0.smithi076.stdout:6/19: write d1/d4/f5 [1462902,4553] 0 2024-03-23T15:44:47.387 INFO:tasks.workunit.client.0.smithi076.stdout:6/20: chown d1/d4 1921495 1 2024-03-23T15:44:47.389 INFO:tasks.workunit.client.0.smithi076.stdout:8/14: symlink l5 0 2024-03-23T15:44:47.390 INFO:tasks.workunit.client.0.smithi076.stdout:7/19: dwrite f1 [0,4194304] 0 2024-03-23T15:44:47.393 INFO:tasks.workunit.client.0.smithi076.stdout:4/33: dread f5 [0,4194304] 0 2024-03-23T15:44:47.394 INFO:tasks.workunit.client.0.smithi076.stdout:9/20: link d1/f3 d1/f4 0 2024-03-23T15:44:47.394 INFO:tasks.workunit.client.0.smithi076.stdout:9/21: chown d1/f4 26315 1 2024-03-23T15:44:47.397 INFO:tasks.workunit.client.0.smithi076.stdout:2/33: mknod d1/ca 0 2024-03-23T15:44:47.397 INFO:tasks.workunit.client.0.smithi076.stdout:2/34: chown d1/d4/f6 221210876 1 2024-03-23T15:44:47.398 INFO:tasks.workunit.client.0.smithi076.stdout:5/22: rename f8 to f9 0 2024-03-23T15:44:47.398 INFO:tasks.workunit.client.0.smithi076.stdout:3/36: mkdir d5 0 2024-03-23T15:44:47.401 INFO:tasks.workunit.client.0.smithi076.stdout:6/21: rename c0 to d1/d4/c6 0 2024-03-23T15:44:47.401 INFO:tasks.workunit.client.0.smithi076.stdout:4/34: getdents d8/d9 0 2024-03-23T15:44:47.403 INFO:tasks.workunit.client.0.smithi076.stdout:8/15: dwrite f0 [0,4194304] 0 2024-03-23T15:44:47.403 INFO:tasks.workunit.client.0.smithi076.stdout:7/20: dread f4 [0,4194304] 0 2024-03-23T15:44:47.405 INFO:tasks.workunit.client.0.smithi076.stdout:2/35: mkdir d1/d4/db 0 2024-03-23T15:44:47.407 INFO:tasks.workunit.client.0.smithi076.stdout:9/22: rename c0 to d1/c5 0 2024-03-23T15:44:47.408 INFO:tasks.workunit.client.0.smithi076.stdout:6/22: creat d1/d2/f7 x:0 0 0 2024-03-23T15:44:47.409 INFO:tasks.workunit.client.0.smithi076.stdout:4/35: rmdir d8 39 2024-03-23T15:44:47.411 INFO:tasks.workunit.client.0.smithi076.stdout:4/36: dread f5 [4194304,4194304] 0 2024-03-23T15:44:47.413 INFO:tasks.workunit.client.0.smithi076.stdout:4/37: write f5 [4290258,65540] 0 2024-03-23T15:44:47.413 INFO:tasks.workunit.client.0.smithi076.stdout:4/38: fsync f2 0 2024-03-23T15:44:47.413 INFO:tasks.workunit.client.0.smithi076.stdout:4/39: write f5 [4184834,93863] 0 2024-03-23T15:44:47.413 INFO:tasks.workunit.client.0.smithi076.stdout:4/40: dread - f6 zero size 2024-03-23T15:44:47.414 INFO:tasks.workunit.client.0.smithi076.stdout:7/21: rename f4 to f5 0 2024-03-23T15:44:47.414 INFO:tasks.workunit.client.0.smithi076.stdout:7/22: chown f5 8593796 1 2024-03-23T15:44:47.414 INFO:tasks.workunit.client.0.smithi076.stdout:5/23: dwrite f9 [0,4194304] 0 2024-03-23T15:44:47.415 INFO:tasks.workunit.client.0.smithi076.stdout:2/36: creat d1/d4/db/fc x:0 0 0 2024-03-23T15:44:47.415 INFO:tasks.workunit.client.0.smithi076.stdout:2/37: truncate d1/d4/db/fc 548196 0 2024-03-23T15:44:47.416 INFO:tasks.workunit.client.0.smithi076.stdout:3/37: rename f3 to d5/f6 0 2024-03-23T15:44:47.417 INFO:tasks.workunit.client.0.smithi076.stdout:3/38: stat f4 0 2024-03-23T15:44:47.417 INFO:tasks.workunit.client.0.smithi076.stdout:9/23: read d1/f4 [3351484,113343] 0 2024-03-23T15:44:47.417 INFO:tasks.workunit.client.0.smithi076.stdout:9/24: write d1/f4 [1084592,25706] 0 2024-03-23T15:44:47.418 INFO:tasks.workunit.client.0.smithi076.stdout:6/23: mknod d1/d4/c8 0 2024-03-23T15:44:47.418 INFO:tasks.workunit.client.0.smithi076.stdout:6/24: write d1/d4/f5 [138619,56899] 0 2024-03-23T15:44:47.419 INFO:tasks.workunit.client.0.smithi076.stdout:8/16: symlink l6 0 2024-03-23T15:44:47.421 INFO:tasks.workunit.client.0.smithi076.stdout:4/41: write f2 [1810452,31318] 0 2024-03-23T15:44:47.421 INFO:tasks.workunit.client.0.smithi076.stdout:4/42: chown l7 27565 1 2024-03-23T15:44:47.421 INFO:tasks.workunit.client.0.smithi076.stdout:4/43: chown f5 64499617 1 2024-03-23T15:44:47.421 INFO:tasks.workunit.client.0.smithi076.stdout:4/44: write f6 [999162,7665] 0 2024-03-23T15:44:47.422 INFO:tasks.workunit.client.0.smithi076.stdout:5/24: symlink la 0 2024-03-23T15:44:47.424 INFO:tasks.workunit.client.0.smithi076.stdout:2/38: rename d1/l2 to d1/ld 0 2024-03-23T15:44:47.427 INFO:tasks.workunit.client.0.smithi076.stdout:8/17: dread f2 [0,4194304] 0 2024-03-23T15:44:47.427 INFO:tasks.workunit.client.0.smithi076.stdout:8/18: rmdir - no directory 2024-03-23T15:44:47.427 INFO:tasks.workunit.client.0.smithi076.stdout:9/25: rename d1/f4 to d1/f6 0 2024-03-23T15:44:47.429 INFO:tasks.workunit.client.0.smithi076.stdout:7/23: write f1 [2033030,51358] 0 2024-03-23T15:44:47.429 INFO:tasks.workunit.client.0.smithi076.stdout:7/24: write f5 [191305,28582] 0 2024-03-23T15:44:47.429 INFO:tasks.workunit.client.0.smithi076.stdout:7/25: stat f5 0 2024-03-23T15:44:47.429 INFO:tasks.workunit.client.0.smithi076.stdout:4/45: unlink l3 0 2024-03-23T15:44:47.430 INFO:tasks.workunit.client.0.smithi076.stdout:4/46: write f5 [5091393,43752] 0 2024-03-23T15:44:47.431 INFO:tasks.workunit.client.0.smithi076.stdout:5/25: mknod cb 0 2024-03-23T15:44:47.431 INFO:tasks.workunit.client.0.smithi076.stdout:5/26: read - f5 zero size 2024-03-23T15:44:47.432 INFO:tasks.workunit.client.0.smithi076.stdout:5/27: write f4 [32462,14481] 0 2024-03-23T15:44:47.432 INFO:tasks.workunit.client.0.smithi076.stdout:5/28: rmdir - no directory 2024-03-23T15:44:47.433 INFO:tasks.workunit.client.0.smithi076.stdout:2/39: mknod d1/d4/ce 0 2024-03-23T15:44:47.435 INFO:tasks.workunit.client.0.smithi076.stdout:8/19: creat f7 x:0 0 0 2024-03-23T15:44:47.436 INFO:tasks.workunit.client.0.smithi076.stdout:7/26: dread f3 [0,4194304] 0 2024-03-23T15:44:47.438 INFO:tasks.workunit.client.0.smithi076.stdout:3/39: dwrite d5/f6 [0,4194304] 0 2024-03-23T15:44:47.439 INFO:tasks.workunit.client.0.smithi076.stdout:4/47: unlink d8/d9/ca 0 2024-03-23T15:44:47.439 INFO:tasks.workunit.client.0.smithi076.stdout:6/25: dwrite d1/d2/f7 [0,4194304] 0 2024-03-23T15:44:47.440 INFO:tasks.workunit.client.0.smithi076.stdout:2/40: symlink d1/lf 0 2024-03-23T15:44:47.444 INFO:tasks.workunit.client.0.smithi076.stdout:2/41: dread d1/d4/db/fc [0,4194304] 0 2024-03-23T15:44:47.444 INFO:tasks.workunit.client.0.smithi076.stdout:2/42: stat d1/d4/f7 0 2024-03-23T15:44:47.446 INFO:tasks.workunit.client.0.smithi076.stdout:3/40: write f4 [387095,111410] 0 2024-03-23T15:44:47.446 INFO:tasks.workunit.client.0.smithi076.stdout:9/26: dwrite d1/f6 [0,4194304] 0 2024-03-23T15:44:47.448 INFO:tasks.workunit.client.0.smithi076.stdout:7/27: creat f6 x:0 0 0 2024-03-23T15:44:47.449 INFO:tasks.workunit.client.0.smithi076.stdout:3/41: fdatasync f4 0 2024-03-23T15:44:47.449 INFO:tasks.workunit.client.0.smithi076.stdout:3/42: write d5/f6 [2431206,77666] 0 2024-03-23T15:44:47.455 INFO:tasks.workunit.client.0.smithi076.stdout:9/27: dread d1/f3 [0,4194304] 0 2024-03-23T15:44:47.457 INFO:tasks.workunit.client.0.smithi076.stdout:4/48: dwrite f6 [0,4194304] 0 2024-03-23T15:44:47.458 INFO:tasks.workunit.client.0.smithi076.stdout:3/43: dread f4 [0,4194304] 0 2024-03-23T15:44:47.460 INFO:tasks.workunit.client.0.smithi076.stdout:9/28: write d1/f6 [2986919,88467] 0 2024-03-23T15:44:47.461 INFO:tasks.workunit.client.0.smithi076.stdout:0/28: truncate d2/f6 2158143 0 2024-03-23T15:44:47.461 INFO:tasks.workunit.client.0.smithi076.stdout:0/29: write d2/f9 [751490,111682] 0 2024-03-23T15:44:47.462 INFO:tasks.workunit.client.0.smithi076.stdout:6/26: creat d1/d2/f9 x:0 0 0 2024-03-23T15:44:47.465 INFO:tasks.workunit.client.0.smithi076.stdout:2/43: rmdir d1/d4/db 39 2024-03-23T15:44:47.465 INFO:tasks.workunit.client.0.smithi076.stdout:7/28: dwrite f1 [0,4194304] 0 2024-03-23T15:44:47.465 INFO:tasks.workunit.client.0.smithi076.stdout:7/29: dread - f6 zero size 2024-03-23T15:44:47.465 INFO:tasks.workunit.client.0.smithi076.stdout:8/20: mkdir d8 0 2024-03-23T15:44:47.470 INFO:tasks.workunit.client.0.smithi076.stdout:4/49: creat d8/d9/fc x:0 0 0 2024-03-23T15:44:47.471 INFO:tasks.workunit.client.0.smithi076.stdout:3/44: creat d5/f7 x:0 0 0 2024-03-23T15:44:47.472 INFO:tasks.workunit.client.0.smithi076.stdout:7/30: write f1 [4158537,67687] 0 2024-03-23T15:44:47.472 INFO:tasks.workunit.client.0.smithi076.stdout:9/29: stat d1/c5 0 2024-03-23T15:44:47.478 INFO:tasks.workunit.client.0.smithi076.stdout:6/27: rename d1/l3 to d1/la 0 2024-03-23T15:44:47.478 INFO:tasks.workunit.client.0.smithi076.stdout:0/30: creat d2/d8/fa x:0 0 0 2024-03-23T15:44:47.478 INFO:tasks.workunit.client.0.smithi076.stdout:2/44: creat d1/d4/f10 x:0 0 0 2024-03-23T15:44:47.478 INFO:tasks.workunit.client.0.smithi076.stdout:8/21: mknod d8/c9 0 2024-03-23T15:44:47.479 INFO:tasks.workunit.client.0.smithi076.stdout:8/22: dread - f3 zero size 2024-03-23T15:44:47.479 INFO:tasks.workunit.client.0.smithi076.stdout:4/50: symlink d8/d9/ld 0 2024-03-23T15:44:47.481 INFO:tasks.workunit.client.0.smithi076.stdout:7/31: link f0 f7 0 2024-03-23T15:44:47.481 INFO:tasks.workunit.client.0.smithi076.stdout:7/32: stat f3 0 2024-03-23T15:44:47.481 INFO:tasks.workunit.client.0.smithi076.stdout:8/23: write f0 [2122334,13893] 0 2024-03-23T15:44:47.482 INFO:tasks.workunit.client.0.smithi076.stdout:9/30: creat d1/f7 x:0 0 0 2024-03-23T15:44:47.484 INFO:tasks.workunit.client.0.smithi076.stdout:6/28: rename d1/d4/c6 to d1/d4/cb 0 2024-03-23T15:44:47.484 INFO:tasks.workunit.client.0.smithi076.stdout:6/29: chown d1/d4/cb 0 1 2024-03-23T15:44:47.485 INFO:tasks.workunit.client.0.smithi076.stdout:6/30: fsync d1/d4/f5 0 2024-03-23T15:44:47.485 INFO:tasks.workunit.client.0.smithi076.stdout:6/31: chown d1/d2 12482 1 2024-03-23T15:44:47.485 INFO:tasks.workunit.client.0.smithi076.stdout:6/32: read - d1/d2/f9 zero size 2024-03-23T15:44:47.485 INFO:tasks.workunit.client.0.smithi076.stdout:2/45: symlink d1/d4/l11 0 2024-03-23T15:44:47.487 INFO:tasks.workunit.client.0.smithi076.stdout:7/33: mkdir d8 0 2024-03-23T15:44:47.488 INFO:tasks.workunit.client.0.smithi076.stdout:8/24: creat d8/fa x:0 0 0 2024-03-23T15:44:47.490 INFO:tasks.workunit.client.0.smithi076.stdout:3/45: dwrite f4 [0,4194304] 0 2024-03-23T15:44:47.491 INFO:tasks.workunit.client.0.smithi076.stdout:6/33: mknod d1/d2/cc 0 2024-03-23T15:44:47.491 INFO:tasks.workunit.client.0.smithi076.stdout:9/31: fsync d1/f6 0 2024-03-23T15:44:47.491 INFO:tasks.workunit.client.0.smithi076.stdout:9/32: write d1/f6 [3341218,66586] 0 2024-03-23T15:44:47.496 INFO:tasks.workunit.client.0.smithi076.stdout:2/46: mkdir d1/d4/d12 0 2024-03-23T15:44:47.496 INFO:tasks.workunit.client.0.smithi076.stdout:3/46: dread f4 [0,4194304] 0 2024-03-23T15:44:47.501 INFO:tasks.workunit.client.0.smithi076.stdout:6/34: mkdir d1/dd 0 2024-03-23T15:44:47.501 INFO:tasks.workunit.client.0.smithi076.stdout:6/35: write d1/d2/f9 [912198,123748] 0 2024-03-23T15:44:47.501 INFO:tasks.workunit.client.0.smithi076.stdout:6/36: stat d1 0 2024-03-23T15:44:47.503 INFO:tasks.workunit.client.0.smithi076.stdout:9/33: rename d1/f3 to d1/f8 0 2024-03-23T15:44:47.505 INFO:tasks.workunit.client.0.smithi076.stdout:3/47: symlink d5/l8 0 2024-03-23T15:44:47.507 INFO:tasks.workunit.client.0.smithi076.stdout:3/48: creat d5/f9 x:0 0 0 2024-03-23T15:44:47.512 INFO:tasks.workunit.client.0.smithi076.stdout:1/4: sync 2024-03-23T15:44:47.521 INFO:tasks.workunit.client.0.smithi076.stdout:7/34: fsync f0 0 2024-03-23T15:44:47.524 INFO:tasks.workunit.client.0.smithi076.stdout:7/35: symlink d8/l9 0 2024-03-23T15:44:47.524 INFO:tasks.workunit.client.0.smithi076.stdout:7/36: chown f0 18889 1 2024-03-23T15:44:47.524 INFO:tasks.workunit.client.0.smithi076.stdout:7/37: write f1 [349589,95181] 0 2024-03-23T15:44:47.525 INFO:tasks.workunit.client.0.smithi076.stdout:7/38: creat d8/fa x:0 0 0 2024-03-23T15:44:47.527 INFO:tasks.workunit.client.0.smithi076.stdout:4/51: dwrite f6 [4194304,4194304] 0 2024-03-23T15:44:47.528 INFO:tasks.workunit.client.0.smithi076.stdout:0/31: truncate d2/f9 271336 0 2024-03-23T15:44:47.534 INFO:tasks.workunit.client.0.smithi076.stdout:6/37: fsync d1/d2/f9 0 2024-03-23T15:44:47.536 INFO:tasks.workunit.client.0.smithi076.stdout:3/49: truncate f4 1274688 0 2024-03-23T15:44:47.537 INFO:tasks.workunit.client.0.smithi076.stdout:4/52: dread f2 [0,4194304] 0 2024-03-23T15:44:47.538 INFO:tasks.workunit.client.0.smithi076.stdout:6/38: mknod d1/d2/ce 0 2024-03-23T15:44:47.539 INFO:tasks.workunit.client.0.smithi076.stdout:4/53: mknod d8/ce 0 2024-03-23T15:44:47.539 INFO:tasks.workunit.client.0.smithi076.stdout:4/54: getdents d8/d9 0 2024-03-23T15:44:47.541 INFO:tasks.workunit.client.0.smithi076.stdout:4/55: link f2 d8/ff 0 2024-03-23T15:44:47.541 INFO:tasks.workunit.client.0.smithi076.stdout:7/39: dwrite d8/fa [0,4194304] 0 2024-03-23T15:44:47.542 INFO:tasks.workunit.client.0.smithi076.stdout:7/40: stat d8 0 2024-03-23T15:44:47.542 INFO:tasks.workunit.client.0.smithi076.stdout:4/56: mkdir d8/d9/d10 0 2024-03-23T15:44:47.544 INFO:tasks.workunit.client.0.smithi076.stdout:7/41: creat d8/fb x:0 0 0 2024-03-23T15:44:47.547 INFO:tasks.workunit.client.0.smithi076.stdout:4/57: link f5 d8/d9/f11 0 2024-03-23T15:44:47.547 INFO:tasks.workunit.client.0.smithi076.stdout:4/58: read d8/ff [802982,71870] 0 2024-03-23T15:44:47.548 INFO:tasks.workunit.client.0.smithi076.stdout:7/42: creat d8/fc x:0 0 0 2024-03-23T15:44:47.551 INFO:tasks.workunit.client.0.smithi076.stdout:7/43: write f3 [3762852,6954] 0 2024-03-23T15:44:47.551 INFO:tasks.workunit.client.0.smithi076.stdout:7/44: chown d8/l9 152 1 2024-03-23T15:44:47.553 INFO:tasks.workunit.client.0.smithi076.stdout:7/45: rmdir d8 39 2024-03-23T15:44:47.554 INFO:tasks.workunit.client.0.smithi076.stdout:7/46: creat d8/fd x:0 0 0 2024-03-23T15:44:47.554 INFO:tasks.workunit.client.0.smithi076.stdout:7/47: dread - d8/fc zero size 2024-03-23T15:44:47.555 INFO:tasks.workunit.client.0.smithi076.stdout:7/48: unlink f7 0 2024-03-23T15:44:47.556 INFO:tasks.workunit.client.0.smithi076.stdout:7/49: chown d8/fd 32 1 2024-03-23T15:44:47.556 INFO:tasks.workunit.client.0.smithi076.stdout:4/59: dread f6 [4194304,4194304] 0 2024-03-23T15:44:47.556 INFO:tasks.workunit.client.0.smithi076.stdout:4/60: truncate d8/ff 3869878 0 2024-03-23T15:44:47.556 INFO:tasks.workunit.client.0.smithi076.stdout:7/50: mkdir d8/de 0 2024-03-23T15:44:47.558 INFO:tasks.workunit.client.0.smithi076.stdout:7/51: symlink d8/lf 0 2024-03-23T15:44:47.561 INFO:tasks.workunit.client.0.smithi076.stdout:6/39: dwrite d1/d4/f5 [0,4194304] 0 2024-03-23T15:44:47.562 INFO:tasks.workunit.client.0.smithi076.stdout:6/40: chown d1/d4/cb 8340005 1 2024-03-23T15:44:47.568 INFO:tasks.workunit.client.0.smithi076.stdout:7/52: dread f1 [0,4194304] 0 2024-03-23T15:44:47.568 INFO:tasks.workunit.client.0.smithi076.stdout:7/53: write f0 [3139577,125048] 0 2024-03-23T15:44:47.568 INFO:tasks.workunit.client.0.smithi076.stdout:7/54: truncate d8/fd 67015 0 2024-03-23T15:44:47.577 INFO:tasks.workunit.client.0.smithi076.stdout:7/55: dread f5 [0,4194304] 0 2024-03-23T15:44:47.577 INFO:tasks.workunit.client.0.smithi076.stdout:7/56: readlink d8/l9 0 2024-03-23T15:44:47.586 INFO:tasks.workunit.client.0.smithi076.stdout:7/57: unlink d8/lf 0 2024-03-23T15:44:47.592 INFO:tasks.workunit.client.0.smithi076.stdout:9/34: unlink d1/f8 0 2024-03-23T15:44:47.598 INFO:tasks.workunit.client.0.smithi076.stdout:0/32: write d2/f9 [320161,56020] 0 2024-03-23T15:44:47.598 INFO:tasks.workunit.client.0.smithi076.stdout:0/33: fsync d2/f4 0 2024-03-23T15:44:47.606 INFO:tasks.workunit.client.0.smithi076.stdout:3/50: dread f4 [0,4194304] 0 2024-03-23T15:44:47.606 INFO:tasks.workunit.client.0.smithi076.stdout:3/51: write f4 [1997159,58121] 0 2024-03-23T15:44:47.612 INFO:tasks.workunit.client.0.smithi076.stdout:3/52: dread d5/f6 [0,4194304] 0 2024-03-23T15:44:47.612 INFO:tasks.workunit.client.0.smithi076.stdout:3/53: write d5/f9 [984260,53478] 0 2024-03-23T15:44:47.618 INFO:tasks.workunit.client.0.smithi076.stdout:3/54: dread d5/f9 [0,4194304] 0 2024-03-23T15:44:47.618 INFO:tasks.workunit.client.0.smithi076.stdout:3/55: write d5/f7 [192408,35408] 0 2024-03-23T15:44:47.619 INFO:tasks.workunit.client.0.smithi076.stdout:3/56: chown d5/f9 159141379 1 2024-03-23T15:44:47.631 INFO:tasks.workunit.client.0.smithi076.stdout:9/35: creat d1/f9 x:0 0 0 2024-03-23T15:44:47.631 INFO:tasks.workunit.client.0.smithi076.stdout:9/36: fdatasync d1/f9 0 2024-03-23T15:44:47.631 INFO:tasks.workunit.client.0.smithi076.stdout:9/37: chown d1/f7 2278 1 2024-03-23T15:44:47.637 INFO:tasks.workunit.client.0.smithi076.stdout:0/34: rename d2/l3 to d2/d8/lb 0 2024-03-23T15:44:47.639 INFO:tasks.workunit.client.0.smithi076.stdout:9/38: write d1/f6 [1134844,35090] 0 2024-03-23T15:44:47.640 INFO:tasks.workunit.client.0.smithi076.stdout:0/35: write d2/f6 [1564970,45476] 0 2024-03-23T15:44:47.656 INFO:tasks.workunit.client.0.smithi076.stdout:9/39: dwrite d1/f7 [0,4194304] 0 2024-03-23T15:44:47.656 INFO:tasks.workunit.client.0.smithi076.stdout:0/36: dwrite d2/d8/fa [0,4194304] 0 2024-03-23T15:44:47.663 INFO:tasks.workunit.client.0.smithi076.stdout:9/40: write d1/f9 [918740,109058] 0 2024-03-23T15:44:47.668 INFO:tasks.workunit.client.0.smithi076.stdout:3/57: dwrite f4 [0,4194304] 0 2024-03-23T15:44:47.671 INFO:tasks.workunit.client.0.smithi076.stdout:0/37: symlink d2/lc 0 2024-03-23T15:44:47.671 INFO:tasks.workunit.client.0.smithi076.stdout:0/38: write d2/f9 [874952,18928] 0 2024-03-23T15:44:47.677 INFO:tasks.workunit.client.0.smithi076.stdout:0/39: dread d2/f9 [0,4194304] 0 2024-03-23T15:44:47.680 INFO:tasks.workunit.client.0.smithi076.stdout:9/41: dwrite d1/f9 [0,4194304] 0 2024-03-23T15:44:47.684 INFO:tasks.workunit.client.0.smithi076.stdout:3/58: dwrite d5/f6 [4194304,4194304] 0 2024-03-23T15:44:47.686 INFO:tasks.workunit.client.0.smithi076.stdout:3/59: write d5/f9 [852329,89767] 0 2024-03-23T15:44:47.696 INFO:tasks.workunit.client.0.smithi076.stdout:0/40: symlink d2/ld 0 2024-03-23T15:44:47.698 INFO:tasks.workunit.client.0.smithi076.stdout:3/60: symlink d5/la 0 2024-03-23T15:44:47.698 INFO:tasks.workunit.client.0.smithi076.stdout:3/61: chown d5 2425 1 2024-03-23T15:44:47.704 INFO:tasks.workunit.client.0.smithi076.stdout:3/62: rename d5/la to d5/lb 0 2024-03-23T15:44:47.704 INFO:tasks.workunit.client.0.smithi076.stdout:3/63: readlink d5/l8 0 2024-03-23T15:44:47.705 INFO:tasks.workunit.client.0.smithi076.stdout:0/41: write d2/d8/fa [3213148,118120] 0 2024-03-23T15:44:47.705 INFO:tasks.workunit.client.0.smithi076.stdout:0/42: dread - d2/f7 zero size 2024-03-23T15:44:47.712 INFO:tasks.workunit.client.0.smithi076.stdout:3/64: rmdir d5 39 2024-03-23T15:44:47.715 INFO:tasks.workunit.client.0.smithi076.stdout:0/43: mkdir d2/de 0 2024-03-23T15:44:47.715 INFO:tasks.workunit.client.0.smithi076.stdout:0/44: readlink d2/lc 0 2024-03-23T15:44:47.717 INFO:tasks.workunit.client.0.smithi076.stdout:9/42: truncate d1/f6 2960990 0 2024-03-23T15:44:47.718 INFO:tasks.workunit.client.0.smithi076.stdout:0/45: symlink d2/lf 0 2024-03-23T15:44:47.718 INFO:tasks.workunit.client.0.smithi076.stdout:0/46: write d2/f6 [850438,58643] 0 2024-03-23T15:44:47.719 INFO:tasks.workunit.client.0.smithi076.stdout:9/43: stat d1/f7 0 2024-03-23T15:44:47.722 INFO:tasks.workunit.client.0.smithi076.stdout:9/44: write d1/f9 [4599215,75661] 0 2024-03-23T15:44:47.727 INFO:tasks.workunit.client.0.smithi076.stdout:9/45: unlink d1/f7 0 2024-03-23T15:44:47.730 INFO:tasks.workunit.client.0.smithi076.stdout:3/65: dwrite d5/f9 [0,4194304] 0 2024-03-23T15:44:47.733 INFO:tasks.workunit.client.0.smithi076.stdout:9/46: write d1/f9 [3803009,126999] 0 2024-03-23T15:44:47.735 INFO:tasks.workunit.client.0.smithi076.stdout:0/47: dwrite d2/d8/fa [0,4194304] 0 2024-03-23T15:44:47.739 INFO:tasks.workunit.client.0.smithi076.stdout:3/66: dread d5/f6 [4194304,4194304] 0 2024-03-23T15:44:47.750 INFO:tasks.workunit.client.0.smithi076.stdout:0/48: mkdir d2/de/d10 0 2024-03-23T15:44:47.751 INFO:tasks.workunit.client.0.smithi076.stdout:0/49: chown d2/f4 1542156462 1 2024-03-23T15:44:47.753 INFO:tasks.workunit.client.0.smithi076.stdout:0/50: fsync d2/f6 0 2024-03-23T15:44:47.753 INFO:tasks.workunit.client.0.smithi076.stdout:9/47: mknod d1/ca 0 2024-03-23T15:44:47.757 INFO:tasks.workunit.client.0.smithi076.stdout:3/67: symlink d5/lc 0 2024-03-23T15:44:47.757 INFO:tasks.workunit.client.0.smithi076.stdout:3/68: readlink d5/lb 0 2024-03-23T15:44:47.758 INFO:tasks.workunit.client.0.smithi076.stdout:0/51: creat d2/f11 x:0 0 0 2024-03-23T15:44:47.762 INFO:tasks.workunit.client.0.smithi076.stdout:3/69: rename f4 to d5/fd 0 2024-03-23T15:44:47.765 INFO:tasks.workunit.client.0.smithi076.stdout:3/70: link d5/lc d5/le 0 2024-03-23T15:44:47.770 INFO:tasks.workunit.client.0.smithi076.stdout:9/48: dwrite d1/f9 [0,4194304] 0 2024-03-23T15:44:47.776 INFO:tasks.workunit.client.0.smithi076.stdout:0/52: dwrite d2/f7 [0,4194304] 0 2024-03-23T15:44:47.783 INFO:tasks.workunit.client.0.smithi076.stdout:9/49: getdents d1 0 2024-03-23T15:44:47.784 INFO:tasks.workunit.client.0.smithi076.stdout:7/58: unlink f5 0 2024-03-23T15:44:47.786 INFO:tasks.workunit.client.0.smithi076.stdout:7/59: getdents d8 0 2024-03-23T15:44:47.786 INFO:tasks.workunit.client.0.smithi076.stdout:7/60: readlink d8/l9 0 2024-03-23T15:44:47.789 INFO:tasks.workunit.client.0.smithi076.stdout:9/50: dread d1/f9 [0,4194304] 0 2024-03-23T15:44:47.794 INFO:tasks.workunit.client.0.smithi076.stdout:0/53: getdents d2 0 2024-03-23T15:44:47.794 INFO:tasks.workunit.client.0.smithi076.stdout:0/54: fsync d2/f4 0 2024-03-23T15:44:47.797 INFO:tasks.workunit.client.0.smithi076.stdout:0/55: mkdir d2/de/d10/d12 0 2024-03-23T15:44:47.800 INFO:tasks.workunit.client.0.smithi076.stdout:0/56: unlink d2/f6 0 2024-03-23T15:44:47.801 INFO:tasks.workunit.client.0.smithi076.stdout:0/57: unlink d2/ld 0 2024-03-23T15:44:47.802 INFO:tasks.workunit.client.0.smithi076.stdout:3/71: getdents d5 0 2024-03-23T15:44:47.804 INFO:tasks.workunit.client.0.smithi076.stdout:9/51: dwrite d1/f6 [0,4194304] 0 2024-03-23T15:44:47.804 INFO:tasks.workunit.client.0.smithi076.stdout:0/58: creat d2/de/f13 x:0 0 0 2024-03-23T15:44:47.804 INFO:tasks.workunit.client.0.smithi076.stdout:0/59: write d2/de/f13 [771814,26254] 0 2024-03-23T15:44:47.806 INFO:tasks.workunit.client.0.smithi076.stdout:9/52: unlink d1/f6 0 2024-03-23T15:44:47.808 INFO:tasks.workunit.client.0.smithi076.stdout:0/60: rmdir d2/d8 39 2024-03-23T15:44:47.808 INFO:tasks.workunit.client.0.smithi076.stdout:3/72: write d5/fd [614464,55114] 0 2024-03-23T15:44:47.809 INFO:tasks.workunit.client.0.smithi076.stdout:3/73: write d5/f7 [1095359,90946] 0 2024-03-23T15:44:47.809 INFO:tasks.workunit.client.0.smithi076.stdout:9/53: creat d1/fb x:0 0 0 2024-03-23T15:44:47.818 INFO:tasks.workunit.client.0.smithi076.stdout:3/74: dread d5/f6 [4194304,4194304] 0 2024-03-23T15:44:47.818 INFO:tasks.workunit.client.0.smithi076.stdout:3/75: read d5/f7 [421759,99193] 0 2024-03-23T15:44:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:47 smithi192 ceph-mon[36653]: pgmap v46: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 1.70k op/s 2024-03-23T15:44:47.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:47 smithi076 ceph-mon[32176]: pgmap v46: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 1.70k op/s 2024-03-23T15:44:47.826 INFO:tasks.workunit.client.0.smithi076.stdout:9/54: dwrite d1/fb [0,4194304] 0 2024-03-23T15:44:47.829 INFO:tasks.workunit.client.0.smithi076.stdout:0/61: truncate d2/d8/fa 245234 0 2024-03-23T15:44:47.830 INFO:tasks.workunit.client.0.smithi076.stdout:9/55: dread d1/f9 [0,4194304] 0 2024-03-23T15:44:47.830 INFO:tasks.workunit.client.0.smithi076.stdout:3/76: dread d5/f9 [0,4194304] 0 2024-03-23T15:44:47.830 INFO:tasks.workunit.client.0.smithi076.stdout:3/77: read d5/f6 [1439686,40508] 0 2024-03-23T15:44:47.831 INFO:tasks.workunit.client.0.smithi076.stdout:9/56: symlink d1/lc 0 2024-03-23T15:44:47.833 INFO:tasks.workunit.client.0.smithi076.stdout:3/78: dread d5/f7 [0,4194304] 0 2024-03-23T15:44:47.835 INFO:tasks.workunit.client.0.smithi076.stdout:7/61: mknod d8/de/c10 0 2024-03-23T15:44:47.835 INFO:tasks.workunit.client.0.smithi076.stdout:9/57: mknod d1/cd 0 2024-03-23T15:44:47.836 INFO:tasks.workunit.client.0.smithi076.stdout:9/58: stat d1/cd 0 2024-03-23T15:44:47.837 INFO:tasks.workunit.client.0.smithi076.stdout:3/79: creat d5/ff x:0 0 0 2024-03-23T15:44:47.837 INFO:tasks.workunit.client.0.smithi076.stdout:3/80: readlink d5/l8 0 2024-03-23T15:44:47.838 INFO:tasks.workunit.client.0.smithi076.stdout:9/59: rename d1/lc to d1/le 0 2024-03-23T15:44:47.838 INFO:tasks.workunit.client.0.smithi076.stdout:9/60: fsync d1/f9 0 2024-03-23T15:44:47.839 INFO:tasks.workunit.client.0.smithi076.stdout:7/62: getdents d8/de 0 2024-03-23T15:44:47.841 INFO:tasks.workunit.client.0.smithi076.stdout:7/63: mkdir d8/d11 0 2024-03-23T15:44:47.842 INFO:tasks.workunit.client.0.smithi076.stdout:7/64: creat d8/f12 x:0 0 0 2024-03-23T15:44:47.845 INFO:tasks.workunit.client.0.smithi076.stdout:7/65: mkdir d8/de/d13 0 2024-03-23T15:44:47.847 INFO:tasks.workunit.client.0.smithi076.stdout:7/66: truncate d8/fa 1247287 0 2024-03-23T15:44:47.849 INFO:tasks.workunit.client.0.smithi076.stdout:7/67: rename d8/f12 to d8/de/d13/f14 0 2024-03-23T15:44:47.850 INFO:tasks.workunit.client.0.smithi076.stdout:7/68: write f6 [622525,2806] 0 2024-03-23T15:44:47.852 INFO:tasks.workunit.client.0.smithi076.stdout:9/61: dread d1/fb [0,4194304] 0 2024-03-23T15:44:47.852 INFO:tasks.workunit.client.0.smithi076.stdout:9/62: chown d1/fb 12 1 2024-03-23T15:44:47.852 INFO:tasks.workunit.client.0.smithi076.stdout:9/63: chown d1/c5 54848 1 2024-03-23T15:44:47.853 INFO:tasks.workunit.client.0.smithi076.stdout:7/69: rename f6 to d8/de/d13/f15 0 2024-03-23T15:44:47.855 INFO:tasks.workunit.client.0.smithi076.stdout:7/70: mknod d8/d11/c16 0 2024-03-23T15:44:47.856 INFO:tasks.workunit.client.0.smithi076.stdout:3/81: dwrite d5/f7 [0,4194304] 0 2024-03-23T15:44:47.856 INFO:tasks.workunit.client.0.smithi076.stdout:3/82: truncate d5/f9 4263229 0 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:4/61: sync 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:1/5: sync 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:1/6: fdatasync - no filename 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:8/25: sync 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:8/26: dread - f3 zero size 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:8/27: dread - f7 zero size 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:4/62: write d8/ff [976396,36721] 0 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:8/28: chown f2 104 1 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:8/29: dread - f7 zero size 2024-03-23T15:44:47.863 INFO:tasks.workunit.client.0.smithi076.stdout:2/47: sync 2024-03-23T15:44:47.864 INFO:tasks.workunit.client.0.smithi076.stdout:6/41: sync 2024-03-23T15:44:47.864 INFO:tasks.workunit.client.0.smithi076.stdout:5/29: sync 2024-03-23T15:44:47.864 INFO:tasks.workunit.client.0.smithi076.stdout:2/48: dread - d1/d4/f10 zero size 2024-03-23T15:44:47.865 INFO:tasks.workunit.client.0.smithi076.stdout:2/49: fdatasync d1/d4/f9 0 2024-03-23T15:44:47.865 INFO:tasks.workunit.client.0.smithi076.stdout:2/50: read - d1/d4/f10 zero size 2024-03-23T15:44:47.865 INFO:tasks.workunit.client.0.smithi076.stdout:2/51: write d1/d4/f10 [121922,38607] 0 2024-03-23T15:44:47.865 INFO:tasks.workunit.client.0.smithi076.stdout:1/7: mkdir d0/d1 0 2024-03-23T15:44:47.865 INFO:tasks.workunit.client.0.smithi076.stdout:4/63: link d8/d9/f11 d8/d9/f12 0 2024-03-23T15:44:47.865 INFO:tasks.workunit.client.0.smithi076.stdout:5/30: write f6 [617682,14673] 0 2024-03-23T15:44:47.865 INFO:tasks.workunit.client.0.smithi076.stdout:5/31: chown cb 3968321 1 2024-03-23T15:44:47.866 INFO:tasks.workunit.client.0.smithi076.stdout:8/30: symlink d8/lb 0 2024-03-23T15:44:47.866 INFO:tasks.workunit.client.0.smithi076.stdout:8/31: dread - d8/fa zero size 2024-03-23T15:44:47.866 INFO:tasks.workunit.client.0.smithi076.stdout:8/32: chown f2 0 1 2024-03-23T15:44:47.866 INFO:tasks.workunit.client.0.smithi076.stdout:8/33: fsync f7 0 2024-03-23T15:44:47.866 INFO:tasks.workunit.client.0.smithi076.stdout:8/34: fdatasync f3 0 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/8: mkdir d0/d2 0 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/9: dread - no filename 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/10: dread - no filename 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/11: dread - no filename 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/12: truncate - no filename 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/13: link - no file 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/14: truncate - no filename 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/15: fdatasync - no filename 2024-03-23T15:44:47.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/16: write - no filename 2024-03-23T15:44:47.868 INFO:tasks.workunit.client.0.smithi076.stdout:4/64: symlink d8/d9/l13 0 2024-03-23T15:44:47.869 INFO:tasks.workunit.client.0.smithi076.stdout:9/64: dwrite d1/fb [0,4194304] 0 2024-03-23T15:44:47.869 INFO:tasks.workunit.client.0.smithi076.stdout:6/42: unlink d1/d2/f7 0 2024-03-23T15:44:47.869 INFO:tasks.workunit.client.0.smithi076.stdout:2/52: symlink d1/l13 0 2024-03-23T15:44:47.870 INFO:tasks.workunit.client.0.smithi076.stdout:8/35: link f0 d8/fc 0 2024-03-23T15:44:47.870 INFO:tasks.workunit.client.0.smithi076.stdout:8/36: fsync f7 0 2024-03-23T15:44:47.870 INFO:tasks.workunit.client.0.smithi076.stdout:5/32: dread f6 [0,4194304] 0 2024-03-23T15:44:47.871 INFO:tasks.workunit.client.0.smithi076.stdout:1/17: mknod d0/c3 0 2024-03-23T15:44:47.873 INFO:tasks.workunit.client.0.smithi076.stdout:4/65: link l7 d8/d9/d10/l14 0 2024-03-23T15:44:47.874 INFO:tasks.workunit.client.0.smithi076.stdout:3/83: dwrite d5/f6 [4194304,4194304] 0 2024-03-23T15:44:47.875 INFO:tasks.workunit.client.0.smithi076.stdout:9/65: unlink d1/le 0 2024-03-23T15:44:47.877 INFO:tasks.workunit.client.0.smithi076.stdout:8/37: creat d8/fd x:0 0 0 2024-03-23T15:44:47.878 INFO:tasks.workunit.client.0.smithi076.stdout:1/18: symlink d0/d2/l4 0 2024-03-23T15:44:47.880 INFO:tasks.workunit.client.0.smithi076.stdout:4/66: mkdir d8/d9/d10/d15 0 2024-03-23T15:44:47.880 INFO:tasks.workunit.client.0.smithi076.stdout:4/67: write d8/d9/f11 [1309079,126807] 0 2024-03-23T15:44:47.880 INFO:tasks.workunit.client.0.smithi076.stdout:6/43: dread d1/d4/f5 [0,4194304] 0 2024-03-23T15:44:47.881 INFO:tasks.workunit.client.0.smithi076.stdout:9/66: mknod d1/cf 0 2024-03-23T15:44:47.882 INFO:tasks.workunit.client.0.smithi076.stdout:8/38: creat d8/fe x:0 0 0 2024-03-23T15:44:47.882 INFO:tasks.workunit.client.0.smithi076.stdout:8/39: dread - d8/fa zero size 2024-03-23T15:44:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:47 smithi183 ceph-mon[36296]: pgmap v46: 97 pgs: 97 active+clean; 68 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 1.70k op/s 2024-03-23T15:44:47.883 INFO:tasks.workunit.client.0.smithi076.stdout:1/19: mknod d0/d1/c5 0 2024-03-23T15:44:47.884 INFO:tasks.workunit.client.0.smithi076.stdout:1/20: read - no filename 2024-03-23T15:44:47.884 INFO:tasks.workunit.client.0.smithi076.stdout:6/44: write d1/d2/f9 [42258,26775] 0 2024-03-23T15:44:47.885 INFO:tasks.workunit.client.0.smithi076.stdout:4/68: creat d8/d9/d10/d15/f16 x:0 0 0 2024-03-23T15:44:47.886 INFO:tasks.workunit.client.0.smithi076.stdout:5/33: dwrite f6 [0,4194304] 0 2024-03-23T15:44:47.887 INFO:tasks.workunit.client.0.smithi076.stdout:9/67: link d1/c5 d1/c10 0 2024-03-23T15:44:47.888 INFO:tasks.workunit.client.0.smithi076.stdout:6/45: mkdir d1/dd/df 0 2024-03-23T15:44:47.888 INFO:tasks.workunit.client.0.smithi076.stdout:6/46: fsync d1/d4/f5 0 2024-03-23T15:44:47.891 INFO:tasks.workunit.client.0.smithi076.stdout:4/69: dread f2 [0,4194304] 0 2024-03-23T15:44:47.891 INFO:tasks.workunit.client.0.smithi076.stdout:6/47: mkdir d1/d4/d10 0 2024-03-23T15:44:47.894 INFO:tasks.workunit.client.0.smithi076.stdout:6/48: rmdir d1/d4 39 2024-03-23T15:44:47.894 INFO:tasks.workunit.client.0.smithi076.stdout:6/49: write d1/d2/f9 [781329,91507] 0 2024-03-23T15:44:47.895 INFO:tasks.workunit.client.0.smithi076.stdout:3/84: dwrite d5/f7 [0,4194304] 0 2024-03-23T15:44:47.896 INFO:tasks.workunit.client.0.smithi076.stdout:4/70: dread d8/d9/f12 [0,4194304] 0 2024-03-23T15:44:47.896 INFO:tasks.workunit.client.0.smithi076.stdout:4/71: stat d8/d9/f11 0 2024-03-23T15:44:47.896 INFO:tasks.workunit.client.0.smithi076.stdout:4/72: fdatasync f5 0 2024-03-23T15:44:47.897 INFO:tasks.workunit.client.0.smithi076.stdout:2/53: dwrite d1/d4/f7 [0,4194304] 0 2024-03-23T15:44:47.897 INFO:tasks.workunit.client.0.smithi076.stdout:6/50: creat d1/f11 x:0 0 0 2024-03-23T15:44:47.898 INFO:tasks.workunit.client.0.smithi076.stdout:2/54: write d1/f5 [442762,66559] 0 2024-03-23T15:44:47.899 INFO:tasks.workunit.client.0.smithi076.stdout:3/85: mknod d5/c10 0 2024-03-23T15:44:47.901 INFO:tasks.workunit.client.0.smithi076.stdout:4/73: write f6 [4710977,32611] 0 2024-03-23T15:44:47.901 INFO:tasks.workunit.client.0.smithi076.stdout:3/86: creat d5/f11 x:0 0 0 2024-03-23T15:44:47.903 INFO:tasks.workunit.client.0.smithi076.stdout:4/74: creat d8/d9/f17 x:0 0 0 2024-03-23T15:44:47.904 INFO:tasks.workunit.client.0.smithi076.stdout:3/87: creat d5/f12 x:0 0 0 2024-03-23T15:44:47.905 INFO:tasks.workunit.client.0.smithi076.stdout:4/75: mkdir d8/d9/d10/d15/d18 0 2024-03-23T15:44:47.908 INFO:tasks.workunit.client.0.smithi076.stdout:3/88: creat d5/f13 x:0 0 0 2024-03-23T15:44:47.909 INFO:tasks.workunit.client.0.smithi076.stdout:4/76: symlink d8/d9/l19 0 2024-03-23T15:44:47.912 INFO:tasks.workunit.client.0.smithi076.stdout:3/89: creat d5/f14 x:0 0 0 2024-03-23T15:44:47.914 INFO:tasks.workunit.client.0.smithi076.stdout:4/77: rename d8/d9/d10/d15/f16 to d8/f1a 0 2024-03-23T15:44:47.915 INFO:tasks.workunit.client.0.smithi076.stdout:3/90: creat d5/f15 x:0 0 0 2024-03-23T15:44:47.918 INFO:tasks.workunit.client.0.smithi076.stdout:6/51: dwrite d1/d4/f5 [0,4194304] 0 2024-03-23T15:44:47.921 INFO:tasks.workunit.client.0.smithi076.stdout:6/52: creat d1/d4/f12 x:0 0 0 2024-03-23T15:44:47.922 INFO:tasks.workunit.client.0.smithi076.stdout:3/91: dread d5/fd [4194304,4194304] 0 2024-03-23T15:44:47.923 INFO:tasks.workunit.client.0.smithi076.stdout:6/53: creat d1/dd/df/f13 x:0 0 0 2024-03-23T15:44:47.923 INFO:tasks.workunit.client.0.smithi076.stdout:6/54: fdatasync d1/f11 0 2024-03-23T15:44:47.923 INFO:tasks.workunit.client.0.smithi076.stdout:6/55: dread - d1/d4/f12 zero size 2024-03-23T15:44:47.924 INFO:tasks.workunit.client.0.smithi076.stdout:2/55: dwrite d1/d4/f7 [0,4194304] 0 2024-03-23T15:44:47.926 INFO:tasks.workunit.client.0.smithi076.stdout:3/92: rename d5/f15 to d5/f16 0 2024-03-23T15:44:47.929 INFO:tasks.workunit.client.0.smithi076.stdout:5/34: truncate f7 2593438 0 2024-03-23T15:44:47.929 INFO:tasks.workunit.client.0.smithi076.stdout:3/93: dread d5/fd [4194304,4194304] 0 2024-03-23T15:44:47.935 INFO:tasks.workunit.client.0.smithi076.stdout:5/35: write f9 [1099135,61171] 0 2024-03-23T15:44:47.938 INFO:tasks.workunit.client.0.smithi076.stdout:5/36: link f5 fc 0 2024-03-23T15:44:47.940 INFO:tasks.workunit.client.0.smithi076.stdout:6/56: dwrite d1/d2/f9 [0,4194304] 0 2024-03-23T15:44:47.940 INFO:tasks.workunit.client.0.smithi076.stdout:6/57: write d1/d4/f12 [474436,60796] 0 2024-03-23T15:44:47.940 INFO:tasks.workunit.client.0.smithi076.stdout:6/58: fsync d1/d4/f5 0 2024-03-23T15:44:47.940 INFO:tasks.workunit.client.0.smithi076.stdout:5/37: creat fd x:0 0 0 2024-03-23T15:44:47.941 INFO:tasks.workunit.client.0.smithi076.stdout:5/38: stat f5 0 2024-03-23T15:44:47.941 INFO:tasks.workunit.client.0.smithi076.stdout:5/39: chown la 0 1 2024-03-23T15:44:47.941 INFO:tasks.workunit.client.0.smithi076.stdout:5/40: stat fc 0 2024-03-23T15:44:47.942 INFO:tasks.workunit.client.0.smithi076.stdout:6/59: symlink d1/l14 0 2024-03-23T15:44:47.943 INFO:tasks.workunit.client.0.smithi076.stdout:5/41: link l3 le 0 2024-03-23T15:44:47.944 INFO:tasks.workunit.client.0.smithi076.stdout:6/60: symlink d1/d4/l15 0 2024-03-23T15:44:47.946 INFO:tasks.workunit.client.0.smithi076.stdout:5/42: rename l3 to lf 0 2024-03-23T15:44:47.946 INFO:tasks.workunit.client.0.smithi076.stdout:5/43: fdatasync fd 0 2024-03-23T15:44:47.958 INFO:tasks.workunit.client.0.smithi076.stdout:5/44: getdents . 0 2024-03-23T15:44:47.958 INFO:tasks.workunit.client.0.smithi076.stdout:5/45: chown f7 237942 1 2024-03-23T15:44:47.959 INFO:tasks.workunit.client.0.smithi076.stdout:5/46: write f4 [31535,118647] 0 2024-03-23T15:44:47.959 INFO:tasks.workunit.client.0.smithi076.stdout:5/47: write f4 [441160,104413] 0 2024-03-23T15:44:47.959 INFO:tasks.workunit.client.0.smithi076.stdout:5/48: chown le 778874 1 2024-03-23T15:44:47.960 INFO:tasks.workunit.client.0.smithi076.stdout:5/49: creat f10 x:0 0 0 2024-03-23T15:44:47.971 INFO:tasks.workunit.client.0.smithi076.stdout:5/50: dwrite fc [0,4194304] 0 2024-03-23T15:44:47.971 INFO:tasks.workunit.client.0.smithi076.stdout:5/51: fsync f7 0 2024-03-23T15:44:47.971 INFO:tasks.workunit.client.0.smithi076.stdout:5/52: fsync f9 0 2024-03-23T15:44:47.973 INFO:tasks.workunit.client.0.smithi076.stdout:5/53: creat f11 x:0 0 0 2024-03-23T15:44:47.993 INFO:tasks.workunit.client.0.smithi076.stdout:5/54: dwrite f4 [0,4194304] 0 2024-03-23T15:44:47.995 INFO:tasks.workunit.client.0.smithi076.stdout:5/55: creat f12 x:0 0 0 2024-03-23T15:44:48.003 INFO:tasks.workunit.client.0.smithi076.stdout:5/56: write fc [273888,85088] 0 2024-03-23T15:44:48.003 INFO:tasks.workunit.client.0.smithi076.stdout:5/57: truncate f11 927308 0 2024-03-23T15:44:48.004 INFO:tasks.workunit.client.0.smithi076.stdout:5/58: creat f13 x:0 0 0 2024-03-23T15:44:48.006 INFO:tasks.workunit.client.0.smithi076.stdout:5/59: mknod c14 0 2024-03-23T15:44:48.006 INFO:tasks.workunit.client.0.smithi076.stdout:5/60: fsync f9 0 2024-03-23T15:44:48.006 INFO:tasks.workunit.client.0.smithi076.stdout:5/61: mknod c15 0 2024-03-23T15:44:48.007 INFO:tasks.workunit.client.0.smithi076.stdout:5/62: rename c14 to c16 0 2024-03-23T15:44:48.008 INFO:tasks.workunit.client.0.smithi076.stdout:5/63: symlink l17 0 2024-03-23T15:44:48.010 INFO:tasks.workunit.client.0.smithi076.stdout:5/64: unlink f10 0 2024-03-23T15:44:48.026 INFO:tasks.workunit.client.0.smithi076.stdout:5/65: dwrite fc [4194304,4194304] 0 2024-03-23T15:44:48.035 INFO:tasks.workunit.client.0.smithi076.stdout:5/66: dread f4 [0,4194304] 0 2024-03-23T15:44:48.036 INFO:tasks.workunit.client.0.smithi076.stdout:5/67: creat f18 x:0 0 0 2024-03-23T15:44:48.038 INFO:tasks.workunit.client.0.smithi076.stdout:5/68: creat f19 x:0 0 0 2024-03-23T15:44:48.039 INFO:tasks.workunit.client.0.smithi076.stdout:5/69: mkdir d1a 0 2024-03-23T15:44:48.040 INFO:tasks.workunit.client.0.smithi076.stdout:5/70: creat d1a/f1b x:0 0 0 2024-03-23T15:44:48.042 INFO:tasks.workunit.client.0.smithi076.stdout:5/71: link c15 d1a/c1c 0 2024-03-23T15:44:48.042 INFO:tasks.workunit.client.0.smithi076.stdout:5/72: chown f19 19909643 1 2024-03-23T15:44:48.060 INFO:tasks.workunit.client.0.smithi076.stdout:5/73: dwrite f12 [0,4194304] 0 2024-03-23T15:44:48.060 INFO:tasks.workunit.client.0.smithi076.stdout:5/74: chown f6 16164 1 2024-03-23T15:44:48.065 INFO:tasks.workunit.client.0.smithi076.stdout:5/75: unlink f9 0 2024-03-23T15:44:48.065 INFO:tasks.workunit.client.0.smithi076.stdout:5/76: chown f11 186 1 2024-03-23T15:44:48.066 INFO:tasks.workunit.client.0.smithi076.stdout:5/77: creat d1a/f1d x:0 0 0 2024-03-23T15:44:48.066 INFO:tasks.workunit.client.0.smithi076.stdout:5/78: fdatasync f4 0 2024-03-23T15:44:48.084 INFO:tasks.workunit.client.0.smithi076.stdout:5/79: dwrite f18 [0,4194304] 0 2024-03-23T15:44:48.088 INFO:tasks.workunit.client.0.smithi076.stdout:5/80: link l17 d1a/l1e 0 2024-03-23T15:44:48.088 INFO:tasks.workunit.client.0.smithi076.stdout:5/81: dread - f19 zero size 2024-03-23T15:44:48.088 INFO:tasks.workunit.client.0.smithi076.stdout:5/82: chown c15 42 1 2024-03-23T15:44:48.089 INFO:tasks.workunit.client.0.smithi076.stdout:5/83: mknod d1a/c1f 0 2024-03-23T15:44:48.090 INFO:tasks.workunit.client.0.smithi076.stdout:5/84: mkdir d1a/d20 0 2024-03-23T15:44:48.091 INFO:tasks.workunit.client.0.smithi076.stdout:5/85: mkdir d1a/d20/d21 0 2024-03-23T15:44:48.092 INFO:tasks.workunit.client.0.smithi076.stdout:5/86: creat d1a/d20/d21/f22 x:0 0 0 2024-03-23T15:44:48.093 INFO:tasks.workunit.client.0.smithi076.stdout:5/87: mknod d1a/d20/c23 0 2024-03-23T15:44:48.107 INFO:tasks.workunit.client.0.smithi076.stdout:5/88: dwrite f11 [0,4194304] 0 2024-03-23T15:44:48.123 INFO:tasks.workunit.client.0.smithi076.stdout:5/89: dwrite fd [0,4194304] 0 2024-03-23T15:44:48.131 INFO:tasks.workunit.client.0.smithi076.stdout:5/90: dread f18 [0,4194304] 0 2024-03-23T15:44:48.132 INFO:tasks.workunit.client.0.smithi076.stdout:5/91: mknod d1a/c24 0 2024-03-23T15:44:48.134 INFO:tasks.workunit.client.0.smithi076.stdout:5/92: rename d1a/d20/d21/f22 to d1a/f25 0 2024-03-23T15:44:48.140 INFO:tasks.workunit.client.0.smithi076.stdout:5/93: rmdir d1a/d20/d21 0 2024-03-23T15:44:48.145 INFO:tasks.workunit.client.0.smithi076.stdout:5/94: read f18 [1238080,44166] 0 2024-03-23T15:44:48.146 INFO:tasks.workunit.client.0.smithi076.stdout:5/95: symlink d1a/d20/l26 0 2024-03-23T15:44:48.149 INFO:tasks.workunit.client.0.smithi076.stdout:5/96: rename c15 to d1a/d20/c27 0 2024-03-23T15:44:48.149 INFO:tasks.workunit.client.0.smithi076.stdout:5/97: truncate d1a/f1d 458507 0 2024-03-23T15:44:48.149 INFO:tasks.workunit.client.0.smithi076.stdout:5/98: chown f7 3974 1 2024-03-23T15:44:48.153 INFO:tasks.workunit.client.0.smithi076.stdout:5/99: dread f5 [0,4194304] 0 2024-03-23T15:44:48.155 INFO:tasks.workunit.client.0.smithi076.stdout:5/100: mknod d1a/c28 0 2024-03-23T15:44:48.157 INFO:tasks.workunit.client.0.smithi076.stdout:5/101: unlink d1a/l1e 0 2024-03-23T15:44:48.157 INFO:tasks.workunit.client.0.smithi076.stdout:5/102: dread - f19 zero size 2024-03-23T15:44:48.158 INFO:tasks.workunit.client.0.smithi076.stdout:5/103: creat d1a/f29 x:0 0 0 2024-03-23T15:44:48.165 INFO:tasks.workunit.client.0.smithi076.stdout:5/104: dread f5 [4194304,4194304] 0 2024-03-23T15:44:48.165 INFO:tasks.workunit.client.0.smithi076.stdout:5/105: chown f11 1606743729 1 2024-03-23T15:44:48.167 INFO:tasks.workunit.client.0.smithi076.stdout:5/106: creat d1a/f2a x:0 0 0 2024-03-23T15:44:48.167 INFO:tasks.workunit.client.0.smithi076.stdout:5/107: truncate d1a/f2a 880558 0 2024-03-23T15:44:48.167 INFO:tasks.workunit.client.0.smithi076.stdout:5/108: dread d1a/f1d [0,4194304] 0 2024-03-23T15:44:48.168 INFO:tasks.workunit.client.0.smithi076.stdout:5/109: mknod d1a/c2b 0 2024-03-23T15:44:48.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:48.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:48.820 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:48.504+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025df [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_child.c auth v12 ap=1 s=374 n(v0 rc2024-03-23T15:44:31.875527+0000 b374 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562ff82f4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025df:[<0x100000025de/selinux_execshare_child.c v12>,<0x10000002549/execshare v408>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025df:[<0x100000025de/selinux_execshare_child.c v12>,<0x10000002549/execshare v408>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.820 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:48.504+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025e0 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/Makefile auth v16 ap=1 s=1097 n(v0 rc2024-03-23T15:44:31.879527+0000 b1097 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562ff82f4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025e0:[<0x100000025de/Makefile v4>,<0x10000002549/execshare v408>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025e0:[<0x100000025de/Makefile v16>,<0x10000002549/execshare v408>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.820 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:48.504+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025e1 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_parent.c auth v14 ap=1 s=2576 n(v0 rc2024-03-23T15:44:31.880527+0000 b2576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562ff82f4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025e1:[<0x100000025de/selinux_execshare_parent.c v6>,<0x10000002549/execshare v408>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025e1:[<0x100000025de/selinux_execshare_parent.c v14>,<0x10000002549/execshare v408>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.820 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:48.505+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025e2 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare.sh auth v10 ap=1 s=1957 n(v0 rc2024-03-23T15:44:31.882527+0000 b1957 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x562ff82f5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025e2:[<0x100000025de/selinux_execshare.sh v10>,<0x10000002549/execshare v408>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025e2:[<0x100000025de/selinux_execshare.sh v10>,<0x10000002549/execshare v408>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.820 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:48.512+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025f6 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/selinux_rxdir.sh auth v6 ap=1 s=2627 n(v0 rc2009-05-11T06:39:46.000000+0000 b2627 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562ff82f5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025f6:[<0x100000025f5/selinux_rxdir.sh v2>,<0x10000002549/rxdir v348>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025f6:[<0x100000025f5/selinux_rxdir.sh v6>,<0x10000002549/rxdir v486>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.821 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:44:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:44:48.512+0000 7fdcec5f9700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025f7 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/Makefile auth v8 ap=1 s=1069 n(v0 rc2024-03-23T15:44:31.913526+0000 b1069 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x562ff82f5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025f7:[<0x100000025f5/Makefile v4>,<0x10000002549/rxdir v348>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025f7:[<0x100000025f5/Makefile v8>,<0x10000002549/rxdir v486>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:48 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:48.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:48 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:48.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:48 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:48.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:48 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:48.884 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:44:48 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: 2024-03-23T15:44:48.516+0000 7fbdd93db700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025d8 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/Makefile auth v12 ap=1 s=1069 n(v0 rc2024-03-23T15:44:31.866527+0000 b1069 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559985829080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025d8:[<0x100000025d6/Makefile v4>,<0x10000002549/ioctl v382>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025d8:[<0x100000025d6/Makefile v12>,<0x10000002549/ioctl v500>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.884 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:44:48 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: 2024-03-23T15:44:48.517+0000 7fbdd93db700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025d9 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_noioctl.c auth v16 ap=1 s=2297 n(v0 rc2024-03-23T15:44:31.867527+0000 b2297 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55998582ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025d9:[<0x100000025d6/selinux_noioctl.c v6>,<0x10000002549/ioctl v382>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025d9:[<0x100000025d6/selinux_noioctl.c v16>,<0x10000002549/ioctl v500>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.884 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:44:48 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: 2024-03-23T15:44:48.517+0000 7fbdd93db700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025da [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.c auth v10 ap=1 s=1690 n(v0 rc2024-03-23T15:44:31.869527+0000 b1690 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x559985828b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025da:[<0x100000025d6/selinux_ioctl.c v10>,<0x10000002549/ioctl v382>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025da:[<0x100000025d6/selinux_ioctl.c v10>,<0x10000002549/ioctl v500>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.884 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:44:48 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: 2024-03-23T15:44:48.517+0000 7fbdd93db700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000025d7 [3,head] /volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.sh auth v14 ap=1 s=1876 n(v0 rc2024-03-23T15:44:31.865527+0000 b1876 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15327=0-4194304@2} caps={15327=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@1},l=15327 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559985828580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000025d7:[<0x100000025d6/selinux_ioctl.sh v2>,<0x10000002549/ioctl v382>,<0x100000024cb/tests v296>,<0x10000002494/selinux-testsuite v164>,<0x10000001f2c/security v468>,<0x10000000204/kernel v605>,<0x10000000202/testcases v82>,<0x10000000007/ltp-full-20091231 v58>,<0x10000000006/fsstress v55>,<0x10000000005/tmp v49>,<0x100000001fb/client.0 v48>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v64>,<0x10000000001/sv_1 v85>,<0x10000000000/qa v76>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x100000025d7:[<0x100000025d6/selinux_ioctl.sh v14>,<0x10000002549/ioctl v500>,<0x100000024cb/tests v479>,<0x10000002494/selinux-testsuite v293>,<0x10000001f2c/security v2080>,<0x10000000204/kernel v1153>,<0x10000000202/testcases v1312>,<0x10000000007/ltp-full-20091231 v212>,<0x10000000006/fsstress v39>,<0x10000000005/tmp v71>,<0x100000001fb/client.0 v68>,<0x100000001fa/3c3ad643-b4c6-4143-9aae-2dd532c299c4 v66>,<0x10000000001/sv_1 v71>,<0x10000000000/qa v49>,<0x1/volumes v89>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-23T15:44:48.897 INFO:tasks.workunit.client.0.smithi076.stdout:4/78: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:0/62: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:2/56: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:0/63: chown d2/f9 2824008 1 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:8/40: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:1/21: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:5/110: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:3/94: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:5/111: readlink lf 0 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:9/68: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:7/71: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:6/61: sync 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:7/72: stat d8/de 0 2024-03-23T15:44:48.898 INFO:tasks.workunit.client.0.smithi076.stdout:6/62: fdatasync d1/d4/f5 0 2024-03-23T15:44:48.899 INFO:tasks.workunit.client.0.smithi076.stdout:6/63: read - d1/f11 zero size 2024-03-23T15:44:48.899 INFO:tasks.workunit.client.0.smithi076.stdout:9/69: chown d1/fb 242566 1 2024-03-23T15:44:48.899 INFO:tasks.workunit.client.0.smithi076.stdout:7/73: write f3 [93304,62850] 0 2024-03-23T15:44:48.899 INFO:tasks.workunit.client.0.smithi076.stdout:8/41: rename f0 to d8/ff 0 2024-03-23T15:44:48.899 INFO:tasks.workunit.client.0.smithi076.stdout:8/42: truncate d8/ff 4897620 0 2024-03-23T15:44:48.899 INFO:tasks.workunit.client.0.smithi076.stdout:7/74: getdents d8/d11 0 2024-03-23T15:44:48.900 INFO:tasks.workunit.client.0.smithi076.stdout:7/75: fdatasync d8/fd 0 2024-03-23T15:44:48.900 INFO:tasks.workunit.client.0.smithi076.stdout:1/22: mkdir d0/d1/d6 0 2024-03-23T15:44:48.900 INFO:tasks.workunit.client.0.smithi076.stdout:1/23: truncate - no filename 2024-03-23T15:44:48.901 INFO:tasks.workunit.client.0.smithi076.stdout:2/57: dread d1/f5 [0,4194304] 0 2024-03-23T15:44:48.901 INFO:tasks.workunit.client.0.smithi076.stdout:0/64: write d2/f4 [360530,79018] 0 2024-03-23T15:44:48.902 INFO:tasks.workunit.client.0.smithi076.stdout:5/112: mkdir d1a/d2c 0 2024-03-23T15:44:48.902 INFO:tasks.workunit.client.0.smithi076.stdout:7/76: dread d8/de/d13/f15 [0,4194304] 0 2024-03-23T15:44:48.902 INFO:tasks.workunit.client.0.smithi076.stdout:6/64: symlink d1/d2/l16 0 2024-03-23T15:44:48.903 INFO:tasks.workunit.client.0.smithi076.stdout:9/70: truncate d1/fb 3468872 0 2024-03-23T15:44:48.904 INFO:tasks.workunit.client.0.smithi076.stdout:2/58: creat d1/f14 x:0 0 0 2024-03-23T15:44:48.905 INFO:tasks.workunit.client.0.smithi076.stdout:3/95: write d5/f7 [1765494,107853] 0 2024-03-23T15:44:48.905 INFO:tasks.workunit.client.0.smithi076.stdout:2/59: chown d1/ca 1462850 1 2024-03-23T15:44:48.905 INFO:tasks.workunit.client.0.smithi076.stdout:1/24: rmdir d0/d1 39 2024-03-23T15:44:48.905 INFO:tasks.workunit.client.0.smithi076.stdout:1/25: fdatasync - no filename 2024-03-23T15:44:48.905 INFO:tasks.workunit.client.0.smithi076.stdout:2/60: write d1/f14 [351196,82928] 0 2024-03-23T15:44:48.905 INFO:tasks.workunit.client.0.smithi076.stdout:2/61: chown d1/f14 7129208 1 2024-03-23T15:44:48.905 INFO:tasks.workunit.client.0.smithi076.stdout:2/62: mkdir d1/d4/db/d15 0 2024-03-23T15:44:48.907 INFO:tasks.workunit.client.0.smithi076.stdout:0/65: creat d2/de/f14 x:0 0 0 2024-03-23T15:44:48.907 INFO:tasks.workunit.client.0.smithi076.stdout:2/63: rename d1/d4/f6 to d1/d4/db/d15/f16 0 2024-03-23T15:44:48.907 INFO:tasks.workunit.client.0.smithi076.stdout:2/64: dread - d1/d4/f9 zero size 2024-03-23T15:44:48.907 INFO:tasks.workunit.client.0.smithi076.stdout:2/65: truncate d1/d4/f7 4329340 0 2024-03-23T15:44:48.908 INFO:tasks.workunit.client.0.smithi076.stdout:5/113: rmdir d1a 39 2024-03-23T15:44:48.909 INFO:tasks.workunit.client.0.smithi076.stdout:5/114: chown f6 2493957 1 2024-03-23T15:44:48.910 INFO:tasks.workunit.client.0.smithi076.stdout:6/65: creat d1/d4/f17 x:0 0 0 2024-03-23T15:44:48.911 INFO:tasks.workunit.client.0.smithi076.stdout:6/66: stat d1/l14 0 2024-03-23T15:44:48.911 INFO:tasks.workunit.client.0.smithi076.stdout:6/67: chown d1/d4/c8 11845 1 2024-03-23T15:44:48.912 INFO:tasks.workunit.client.0.smithi076.stdout:9/71: unlink d1/c10 0 2024-03-23T15:44:48.914 INFO:tasks.workunit.client.0.smithi076.stdout:4/79: dwrite d8/d9/f12 [0,4194304] 0 2024-03-23T15:44:48.914 INFO:tasks.workunit.client.0.smithi076.stdout:4/80: dread - d8/f1a zero size 2024-03-23T15:44:48.914 INFO:tasks.workunit.client.0.smithi076.stdout:2/66: dread d1/d4/db/d15/f16 [0,4194304] 0 2024-03-23T15:44:48.914 INFO:tasks.workunit.client.0.smithi076.stdout:2/67: stat d1/d4/f9 0 2024-03-23T15:44:48.916 INFO:tasks.workunit.client.0.smithi076.stdout:3/96: mkdir d5/d17 0 2024-03-23T15:44:48.920 INFO:tasks.workunit.client.0.smithi076.stdout:1/26: unlink d0/d2/l4 0 2024-03-23T15:44:48.921 INFO:tasks.workunit.client.0.smithi076.stdout:0/66: rmdir d2 39 2024-03-23T15:44:48.923 INFO:tasks.workunit.client.0.smithi076.stdout:5/115: creat d1a/d2c/f2d x:0 0 0 2024-03-23T15:44:48.923 INFO:tasks.workunit.client.0.smithi076.stdout:8/43: truncate d8/fc 3958967 0 2024-03-23T15:44:48.925 INFO:tasks.workunit.client.0.smithi076.stdout:6/68: mkdir d1/d4/d10/d18 0 2024-03-23T15:44:48.926 INFO:tasks.workunit.client.0.smithi076.stdout:9/72: creat d1/f11 x:0 0 0 2024-03-23T15:44:48.929 INFO:tasks.workunit.client.0.smithi076.stdout:2/68: dwrite d1/d4/db/d15/f16 [0,4194304] 0 2024-03-23T15:44:48.929 INFO:tasks.workunit.client.0.smithi076.stdout:4/81: mkdir d8/d9/d10/d15/d18/d1b 0 2024-03-23T15:44:48.929 INFO:tasks.workunit.client.0.smithi076.stdout:4/82: readlink l7 0 2024-03-23T15:44:48.931 INFO:tasks.workunit.client.0.smithi076.stdout:3/97: link d5/c10 d5/c18 0 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/27: mknod d0/d2/c7 0 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/28: dread - no filename 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/29: dread - no filename 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/30: dread - no filename 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/31: write - no filename 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/32: dwrite - no filename 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/33: dwrite - no filename 2024-03-23T15:44:48.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/34: write - no filename 2024-03-23T15:44:48.934 INFO:tasks.workunit.client.0.smithi076.stdout:0/67: mknod d2/de/d10/c15 0 2024-03-23T15:44:48.936 INFO:tasks.workunit.client.0.smithi076.stdout:5/116: creat d1a/d2c/f2e x:0 0 0 2024-03-23T15:44:48.937 INFO:tasks.workunit.client.0.smithi076.stdout:8/44: unlink d8/fd 0 2024-03-23T15:44:48.938 INFO:tasks.workunit.client.0.smithi076.stdout:9/73: creat d1/f12 x:0 0 0 2024-03-23T15:44:48.940 INFO:tasks.workunit.client.0.smithi076.stdout:1/35: creat d0/f8 x:0 0 0 2024-03-23T15:44:48.940 INFO:tasks.workunit.client.0.smithi076.stdout:1/36: write d0/f8 [520511,81171] 0 2024-03-23T15:44:48.942 INFO:tasks.workunit.client.0.smithi076.stdout:0/68: rename d2/d8/lb to d2/l16 0 2024-03-23T15:44:48.942 INFO:tasks.workunit.client.0.smithi076.stdout:0/69: readlink d2/lf 0 2024-03-23T15:44:48.944 INFO:tasks.workunit.client.0.smithi076.stdout:5/117: symlink d1a/l2f 0 2024-03-23T15:44:48.944 INFO:tasks.workunit.client.0.smithi076.stdout:6/69: dwrite d1/d2/f9 [0,4194304] 0 2024-03-23T15:44:48.945 INFO:tasks.workunit.client.0.smithi076.stdout:9/74: rename d1/cd to d1/c13 0 2024-03-23T15:44:48.945 INFO:tasks.workunit.client.0.smithi076.stdout:9/75: chown d1/ca 66151852 1 2024-03-23T15:44:48.947 INFO:tasks.workunit.client.0.smithi076.stdout:5/118: dread fc [4194304,4194304] 0 2024-03-23T15:44:48.959 INFO:tasks.workunit.client.0.smithi076.stdout:4/83: dwrite d8/d9/fc [0,4194304] 0 2024-03-23T15:44:48.963 INFO:tasks.workunit.client.0.smithi076.stdout:4/84: symlink d8/d9/d10/d15/d18/l1c 0 2024-03-23T15:44:48.963 INFO:tasks.workunit.client.0.smithi076.stdout:4/85: readlink d8/d9/d10/d15/d18/l1c 0 2024-03-23T15:44:48.963 INFO:tasks.workunit.client.0.smithi076.stdout:4/86: read f6 [4454042,126584] 0 2024-03-23T15:44:48.966 INFO:tasks.workunit.client.0.smithi076.stdout:8/45: dwrite d8/fc [4194304,4194304] 0 2024-03-23T15:44:48.966 INFO:tasks.workunit.client.0.smithi076.stdout:8/46: readlink d8/lb 0 2024-03-23T15:44:48.966 INFO:tasks.workunit.client.0.smithi076.stdout:8/47: chown d8/fe 3238634 1 2024-03-23T15:44:48.968 INFO:tasks.workunit.client.0.smithi076.stdout:8/48: rmdir d8 39 2024-03-23T15:44:48.971 INFO:tasks.workunit.client.0.smithi076.stdout:8/49: creat d8/f10 x:0 0 0 2024-03-23T15:44:48.971 INFO:tasks.workunit.client.0.smithi076.stdout:9/76: dread d1/f9 [4194304,4194304] 0 2024-03-23T15:44:48.971 INFO:tasks.workunit.client.0.smithi076.stdout:9/77: truncate d1/f11 750289 0 2024-03-23T15:44:48.971 INFO:tasks.workunit.client.0.smithi076.stdout:6/70: dwrite d1/f11 [0,4194304] 0 2024-03-23T15:44:48.972 INFO:tasks.workunit.client.0.smithi076.stdout:8/50: creat d8/f11 x:0 0 0 2024-03-23T15:44:48.972 INFO:tasks.workunit.client.0.smithi076.stdout:8/51: stat f2 0 2024-03-23T15:44:48.973 INFO:tasks.workunit.client.0.smithi076.stdout:3/98: dwrite d5/f11 [0,4194304] 0 2024-03-23T15:44:48.975 INFO:tasks.workunit.client.0.smithi076.stdout:0/70: dwrite d2/de/f14 [0,4194304] 0 2024-03-23T15:44:48.975 INFO:tasks.workunit.client.0.smithi076.stdout:0/71: truncate d2/f11 996404 0 2024-03-23T15:44:48.975 INFO:tasks.workunit.client.0.smithi076.stdout:0/72: fdatasync d2/f7 0 2024-03-23T15:44:48.975 INFO:tasks.workunit.client.0.smithi076.stdout:9/78: dread d1/f9 [4194304,4194304] 0 2024-03-23T15:44:48.975 INFO:tasks.workunit.client.0.smithi076.stdout:8/52: symlink d8/l12 0 2024-03-23T15:44:48.975 INFO:tasks.workunit.client.0.smithi076.stdout:8/53: fdatasync d8/f10 0 2024-03-23T15:44:48.975 INFO:tasks.workunit.client.0.smithi076.stdout:8/54: readlink d8/lb 0 2024-03-23T15:44:48.976 INFO:tasks.workunit.client.0.smithi076.stdout:5/119: dwrite f11 [0,4194304] 0 2024-03-23T15:44:48.976 INFO:tasks.workunit.client.0.smithi076.stdout:5/120: chown d1a/d20 54713 1 2024-03-23T15:44:48.977 INFO:tasks.workunit.client.0.smithi076.stdout:4/87: truncate d8/d9/f12 244680 0 2024-03-23T15:44:48.979 INFO:tasks.workunit.client.0.smithi076.stdout:3/99: mknod d5/d17/c19 0 2024-03-23T15:44:48.979 INFO:tasks.workunit.client.0.smithi076.stdout:3/100: write d5/f16 [662573,89132] 0 2024-03-23T15:44:48.983 INFO:tasks.workunit.client.0.smithi076.stdout:0/73: mknod d2/d8/c17 0 2024-03-23T15:44:48.984 INFO:tasks.workunit.client.0.smithi076.stdout:9/79: unlink d1/f11 0 2024-03-23T15:44:48.985 INFO:tasks.workunit.client.0.smithi076.stdout:6/71: dread d1/d4/f5 [0,4194304] 0 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:6/72: truncate d1/d4/f17 451943 0 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:6/73: fdatasync d1/f11 0 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:4/88: dread f6 [4194304,4194304] 0 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:4/89: chown d8/d9/f12 123137904 1 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:4/90: fdatasync d8/f1a 0 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:8/55: creat d8/f13 x:0 0 0 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:8/56: write d8/f11 [585558,72536] 0 2024-03-23T15:44:48.986 INFO:tasks.workunit.client.0.smithi076.stdout:8/57: fdatasync d8/f13 0 2024-03-23T15:44:48.987 INFO:tasks.workunit.client.0.smithi076.stdout:3/101: creat d5/d17/f1a x:0 0 0 2024-03-23T15:44:48.987 INFO:tasks.workunit.client.0.smithi076.stdout:1/37: dwrite d0/f8 [0,4194304] 0 2024-03-23T15:44:48.988 INFO:tasks.workunit.client.0.smithi076.stdout:9/80: truncate d1/fb 1238052 0 2024-03-23T15:44:48.991 INFO:tasks.workunit.client.0.smithi076.stdout:6/74: getdents d1/dd 0 2024-03-23T15:44:48.991 INFO:tasks.workunit.client.0.smithi076.stdout:4/91: mkdir d8/d9/d1d 0 2024-03-23T15:44:48.992 INFO:tasks.workunit.client.0.smithi076.stdout:8/58: creat d8/f14 x:0 0 0 2024-03-23T15:44:48.992 INFO:tasks.workunit.client.0.smithi076.stdout:8/59: dread - d8/f10 zero size 2024-03-23T15:44:48.995 INFO:tasks.workunit.client.0.smithi076.stdout:6/75: mkdir d1/dd/d19 0 2024-03-23T15:44:48.996 INFO:tasks.workunit.client.0.smithi076.stdout:9/81: dread d1/f9 [4194304,4194304] 0 2024-03-23T15:44:48.997 INFO:tasks.workunit.client.0.smithi076.stdout:4/92: symlink d8/d9/l1e 0 2024-03-23T15:44:48.998 INFO:tasks.workunit.client.0.smithi076.stdout:8/60: symlink d8/l15 0 2024-03-23T15:44:48.998 INFO:tasks.workunit.client.0.smithi076.stdout:8/61: dread - d8/f10 zero size 2024-03-23T15:44:49.000 INFO:tasks.workunit.client.0.smithi076.stdout:4/93: symlink d8/d9/l1f 0 2024-03-23T15:44:49.000 INFO:tasks.workunit.client.0.smithi076.stdout:4/94: write f2 [4668367,970] 0 2024-03-23T15:44:49.001 INFO:tasks.workunit.client.0.smithi076.stdout:4/95: dread d8/d9/f11 [0,4194304] 0 2024-03-23T15:44:49.002 INFO:tasks.workunit.client.0.smithi076.stdout:0/74: dwrite d2/f9 [0,4194304] 0 2024-03-23T15:44:49.002 INFO:tasks.workunit.client.0.smithi076.stdout:8/62: creat d8/f16 x:0 0 0 2024-03-23T15:44:49.003 INFO:tasks.workunit.client.0.smithi076.stdout:3/102: write d5/f9 [616257,83401] 0 2024-03-23T15:44:49.004 INFO:tasks.workunit.client.0.smithi076.stdout:3/103: rename d5/d17 to d5/d17/d1b 22 2024-03-23T15:44:49.004 INFO:tasks.workunit.client.0.smithi076.stdout:3/104: chown d5/lc 9 1 2024-03-23T15:44:49.004 INFO:tasks.workunit.client.0.smithi076.stdout:0/75: write d2/de/f13 [380046,122893] 0 2024-03-23T15:44:49.006 INFO:tasks.workunit.client.0.smithi076.stdout:3/105: getdents d5 0 2024-03-23T15:44:49.011 INFO:tasks.workunit.client.0.smithi076.stdout:4/96: dread d8/d9/fc [0,4194304] 0 2024-03-23T15:44:49.014 INFO:tasks.workunit.client.0.smithi076.stdout:1/38: dwrite d0/f8 [0,4194304] 0 2024-03-23T15:44:49.014 INFO:tasks.workunit.client.0.smithi076.stdout:1/39: getdents d0/d1/d6 0 2024-03-23T15:44:49.014 INFO:tasks.workunit.client.0.smithi076.stdout:4/97: creat d8/d9/d10/d15/d18/f20 x:0 0 0 2024-03-23T15:44:49.015 INFO:tasks.workunit.client.0.smithi076.stdout:3/106: dread d5/f6 [4194304,4194304] 0 2024-03-23T15:44:49.016 INFO:tasks.workunit.client.0.smithi076.stdout:3/107: write d5/f16 [702198,55001] 0 2024-03-23T15:44:49.016 INFO:tasks.workunit.client.0.smithi076.stdout:9/82: dwrite d1/f9 [4194304,4194304] 0 2024-03-23T15:44:49.016 INFO:tasks.workunit.client.0.smithi076.stdout:4/98: creat d8/d9/d10/d15/d18/f21 x:0 0 0 2024-03-23T15:44:49.016 INFO:tasks.workunit.client.0.smithi076.stdout:3/108: rmdir d5/d17 39 2024-03-23T15:44:49.017 INFO:tasks.workunit.client.0.smithi076.stdout:9/83: mknod d1/c14 0 2024-03-23T15:44:49.022 INFO:tasks.workunit.client.0.smithi076.stdout:3/109: mkdir d5/d17/d1c 0 2024-03-23T15:44:49.022 INFO:tasks.workunit.client.0.smithi076.stdout:3/110: chown d5/d17/c19 14477394 1 2024-03-23T15:44:49.022 INFO:tasks.workunit.client.0.smithi076.stdout:4/99: write d8/d9/fc [2941025,78230] 0 2024-03-23T15:44:49.022 INFO:tasks.workunit.client.0.smithi076.stdout:9/84: creat d1/f15 x:0 0 0 2024-03-23T15:44:49.022 INFO:tasks.workunit.client.0.smithi076.stdout:0/76: dwrite d2/f9 [0,4194304] 0 2024-03-23T15:44:49.022 INFO:tasks.workunit.client.0.smithi076.stdout:3/111: link d5/d17/f1a d5/d17/d1c/f1d 0 2024-03-23T15:44:49.022 INFO:tasks.workunit.client.0.smithi076.stdout:8/63: dwrite f2 [8388608,4194304] 0 2024-03-23T15:44:49.023 INFO:tasks.workunit.client.0.smithi076.stdout:4/100: creat d8/d9/d10/f22 x:0 0 0 2024-03-23T15:44:49.023 INFO:tasks.workunit.client.0.smithi076.stdout:0/77: creat d2/f18 x:0 0 0 2024-03-23T15:44:49.023 INFO:tasks.workunit.client.0.smithi076.stdout:0/78: chown d2/de/d10/c15 226 1 2024-03-23T15:44:49.024 INFO:tasks.workunit.client.0.smithi076.stdout:3/112: creat d5/d17/d1c/f1e x:0 0 0 2024-03-23T15:44:49.024 INFO:tasks.workunit.client.0.smithi076.stdout:1/40: dread d0/f8 [0,4194304] 0 2024-03-23T15:44:49.025 INFO:tasks.workunit.client.0.smithi076.stdout:9/85: dread d1/f9 [0,4194304] 0 2024-03-23T15:44:49.025 INFO:tasks.workunit.client.0.smithi076.stdout:8/64: link f7 d8/f17 0 2024-03-23T15:44:49.026 INFO:tasks.workunit.client.0.smithi076.stdout:4/101: mknod d8/c23 0 2024-03-23T15:44:49.030 INFO:tasks.workunit.client.0.smithi076.stdout:8/65: creat d8/f18 x:0 0 0 2024-03-23T15:44:49.032 INFO:tasks.workunit.client.0.smithi076.stdout:8/66: rename d8/f17 to d8/f19 0 2024-03-23T15:44:49.034 INFO:tasks.workunit.client.0.smithi076.stdout:9/86: getdents d1 0 2024-03-23T15:44:49.034 INFO:tasks.workunit.client.0.smithi076.stdout:9/87: getdents d1 0 2024-03-23T15:44:49.037 INFO:tasks.workunit.client.0.smithi076.stdout:3/113: dread d5/f11 [0,4194304] 0 2024-03-23T15:44:49.038 INFO:tasks.workunit.client.0.smithi076.stdout:1/41: write d0/f8 [946771,117141] 0 2024-03-23T15:44:49.039 INFO:tasks.workunit.client.0.smithi076.stdout:9/88: write d1/fb [559917,25179] 0 2024-03-23T15:44:49.039 INFO:tasks.workunit.client.0.smithi076.stdout:3/114: creat d5/f1f x:0 0 0 2024-03-23T15:44:49.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/79: dread d2/de/f14 [0,4194304] 0 2024-03-23T15:44:49.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/80: fsync d2/f18 0 2024-03-23T15:44:49.040 INFO:tasks.workunit.client.0.smithi076.stdout:1/42: creat d0/d1/d6/f9 x:0 0 0 2024-03-23T15:44:49.042 INFO:tasks.workunit.client.0.smithi076.stdout:9/89: mknod d1/c16 0 2024-03-23T15:44:49.042 INFO:tasks.workunit.client.0.smithi076.stdout:9/90: dread - d1/f12 zero size 2024-03-23T15:44:49.042 INFO:tasks.workunit.client.0.smithi076.stdout:9/91: write d1/fb [2102470,10469] 0 2024-03-23T15:44:49.042 INFO:tasks.workunit.client.0.smithi076.stdout:1/43: rmdir d0/d2 39 2024-03-23T15:44:49.044 INFO:tasks.workunit.client.0.smithi076.stdout:9/92: mkdir d1/d17 0 2024-03-23T15:44:49.047 INFO:tasks.workunit.client.0.smithi076.stdout:8/67: dread d8/ff [4194304,4194304] 0 2024-03-23T15:44:49.047 INFO:tasks.workunit.client.0.smithi076.stdout:8/68: chown d8/ff 0 1 2024-03-23T15:44:49.047 INFO:tasks.workunit.client.0.smithi076.stdout:9/93: creat d1/f18 x:0 0 0 2024-03-23T15:44:49.047 INFO:tasks.workunit.client.0.smithi076.stdout:9/94: write d1/f12 [167736,72620] 0 2024-03-23T15:44:49.048 INFO:tasks.workunit.client.0.smithi076.stdout:0/81: write d2/de/f14 [2752343,54358] 0 2024-03-23T15:44:49.049 INFO:tasks.workunit.client.0.smithi076.stdout:4/102: dwrite d8/d9/d10/f22 [0,4194304] 0 2024-03-23T15:44:49.050 INFO:tasks.workunit.client.0.smithi076.stdout:8/69: creat d8/f1a x:0 0 0 2024-03-23T15:44:49.050 INFO:tasks.workunit.client.0.smithi076.stdout:8/70: dread - d8/f19 zero size 2024-03-23T15:44:49.051 INFO:tasks.workunit.client.0.smithi076.stdout:9/95: creat d1/f19 x:0 0 0 2024-03-23T15:44:49.053 INFO:tasks.workunit.client.0.smithi076.stdout:3/115: dwrite d5/fd [4194304,4194304] 0 2024-03-23T15:44:49.054 INFO:tasks.workunit.client.0.smithi076.stdout:0/82: getdents d2/de/d10 0 2024-03-23T15:44:49.059 INFO:tasks.workunit.client.0.smithi076.stdout:4/103: creat d8/d9/d1d/f24 x:0 0 0 2024-03-23T15:44:49.060 INFO:tasks.workunit.client.0.smithi076.stdout:8/71: symlink d8/l1b 0 2024-03-23T15:44:49.061 INFO:tasks.workunit.client.0.smithi076.stdout:8/72: read f2 [696897,106441] 0 2024-03-23T15:44:49.061 INFO:tasks.workunit.client.0.smithi076.stdout:8/73: write f7 [895658,113016] 0 2024-03-23T15:44:49.061 INFO:tasks.workunit.client.0.smithi076.stdout:8/74: chown l5 14134 1 2024-03-23T15:44:49.062 INFO:tasks.workunit.client.0.smithi076.stdout:9/96: rmdir d1 39 2024-03-23T15:44:49.062 INFO:tasks.workunit.client.0.smithi076.stdout:9/97: truncate d1/f9 9265565 0 2024-03-23T15:44:49.065 INFO:tasks.workunit.client.0.smithi076.stdout:3/116: rename d5/le to d5/l20 0 2024-03-23T15:44:49.069 INFO:tasks.workunit.client.0.smithi076.stdout:0/83: symlink d2/de/d10/d12/l19 0 2024-03-23T15:44:49.070 INFO:tasks.workunit.client.0.smithi076.stdout:0/84: fdatasync d2/f7 0 2024-03-23T15:44:49.070 INFO:tasks.workunit.client.0.smithi076.stdout:0/85: chown d2/f18 58228893 1 2024-03-23T15:44:49.076 INFO:tasks.workunit.client.0.smithi076.stdout:4/104: dwrite f5 [0,4194304] 0 2024-03-23T15:44:49.077 INFO:tasks.workunit.client.0.smithi076.stdout:9/98: creat d1/f1a x:0 0 0 2024-03-23T15:44:49.080 INFO:tasks.workunit.client.0.smithi076.stdout:8/75: dwrite d8/f19 [0,4194304] 0 2024-03-23T15:44:49.080 INFO:tasks.workunit.client.0.smithi076.stdout:8/76: chown d8/f11 154 1 2024-03-23T15:44:49.081 INFO:tasks.workunit.client.0.smithi076.stdout:3/117: getdents d5/d17 0 2024-03-23T15:44:49.082 INFO:tasks.workunit.client.0.smithi076.stdout:0/86: rmdir d2/d8 39 2024-03-23T15:44:49.084 INFO:tasks.workunit.client.0.smithi076.stdout:9/99: rmdir d1 39 2024-03-23T15:44:49.086 INFO:tasks.workunit.client.0.smithi076.stdout:8/77: mknod d8/c1c 0 2024-03-23T15:44:49.086 INFO:tasks.workunit.client.0.smithi076.stdout:0/87: stat d2/l16 0 2024-03-23T15:44:49.091 INFO:tasks.workunit.client.0.smithi076.stdout:9/100: rmdir d1/d17 0 2024-03-23T15:44:49.091 INFO:tasks.workunit.client.0.smithi076.stdout:9/101: stat d1/f19 0 2024-03-23T15:44:49.093 INFO:tasks.workunit.client.0.smithi076.stdout:8/78: creat d8/f1d x:0 0 0 2024-03-23T15:44:49.095 INFO:tasks.workunit.client.0.smithi076.stdout:8/79: dread d8/f11 [0,4194304] 0 2024-03-23T15:44:49.095 INFO:tasks.workunit.client.0.smithi076.stdout:8/80: fdatasync d8/f14 0 2024-03-23T15:44:49.095 INFO:tasks.workunit.client.0.smithi076.stdout:8/81: chown d8/f10 189621 1 2024-03-23T15:44:49.095 INFO:tasks.workunit.client.0.smithi076.stdout:9/102: symlink d1/l1b 0 2024-03-23T15:44:49.097 INFO:tasks.workunit.client.0.smithi076.stdout:9/103: unlink d1/f9 0 2024-03-23T15:44:49.099 INFO:tasks.workunit.client.0.smithi076.stdout:9/104: mknod d1/c1c 0 2024-03-23T15:44:49.104 INFO:tasks.workunit.client.0.smithi076.stdout:4/105: truncate f5 2423356 0 2024-03-23T15:44:49.104 INFO:tasks.workunit.client.0.smithi076.stdout:4/106: fsync d8/d9/d1d/f24 0 2024-03-23T15:44:49.104 INFO:tasks.workunit.client.0.smithi076.stdout:4/107: readlink l7 0 2024-03-23T15:44:49.105 INFO:tasks.workunit.client.0.smithi076.stdout:0/88: dwrite d2/f7 [0,4194304] 0 2024-03-23T15:44:49.105 INFO:tasks.workunit.client.0.smithi076.stdout:0/89: chown d2/de/d10/d12/l19 10741677 1 2024-03-23T15:44:49.105 INFO:tasks.workunit.client.0.smithi076.stdout:4/108: creat d8/d9/d10/f25 x:0 0 0 2024-03-23T15:44:49.106 INFO:tasks.workunit.client.0.smithi076.stdout:0/90: creat d2/de/f1a x:0 0 0 2024-03-23T15:44:49.107 INFO:tasks.workunit.client.0.smithi076.stdout:4/109: creat d8/d9/d10/f26 x:0 0 0 2024-03-23T15:44:49.107 INFO:tasks.workunit.client.0.smithi076.stdout:4/110: truncate d8/d9/d10/d15/d18/f21 633058 0 2024-03-23T15:44:49.107 INFO:tasks.workunit.client.0.smithi076.stdout:4/111: fdatasync d8/ff 0 2024-03-23T15:44:49.109 INFO:tasks.workunit.client.0.smithi076.stdout:0/91: mkdir d2/d1b 0 2024-03-23T15:44:49.110 INFO:tasks.workunit.client.0.smithi076.stdout:4/112: mknod d8/d9/c27 0 2024-03-23T15:44:49.111 INFO:tasks.workunit.client.0.smithi076.stdout:8/82: dwrite d8/f1d [0,4194304] 0 2024-03-23T15:44:49.111 INFO:tasks.workunit.client.0.smithi076.stdout:8/83: chown d8/lb 24960772 1 2024-03-23T15:44:49.112 INFO:tasks.workunit.client.0.smithi076.stdout:8/84: dread - d8/f1a zero size 2024-03-23T15:44:49.112 INFO:tasks.workunit.client.0.smithi076.stdout:0/92: mknod d2/c1c 0 2024-03-23T15:44:49.118 INFO:tasks.workunit.client.0.smithi076.stdout:0/93: fsync d2/de/f1a 0 2024-03-23T15:44:49.118 INFO:tasks.workunit.client.0.smithi076.stdout:0/94: readlink d2/lc 0 2024-03-23T15:44:49.119 INFO:tasks.workunit.client.0.smithi076.stdout:4/113: fsync d8/d9/d10/f26 0 2024-03-23T15:44:49.120 INFO:tasks.workunit.client.0.smithi076.stdout:9/105: dwrite d1/f18 [0,4194304] 0 2024-03-23T15:44:49.120 INFO:tasks.workunit.client.0.smithi076.stdout:9/106: write d1/f12 [322853,121573] 0 2024-03-23T15:44:49.120 INFO:tasks.workunit.client.0.smithi076.stdout:9/107: chown d1/ca 273 1 2024-03-23T15:44:49.124 INFO:tasks.workunit.client.0.smithi076.stdout:9/108: symlink d1/l1d 0 2024-03-23T15:44:49.124 INFO:tasks.workunit.client.0.smithi076.stdout:9/109: fsync d1/fb 0 2024-03-23T15:44:49.126 INFO:tasks.workunit.client.0.smithi076.stdout:0/95: truncate d2/f7 3190738 0 2024-03-23T15:44:49.128 INFO:tasks.workunit.client.0.smithi076.stdout:0/96: creat d2/f1d x:0 0 0 2024-03-23T15:44:49.129 INFO:tasks.workunit.client.0.smithi076.stdout:9/110: truncate d1/f18 700537 0 2024-03-23T15:44:49.131 INFO:tasks.workunit.client.0.smithi076.stdout:4/114: dwrite d8/d9/d10/f22 [4194304,4194304] 0 2024-03-23T15:44:49.132 INFO:tasks.workunit.client.0.smithi076.stdout:9/111: write d1/f18 [1433317,47740] 0 2024-03-23T15:44:49.134 INFO:tasks.workunit.client.0.smithi076.stdout:4/115: creat d8/f28 x:0 0 0 2024-03-23T15:44:49.138 INFO:tasks.workunit.client.0.smithi076.stdout:4/116: symlink d8/d9/d1d/l29 0 2024-03-23T15:44:49.138 INFO:tasks.workunit.client.0.smithi076.stdout:4/117: truncate d8/f28 804054 0 2024-03-23T15:44:49.145 INFO:tasks.workunit.client.0.smithi076.stdout:4/118: dread d8/ff [0,4194304] 0 2024-03-23T15:44:49.150 INFO:tasks.workunit.client.0.smithi076.stdout:0/97: dwrite d2/de/f1a [0,4194304] 0 2024-03-23T15:44:49.152 INFO:tasks.workunit.client.0.smithi076.stdout:0/98: creat d2/d1b/f1e x:0 0 0 2024-03-23T15:44:49.154 INFO:tasks.workunit.client.0.smithi076.stdout:0/99: creat d2/de/d10/d12/f1f x:0 0 0 2024-03-23T15:44:49.154 INFO:tasks.workunit.client.0.smithi076.stdout:0/100: truncate d2/f18 997622 0 2024-03-23T15:44:49.164 INFO:tasks.workunit.client.0.smithi076.stdout:4/119: dwrite d8/d9/d10/d15/d18/f21 [0,4194304] 0 2024-03-23T15:44:49.171 INFO:tasks.workunit.client.0.smithi076.stdout:4/120: dread d8/d9/d10/d15/d18/f21 [0,4194304] 0 2024-03-23T15:44:49.172 INFO:tasks.workunit.client.0.smithi076.stdout:4/121: unlink d8/d9/d10/d15/d18/l1c 0 2024-03-23T15:44:49.173 INFO:tasks.workunit.client.0.smithi076.stdout:4/122: symlink d8/d9/l2a 0 2024-03-23T15:44:49.174 INFO:tasks.workunit.client.0.smithi076.stdout:4/123: creat d8/d9/d10/d15/d18/f2b x:0 0 0 2024-03-23T15:44:49.176 INFO:tasks.workunit.client.0.smithi076.stdout:4/124: rename d8/d9/l19 to d8/d9/d10/d15/d18/d1b/l2c 0 2024-03-23T15:44:49.192 INFO:tasks.workunit.client.0.smithi076.stdout:4/125: dwrite d8/ff [0,4194304] 0 2024-03-23T15:44:49.193 INFO:tasks.workunit.client.0.smithi076.stdout:4/126: creat d8/d9/f2d x:0 0 0 2024-03-23T15:44:49.193 INFO:tasks.workunit.client.0.smithi076.stdout:4/127: readlink d8/d9/l2a 0 2024-03-23T15:44:49.196 INFO:tasks.workunit.client.0.smithi076.stdout:4/128: dread d8/d9/fc [0,4194304] 0 2024-03-23T15:44:49.196 INFO:tasks.workunit.client.0.smithi076.stdout:4/129: fdatasync d8/d9/f17 0 2024-03-23T15:44:49.197 INFO:tasks.workunit.client.0.smithi076.stdout:4/130: symlink d8/d9/l2e 0 2024-03-23T15:44:49.197 INFO:tasks.workunit.client.0.smithi076.stdout:4/131: read - d8/d9/f17 zero size 2024-03-23T15:44:49.197 INFO:tasks.workunit.client.0.smithi076.stdout:4/132: fdatasync d8/f28 0 2024-03-23T15:44:49.197 INFO:tasks.workunit.client.0.smithi076.stdout:4/133: fdatasync d8/ff 0 2024-03-23T15:44:49.197 INFO:tasks.workunit.client.0.smithi076.stdout:4/134: write d8/d9/f12 [3135047,82091] 0 2024-03-23T15:44:49.208 INFO:tasks.workunit.client.0.smithi076.stdout:4/135: dread f2 [0,4194304] 0 2024-03-23T15:44:49.208 INFO:tasks.workunit.client.0.smithi076.stdout:4/136: chown d8/d9/f11 834724 1 2024-03-23T15:44:49.219 INFO:tasks.workunit.client.0.smithi076.stdout:4/137: dread f5 [0,4194304] 0 2024-03-23T15:44:49.220 INFO:tasks.workunit.client.0.smithi076.stdout:4/138: mknod d8/d9/d10/c2f 0 2024-03-23T15:44:49.231 INFO:tasks.workunit.client.0.smithi076.stdout:4/139: dwrite d8/d9/d10/f26 [0,4194304] 0 2024-03-23T15:44:49.232 INFO:tasks.workunit.client.0.smithi076.stdout:4/140: readlink l7 0 2024-03-23T15:44:49.233 INFO:tasks.workunit.client.0.smithi076.stdout:4/141: creat d8/f30 x:0 0 0 2024-03-23T15:44:49.239 INFO:tasks.workunit.client.0.smithi076.stdout:4/142: dread d8/d9/d10/f22 [4194304,4194304] 0 2024-03-23T15:44:49.239 INFO:tasks.workunit.client.0.smithi076.stdout:4/143: dread - d8/d9/f17 zero size 2024-03-23T15:44:49.240 INFO:tasks.workunit.client.0.smithi076.stdout:4/144: dread d8/f28 [0,4194304] 0 2024-03-23T15:44:49.258 INFO:tasks.workunit.client.0.smithi076.stdout:4/145: dwrite d8/d9/f17 [0,4194304] 0 2024-03-23T15:44:49.263 INFO:tasks.workunit.client.0.smithi076.stdout:4/146: write f2 [1836629,97317] 0 2024-03-23T15:44:49.263 INFO:tasks.workunit.client.0.smithi076.stdout:4/147: write d8/f28 [1600387,50837] 0 2024-03-23T15:44:49.263 INFO:tasks.workunit.client.0.smithi076.stdout:4/148: chown d8/f30 78322 1 2024-03-23T15:44:49.272 INFO:tasks.workunit.client.0.smithi076.stdout:1/44: sync 2024-03-23T15:44:49.272 INFO:tasks.workunit.client.0.smithi076.stdout:2/69: sync 2024-03-23T15:44:49.272 INFO:tasks.workunit.client.0.smithi076.stdout:7/77: sync 2024-03-23T15:44:49.272 INFO:tasks.workunit.client.0.smithi076.stdout:1/45: rename d0/d1 to d0/d1/da 22 2024-03-23T15:44:49.272 INFO:tasks.workunit.client.0.smithi076.stdout:1/46: chown d0 877 1 2024-03-23T15:44:49.273 INFO:tasks.workunit.client.0.smithi076.stdout:5/121: sync 2024-03-23T15:44:49.273 INFO:tasks.workunit.client.0.smithi076.stdout:6/76: sync 2024-03-23T15:44:49.273 INFO:tasks.workunit.client.0.smithi076.stdout:5/122: fdatasync d1a/f1b 0 2024-03-23T15:44:49.273 INFO:tasks.workunit.client.0.smithi076.stdout:1/47: write d0/f8 [428829,38090] 0 2024-03-23T15:44:49.273 INFO:tasks.workunit.client.0.smithi076.stdout:1/48: fsync d0/d1/d6/f9 0 2024-03-23T15:44:49.273 INFO:tasks.workunit.client.0.smithi076.stdout:7/78: mknod d8/c17 0 2024-03-23T15:44:49.274 INFO:tasks.workunit.client.0.smithi076.stdout:2/70: symlink d1/d4/l17 0 2024-03-23T15:44:49.275 INFO:tasks.workunit.client.0.smithi076.stdout:7/79: mkdir d8/de/d13/d18 0 2024-03-23T15:44:49.276 INFO:tasks.workunit.client.0.smithi076.stdout:7/80: creat d8/d11/f19 x:0 0 0 2024-03-23T15:44:49.277 INFO:tasks.workunit.client.0.smithi076.stdout:4/149: dwrite d8/f1a [0,4194304] 0 2024-03-23T15:44:49.278 INFO:tasks.workunit.client.0.smithi076.stdout:7/81: mkdir d8/de/d13/d18/d1a 0 2024-03-23T15:44:49.278 INFO:tasks.workunit.client.0.smithi076.stdout:7/82: truncate d8/fd 483932 0 2024-03-23T15:44:49.278 INFO:tasks.workunit.client.0.smithi076.stdout:7/83: fdatasync d8/fd 0 2024-03-23T15:44:49.282 INFO:tasks.workunit.client.0.smithi076.stdout:4/150: write d8/d9/f12 [2564284,112588] 0 2024-03-23T15:44:49.283 INFO:tasks.workunit.client.0.smithi076.stdout:4/151: truncate d8/d9/d10/f26 5233497 0 2024-03-23T15:44:49.283 INFO:tasks.workunit.client.0.smithi076.stdout:4/152: mknod d8/c31 0 2024-03-23T15:44:49.284 INFO:tasks.workunit.client.0.smithi076.stdout:4/153: stat d8/d9/c27 0 2024-03-23T15:44:49.286 INFO:tasks.workunit.client.0.smithi076.stdout:2/71: dread d1/d4/f7 [0,4194304] 0 2024-03-23T15:44:49.287 INFO:tasks.workunit.client.0.smithi076.stdout:1/49: dwrite d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:49.290 INFO:tasks.workunit.client.0.smithi076.stdout:1/50: truncate d0/f8 1516086 0 2024-03-23T15:44:49.292 INFO:tasks.workunit.client.0.smithi076.stdout:1/51: mknod d0/d1/cb 0 2024-03-23T15:44:49.293 INFO:tasks.workunit.client.0.smithi076.stdout:1/52: write d0/f8 [616794,90831] 0 2024-03-23T15:44:49.293 INFO:tasks.workunit.client.0.smithi076.stdout:1/53: chown d0/d2 7715 1 2024-03-23T15:44:49.295 INFO:tasks.workunit.client.0.smithi076.stdout:1/54: symlink d0/d2/lc 0 2024-03-23T15:44:49.296 INFO:tasks.workunit.client.0.smithi076.stdout:1/55: mknod d0/d1/cd 0 2024-03-23T15:44:49.297 INFO:tasks.workunit.client.0.smithi076.stdout:1/56: chown d0/f8 42 1 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:4/154: dwrite d8/f1a [4194304,4194304] 0 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:3/118: sync 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:0/101: sync 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:8/85: sync 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:9/112: sync 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:0/102: readlink d2/lc 0 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:6/77: rename d1/d2/cc to d1/dd/c1a 0 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:9/113: truncate d1/f15 463390 0 2024-03-23T15:44:49.307 INFO:tasks.workunit.client.0.smithi076.stdout:9/114: chown d1/f1a 35606 1 2024-03-23T15:44:49.308 INFO:tasks.workunit.client.0.smithi076.stdout:9/115: read d1/f15 [112708,39999] 0 2024-03-23T15:44:49.308 INFO:tasks.workunit.client.0.smithi076.stdout:2/72: dwrite d1/f5 [0,4194304] 0 2024-03-23T15:44:49.308 INFO:tasks.workunit.client.0.smithi076.stdout:2/73: fsync d1/f14 0 2024-03-23T15:44:49.308 INFO:tasks.workunit.client.0.smithi076.stdout:7/84: dwrite f0 [0,4194304] 0 2024-03-23T15:44:49.309 INFO:tasks.workunit.client.0.smithi076.stdout:3/119: creat d5/d17/f21 x:0 0 0 2024-03-23T15:44:49.310 INFO:tasks.workunit.client.0.smithi076.stdout:0/103: symlink d2/d8/l20 0 2024-03-23T15:44:49.310 INFO:tasks.workunit.client.0.smithi076.stdout:0/104: write d2/f4 [3578605,120978] 0 2024-03-23T15:44:49.311 INFO:tasks.workunit.client.0.smithi076.stdout:0/105: fsync d2/de/f13 0 2024-03-23T15:44:49.311 INFO:tasks.workunit.client.0.smithi076.stdout:2/74: unlink d1/d4/db/d15/f16 0 2024-03-23T15:44:49.311 INFO:tasks.workunit.client.0.smithi076.stdout:2/75: truncate d1/d4/f10 712351 0 2024-03-23T15:44:49.311 INFO:tasks.workunit.client.0.smithi076.stdout:8/86: creat d8/f1e x:0 0 0 2024-03-23T15:44:49.312 INFO:tasks.workunit.client.0.smithi076.stdout:6/78: creat d1/d4/d10/d18/f1b x:0 0 0 2024-03-23T15:44:49.313 INFO:tasks.workunit.client.0.smithi076.stdout:7/85: link d8/c17 d8/d11/c1b 0 2024-03-23T15:44:49.313 INFO:tasks.workunit.client.0.smithi076.stdout:7/86: truncate d8/de/d13/f15 1429709 0 2024-03-23T15:44:49.313 INFO:tasks.workunit.client.0.smithi076.stdout:7/87: chown f0 0 1 2024-03-23T15:44:49.313 INFO:tasks.workunit.client.0.smithi076.stdout:9/116: mknod d1/c1e 0 2024-03-23T15:44:49.314 INFO:tasks.workunit.client.0.smithi076.stdout:1/57: dwrite d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:49.316 INFO:tasks.workunit.client.0.smithi076.stdout:9/117: dread d1/fb [0,4194304] 0 2024-03-23T15:44:49.316 INFO:tasks.workunit.client.0.smithi076.stdout:3/120: rename d5/lb to d5/d17/l22 0 2024-03-23T15:44:49.317 INFO:tasks.workunit.client.0.smithi076.stdout:0/106: link d2/c1c d2/de/d10/d12/c21 0 2024-03-23T15:44:49.319 INFO:tasks.workunit.client.0.smithi076.stdout:8/87: rename d8/lb to d8/l1f 0 2024-03-23T15:44:49.320 INFO:tasks.workunit.client.0.smithi076.stdout:9/118: mkdir d1/d1f 0 2024-03-23T15:44:49.320 INFO:tasks.workunit.client.0.smithi076.stdout:6/79: write d1/d4/f5 [1126164,103571] 0 2024-03-23T15:44:49.320 INFO:tasks.workunit.client.0.smithi076.stdout:6/80: stat d1/d2/f9 0 2024-03-23T15:44:49.320 INFO:tasks.workunit.client.0.smithi076.stdout:6/81: write d1/d4/f5 [1019989,124558] 0 2024-03-23T15:44:49.321 INFO:tasks.workunit.client.0.smithi076.stdout:3/121: symlink d5/l23 0 2024-03-23T15:44:49.323 INFO:tasks.workunit.client.0.smithi076.stdout:6/82: creat d1/d2/f1c x:0 0 0 2024-03-23T15:44:49.325 INFO:tasks.workunit.client.0.smithi076.stdout:8/88: mknod d8/c20 0 2024-03-23T15:44:49.326 INFO:tasks.workunit.client.0.smithi076.stdout:9/119: mkdir d1/d20 0 2024-03-23T15:44:49.328 INFO:tasks.workunit.client.0.smithi076.stdout:3/122: rmdir d5 39 2024-03-23T15:44:49.328 INFO:tasks.workunit.client.0.smithi076.stdout:3/123: rename d5 to d5/d24 22 2024-03-23T15:44:49.330 INFO:tasks.workunit.client.0.smithi076.stdout:4/155: dwrite d8/ff [4194304,4194304] 0 2024-03-23T15:44:49.330 INFO:tasks.workunit.client.0.smithi076.stdout:6/83: symlink d1/d4/d10/d18/l1d 0 2024-03-23T15:44:49.330 INFO:tasks.workunit.client.0.smithi076.stdout:6/84: chown d1/d4/cb 0 1 2024-03-23T15:44:49.330 INFO:tasks.workunit.client.0.smithi076.stdout:6/85: write d1/d4/f5 [4968801,49083] 0 2024-03-23T15:44:49.330 INFO:tasks.workunit.client.0.smithi076.stdout:0/107: dread d2/f18 [0,4194304] 0 2024-03-23T15:44:49.331 INFO:tasks.workunit.client.0.smithi076.stdout:0/108: fdatasync d2/de/f14 0 2024-03-23T15:44:49.333 INFO:tasks.workunit.client.0.smithi076.stdout:4/156: dread d8/d9/fc [0,4194304] 0 2024-03-23T15:44:49.333 INFO:tasks.workunit.client.0.smithi076.stdout:9/120: link d1/f1a d1/d1f/f21 0 2024-03-23T15:44:49.333 INFO:tasks.workunit.client.0.smithi076.stdout:8/89: symlink d8/l21 0 2024-03-23T15:44:49.333 INFO:tasks.workunit.client.0.smithi076.stdout:4/157: read f2 [847594,75169] 0 2024-03-23T15:44:49.333 INFO:tasks.workunit.client.0.smithi076.stdout:6/86: rmdir d1/d4/d10 39 2024-03-23T15:44:49.334 INFO:tasks.workunit.client.0.smithi076.stdout:4/158: mkdir d8/d9/d32 0 2024-03-23T15:44:49.335 INFO:tasks.workunit.client.0.smithi076.stdout:8/90: symlink d8/l22 0 2024-03-23T15:44:49.335 INFO:tasks.workunit.client.0.smithi076.stdout:8/91: truncate d8/f18 402451 0 2024-03-23T15:44:49.336 INFO:tasks.workunit.client.0.smithi076.stdout:6/87: getdents d1/dd 0 2024-03-23T15:44:49.336 INFO:tasks.workunit.client.0.smithi076.stdout:4/159: mknod d8/d9/d10/d15/d18/c33 0 2024-03-23T15:44:49.337 INFO:tasks.workunit.client.0.smithi076.stdout:9/121: write d1/f18 [1094467,34354] 0 2024-03-23T15:44:49.337 INFO:tasks.workunit.client.0.smithi076.stdout:9/122: chown d1/l1b 0 1 2024-03-23T15:44:49.337 INFO:tasks.workunit.client.0.smithi076.stdout:8/92: dread d8/f18 [0,4194304] 0 2024-03-23T15:44:49.337 INFO:tasks.workunit.client.0.smithi076.stdout:4/160: creat d8/d9/d10/d15/f34 x:0 0 0 2024-03-23T15:44:49.338 INFO:tasks.workunit.client.0.smithi076.stdout:4/161: chown d8/d9/d10/d15/d18 259762 1 2024-03-23T15:44:49.338 INFO:tasks.workunit.client.0.smithi076.stdout:3/124: dread d5/fd [4194304,4194304] 0 2024-03-23T15:44:49.339 INFO:tasks.workunit.client.0.smithi076.stdout:9/123: creat d1/d1f/f22 x:0 0 0 2024-03-23T15:44:49.339 INFO:tasks.workunit.client.0.smithi076.stdout:7/88: dwrite d8/fa [0,4194304] 0 2024-03-23T15:44:49.340 INFO:tasks.workunit.client.0.smithi076.stdout:9/124: rmdir d1 39 2024-03-23T15:44:49.346 INFO:tasks.workunit.client.0.smithi076.stdout:6/88: write d1/d2/f9 [1934830,101454] 0 2024-03-23T15:44:49.347 INFO:tasks.workunit.client.0.smithi076.stdout:7/89: link d8/l9 d8/de/l1c 0 2024-03-23T15:44:49.347 INFO:tasks.workunit.client.0.smithi076.stdout:7/90: write d8/fc [906256,112864] 0 2024-03-23T15:44:49.347 INFO:tasks.workunit.client.0.smithi076.stdout:7/91: dread - d8/fb zero size 2024-03-23T15:44:49.347 INFO:tasks.workunit.client.0.smithi076.stdout:7/92: readlink d8/de/l1c 0 2024-03-23T15:44:49.349 INFO:tasks.workunit.client.0.smithi076.stdout:2/76: dwrite d1/d4/f10 [0,4194304] 0 2024-03-23T15:44:49.352 INFO:tasks.workunit.client.0.smithi076.stdout:4/162: write d8/f1a [6681658,56428] 0 2024-03-23T15:44:49.354 INFO:tasks.workunit.client.0.smithi076.stdout:0/109: dwrite d2/f11 [0,4194304] 0 2024-03-23T15:44:49.358 INFO:tasks.workunit.client.0.smithi076.stdout:2/77: creat d1/d4/d12/f18 x:0 0 0 2024-03-23T15:44:49.358 INFO:tasks.workunit.client.0.smithi076.stdout:8/93: dwrite d8/ff [0,4194304] 0 2024-03-23T15:44:49.359 INFO:tasks.workunit.client.0.smithi076.stdout:0/110: truncate d2/f7 2090504 0 2024-03-23T15:44:49.359 INFO:tasks.workunit.client.0.smithi076.stdout:0/111: chown d2/f7 1334 1 2024-03-23T15:44:49.360 INFO:tasks.workunit.client.0.smithi076.stdout:8/94: rmdir d8 39 2024-03-23T15:44:49.361 INFO:tasks.workunit.client.0.smithi076.stdout:0/112: link d2/d1b/f1e d2/de/f22 0 2024-03-23T15:44:49.363 INFO:tasks.workunit.client.0.smithi076.stdout:0/113: creat d2/de/d10/d12/f23 x:0 0 0 2024-03-23T15:44:49.369 INFO:tasks.workunit.client.0.smithi076.stdout:3/125: dwrite d5/f7 [0,4194304] 0 2024-03-23T15:44:49.369 INFO:tasks.workunit.client.0.smithi076.stdout:3/126: chown d5/c10 4037 1 2024-03-23T15:44:49.372 INFO:tasks.workunit.client.0.smithi076.stdout:3/127: symlink d5/l25 0 2024-03-23T15:44:49.372 INFO:tasks.workunit.client.0.smithi076.stdout:3/128: read d5/f6 [1505886,57715] 0 2024-03-23T15:44:49.372 INFO:tasks.workunit.client.0.smithi076.stdout:3/129: write d5/d17/f1a [3980,70200] 0 2024-03-23T15:44:49.374 INFO:tasks.workunit.client.0.smithi076.stdout:6/89: dwrite d1/f11 [0,4194304] 0 2024-03-23T15:44:49.377 INFO:tasks.workunit.client.0.smithi076.stdout:0/114: read d2/f9 [2414112,10099] 0 2024-03-23T15:44:49.377 INFO:tasks.workunit.client.0.smithi076.stdout:6/90: creat d1/d4/d10/d18/f1e x:0 0 0 2024-03-23T15:44:49.378 INFO:tasks.workunit.client.0.smithi076.stdout:3/130: write d5/f11 [329531,57926] 0 2024-03-23T15:44:49.379 INFO:tasks.workunit.client.0.smithi076.stdout:0/115: symlink d2/de/d10/l24 0 2024-03-23T15:44:49.379 INFO:tasks.workunit.client.0.smithi076.stdout:0/116: chown d2/f18 1 1 2024-03-23T15:44:49.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/91: link d1/d4/l15 d1/l1f 0 2024-03-23T15:44:49.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/92: write d1/d2/f1c [170053,122526] 0 2024-03-23T15:44:49.381 INFO:tasks.workunit.client.0.smithi076.stdout:0/117: creat d2/d8/f25 x:0 0 0 2024-03-23T15:44:49.381 INFO:tasks.workunit.client.0.smithi076.stdout:8/95: dwrite d8/f18 [0,4194304] 0 2024-03-23T15:44:49.381 INFO:tasks.workunit.client.0.smithi076.stdout:0/118: readlink d2/de/d10/l24 0 2024-03-23T15:44:49.385 INFO:tasks.workunit.client.0.smithi076.stdout:3/131: write d5/f7 [265664,20646] 0 2024-03-23T15:44:49.385 INFO:tasks.workunit.client.0.smithi076.stdout:3/132: dread - d5/f13 zero size 2024-03-23T15:44:49.386 INFO:tasks.workunit.client.0.smithi076.stdout:7/93: write f1 [4544749,64511] 0 2024-03-23T15:44:49.386 INFO:tasks.workunit.client.0.smithi076.stdout:9/125: dwrite d1/f15 [0,4194304] 0 2024-03-23T15:44:49.386 INFO:tasks.workunit.client.0.smithi076.stdout:9/126: dread - d1/f19 zero size 2024-03-23T15:44:49.387 INFO:tasks.workunit.client.0.smithi076.stdout:7/94: mknod d8/de/d13/d18/c1d 0 2024-03-23T15:44:49.387 INFO:tasks.workunit.client.0.smithi076.stdout:7/95: readlink d8/de/l1c 0 2024-03-23T15:44:49.387 INFO:tasks.workunit.client.0.smithi076.stdout:7/96: stat d8/fd 0 2024-03-23T15:44:49.388 INFO:tasks.workunit.client.0.smithi076.stdout:8/96: dread f7 [0,4194304] 0 2024-03-23T15:44:49.393 INFO:tasks.workunit.client.0.smithi076.stdout:6/93: creat d1/d4/d10/f20 x:0 0 0 2024-03-23T15:44:49.397 INFO:tasks.workunit.client.0.smithi076.stdout:0/119: symlink d2/de/d10/d12/l26 0 2024-03-23T15:44:49.400 INFO:tasks.workunit.client.0.smithi076.stdout:3/133: mknod d5/c26 0 2024-03-23T15:44:49.402 INFO:tasks.workunit.client.0.smithi076.stdout:9/127: rename d1/fb to d1/d1f/f23 0 2024-03-23T15:44:49.404 INFO:tasks.workunit.client.0.smithi076.stdout:0/120: dread d2/f11 [0,4194304] 0 2024-03-23T15:44:49.404 INFO:tasks.workunit.client.0.smithi076.stdout:0/121: chown d2/de/d10/d12/c21 25 1 2024-03-23T15:44:49.404 INFO:tasks.workunit.client.0.smithi076.stdout:7/97: dwrite d8/de/d13/f14 [0,4194304] 0 2024-03-23T15:44:49.405 INFO:tasks.workunit.client.0.smithi076.stdout:3/134: mkdir d5/d27 0 2024-03-23T15:44:49.408 INFO:tasks.workunit.client.0.smithi076.stdout:6/94: rename d1/d4/f17 to d1/f21 0 2024-03-23T15:44:49.408 INFO:tasks.workunit.client.0.smithi076.stdout:3/135: dread d5/f7 [0,4194304] 0 2024-03-23T15:44:49.408 INFO:tasks.workunit.client.0.smithi076.stdout:3/136: write d5/f14 [112845,126800] 0 2024-03-23T15:44:49.409 INFO:tasks.workunit.client.0.smithi076.stdout:9/128: symlink d1/d1f/l24 0 2024-03-23T15:44:49.411 INFO:tasks.workunit.client.0.smithi076.stdout:6/95: unlink d1/l1f 0 2024-03-23T15:44:49.412 INFO:tasks.workunit.client.0.smithi076.stdout:6/96: fdatasync d1/d4/d10/d18/f1e 0 2024-03-23T15:44:49.412 INFO:tasks.workunit.client.0.smithi076.stdout:6/97: fdatasync d1/dd/df/f13 0 2024-03-23T15:44:49.418 INFO:tasks.workunit.client.0.smithi076.stdout:7/98: dread f1 [0,4194304] 0 2024-03-23T15:44:49.421 INFO:tasks.workunit.client.0.smithi076.stdout:7/99: stat f0 0 2024-03-23T15:44:49.426 INFO:tasks.workunit.client.0.smithi076.stdout:0/122: dwrite d2/f4 [0,4194304] 0 2024-03-23T15:44:49.429 INFO:tasks.workunit.client.0.smithi076.stdout:0/123: creat d2/d8/f27 x:0 0 0 2024-03-23T15:44:49.429 INFO:tasks.workunit.client.0.smithi076.stdout:0/124: truncate d2/d8/f25 323320 0 2024-03-23T15:44:49.430 INFO:tasks.workunit.client.0.smithi076.stdout:0/125: rename d2/de/d10/d12/f1f to d2/d8/f28 0 2024-03-23T15:44:49.430 INFO:tasks.workunit.client.0.smithi076.stdout:0/126: chown d2/lc 109164 1 2024-03-23T15:44:49.431 INFO:tasks.workunit.client.0.smithi076.stdout:0/127: dread - d2/d8/f27 zero size 2024-03-23T15:44:49.432 INFO:tasks.workunit.client.0.smithi076.stdout:0/128: mknod d2/d8/c29 0 2024-03-23T15:44:49.432 INFO:tasks.workunit.client.0.smithi076.stdout:0/129: write d2/de/f14 [959872,80758] 0 2024-03-23T15:44:49.432 INFO:tasks.workunit.client.0.smithi076.stdout:0/130: stat d2/f11 0 2024-03-23T15:44:49.432 INFO:tasks.workunit.client.0.smithi076.stdout:9/129: dwrite d1/d1f/f23 [0,4194304] 0 2024-03-23T15:44:49.435 INFO:tasks.workunit.client.0.smithi076.stdout:3/137: dwrite d5/ff [0,4194304] 0 2024-03-23T15:44:49.435 INFO:tasks.workunit.client.0.smithi076.stdout:0/131: symlink d2/l2a 0 2024-03-23T15:44:49.435 INFO:tasks.workunit.client.0.smithi076.stdout:0/132: chown d2/f11 252 1 2024-03-23T15:44:49.435 INFO:tasks.workunit.client.0.smithi076.stdout:0/133: stat d2/lf 0 2024-03-23T15:44:49.435 INFO:tasks.workunit.client.0.smithi076.stdout:0/134: chown d2/de/d10/d12/f23 1 1 2024-03-23T15:44:49.436 INFO:tasks.workunit.client.0.smithi076.stdout:0/135: chown d2 59 1 2024-03-23T15:44:49.436 INFO:tasks.workunit.client.0.smithi076.stdout:9/130: creat d1/d20/f25 x:0 0 0 2024-03-23T15:44:49.443 INFO:tasks.workunit.client.0.smithi076.stdout:6/98: getdents d1/d2 0 2024-03-23T15:44:49.443 INFO:tasks.workunit.client.0.smithi076.stdout:6/99: stat d1/d2/l16 0 2024-03-23T15:44:49.443 INFO:tasks.workunit.client.0.smithi076.stdout:6/100: fsync d1/d4/d10/d18/f1e 0 2024-03-23T15:44:49.449 INFO:tasks.workunit.client.0.smithi076.stdout:6/101: creat d1/d2/f22 x:0 0 0 2024-03-23T15:44:49.451 INFO:tasks.workunit.client.0.smithi076.stdout:6/102: mkdir d1/dd/d19/d23 0 2024-03-23T15:44:49.452 INFO:tasks.workunit.client.0.smithi076.stdout:0/136: dwrite d2/f18 [0,4194304] 0 2024-03-23T15:44:49.452 INFO:tasks.workunit.client.0.smithi076.stdout:5/123: sync 2024-03-23T15:44:49.453 INFO:tasks.workunit.client.0.smithi076.stdout:9/131: dwrite d1/d20/f25 [0,4194304] 0 2024-03-23T15:44:49.453 INFO:tasks.workunit.client.0.smithi076.stdout:9/132: chown d1/l1d 2 1 2024-03-23T15:44:49.455 INFO:tasks.workunit.client.0.smithi076.stdout:0/137: symlink d2/de/d10/d12/l2b 0 2024-03-23T15:44:49.456 INFO:tasks.workunit.client.0.smithi076.stdout:5/124: creat d1a/d20/f30 x:0 0 0 2024-03-23T15:44:49.457 INFO:tasks.workunit.client.0.smithi076.stdout:6/103: write d1/f11 [2700625,361] 0 2024-03-23T15:44:49.457 INFO:tasks.workunit.client.0.smithi076.stdout:6/104: dread - d1/d4/d10/d18/f1e zero size 2024-03-23T15:44:49.458 INFO:tasks.workunit.client.0.smithi076.stdout:9/133: rename d1/d1f/f23 to d1/f26 0 2024-03-23T15:44:49.459 INFO:tasks.workunit.client.0.smithi076.stdout:0/138: rmdir d2/d1b 39 2024-03-23T15:44:49.459 INFO:tasks.workunit.client.0.smithi076.stdout:0/139: chown d2/f9 615722044 1 2024-03-23T15:44:49.460 INFO:tasks.workunit.client.0.smithi076.stdout:5/125: creat d1a/f31 x:0 0 0 2024-03-23T15:44:49.461 INFO:tasks.workunit.client.0.smithi076.stdout:5/126: read - f19 zero size 2024-03-23T15:44:49.462 INFO:tasks.workunit.client.0.smithi076.stdout:6/105: unlink d1/d4/d10/f20 0 2024-03-23T15:44:49.463 INFO:tasks.workunit.client.0.smithi076.stdout:0/140: truncate d2/f7 1576553 0 2024-03-23T15:44:49.464 INFO:tasks.workunit.client.0.smithi076.stdout:5/127: creat d1a/d20/f32 x:0 0 0 2024-03-23T15:44:49.465 INFO:tasks.workunit.client.0.smithi076.stdout:6/106: mknod d1/dd/c24 0 2024-03-23T15:44:49.466 INFO:tasks.workunit.client.0.smithi076.stdout:5/128: mkdir d1a/d33 0 2024-03-23T15:44:49.467 INFO:tasks.workunit.client.0.smithi076.stdout:6/107: mknod d1/d4/c25 0 2024-03-23T15:44:49.468 INFO:tasks.workunit.client.0.smithi076.stdout:6/108: rename d1/d4/f12 to d1/dd/f26 0 2024-03-23T15:44:49.469 INFO:tasks.workunit.client.0.smithi076.stdout:6/109: read d1/f21 [58964,130408] 0 2024-03-23T15:44:49.471 INFO:tasks.workunit.client.0.smithi076.stdout:6/110: mkdir d1/d2/d27 0 2024-03-23T15:44:49.471 INFO:tasks.workunit.client.0.smithi076.stdout:6/111: fdatasync d1/d4/d10/d18/f1b 0 2024-03-23T15:44:49.483 INFO:tasks.workunit.client.0.smithi076.stdout:0/141: dwrite d2/d8/f25 [0,4194304] 0 2024-03-23T15:44:49.485 INFO:tasks.workunit.client.0.smithi076.stdout:0/142: creat d2/de/f2c x:0 0 0 2024-03-23T15:44:49.486 INFO:tasks.workunit.client.0.smithi076.stdout:0/143: mkdir d2/d8/d2d 0 2024-03-23T15:44:49.488 INFO:tasks.workunit.client.0.smithi076.stdout:0/144: unlink d2/lf 0 2024-03-23T15:44:49.490 INFO:tasks.workunit.client.0.smithi076.stdout:0/145: link d2/de/d10/c15 d2/d8/d2d/c2e 0 2024-03-23T15:44:49.493 INFO:tasks.workunit.client.0.smithi076.stdout:6/112: dwrite d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:49.493 INFO:tasks.workunit.client.0.smithi076.stdout:6/113: read - d1/dd/df/f13 zero size 2024-03-23T15:44:49.493 INFO:tasks.workunit.client.0.smithi076.stdout:6/114: dread - d1/dd/df/f13 zero size 2024-03-23T15:44:49.493 INFO:tasks.workunit.client.0.smithi076.stdout:6/115: readlink d1/l14 0 2024-03-23T15:44:49.493 INFO:tasks.workunit.client.0.smithi076.stdout:6/116: getdents d1/d2/d27 0 2024-03-23T15:44:49.494 INFO:tasks.workunit.client.0.smithi076.stdout:6/117: mkdir d1/d2/d27/d28 0 2024-03-23T15:44:49.495 INFO:tasks.workunit.client.0.smithi076.stdout:6/118: symlink d1/d4/d10/l29 0 2024-03-23T15:44:49.495 INFO:tasks.workunit.client.0.smithi076.stdout:6/119: write d1/d4/d10/d18/f1e [344061,124437] 0 2024-03-23T15:44:49.496 INFO:tasks.workunit.client.0.smithi076.stdout:6/120: rmdir d1/d4/d10 39 2024-03-23T15:44:49.496 INFO:tasks.workunit.client.0.smithi076.stdout:6/121: read d1/f21 [22859,96988] 0 2024-03-23T15:44:49.496 INFO:tasks.workunit.client.0.smithi076.stdout:6/122: stat d1/d4 0 2024-03-23T15:44:49.497 INFO:tasks.workunit.client.0.smithi076.stdout:0/146: dread d2/f4 [0,4194304] 0 2024-03-23T15:44:49.497 INFO:tasks.workunit.client.0.smithi076.stdout:0/147: chown d2/c1c 26 1 2024-03-23T15:44:49.497 INFO:tasks.workunit.client.0.smithi076.stdout:0/148: write d2/f9 [1246194,42144] 0 2024-03-23T15:44:49.497 INFO:tasks.workunit.client.0.smithi076.stdout:0/149: dread - d2/d8/f27 zero size 2024-03-23T15:44:49.497 INFO:tasks.workunit.client.0.smithi076.stdout:0/150: creat d2/d8/d2d/f2f x:0 0 0 2024-03-23T15:44:49.498 INFO:tasks.workunit.client.0.smithi076.stdout:0/151: mknod d2/d1b/c30 0 2024-03-23T15:44:49.498 INFO:tasks.workunit.client.0.smithi076.stdout:0/152: write d2/d8/f28 [202925,119446] 0 2024-03-23T15:44:49.504 INFO:tasks.workunit.client.0.smithi076.stdout:0/153: dread d2/de/f14 [0,4194304] 0 2024-03-23T15:44:49.504 INFO:tasks.workunit.client.0.smithi076.stdout:0/154: chown d2/de/d10/d12/f23 2909681 1 2024-03-23T15:44:49.504 INFO:tasks.workunit.client.0.smithi076.stdout:0/155: stat d2/d8/d2d/f2f 0 2024-03-23T15:44:49.504 INFO:tasks.workunit.client.0.smithi076.stdout:0/156: read - d2/de/d10/d12/f23 zero size 2024-03-23T15:44:49.560 INFO:tasks.workunit.client.0.smithi076.stdout:0/157: dwrite d2/d1b/f1e [0,4194304] 0 2024-03-23T15:44:49.560 INFO:tasks.workunit.client.0.smithi076.stdout:0/158: fsync d2/d8/f28 0 2024-03-23T15:44:49.561 INFO:tasks.workunit.client.0.smithi076.stdout:6/123: rename d1/dd/d19 to d1/dd/d2a 0 2024-03-23T15:44:49.565 INFO:tasks.workunit.client.0.smithi076.stdout:6/124: creat d1/f2b x:0 0 0 2024-03-23T15:44:49.573 INFO:tasks.workunit.client.0.smithi076.stdout:0/159: dwrite d2/d8/d2d/f2f [0,4194304] 0 2024-03-23T15:44:49.577 INFO:tasks.workunit.client.0.smithi076.stdout:0/160: rename d2/d1b/c30 to d2/d8/d2d/c31 0 2024-03-23T15:44:49.577 INFO:tasks.workunit.client.0.smithi076.stdout:0/161: readlink d2/de/d10/d12/l2b 0 2024-03-23T15:44:49.584 INFO:tasks.workunit.client.0.smithi076.stdout:0/162: write d2/d8/d2d/f2f [2481501,122111] 0 2024-03-23T15:44:49.585 INFO:tasks.workunit.client.0.smithi076.stdout:0/163: write d2/de/d10/d12/f23 [1014546,75420] 0 2024-03-23T15:44:49.585 INFO:tasks.workunit.client.0.smithi076.stdout:0/164: readlink d2/de/d10/d12/l26 0 2024-03-23T15:44:49.585 INFO:tasks.workunit.client.0.smithi076.stdout:0/165: stat d2/d1b 0 2024-03-23T15:44:49.585 INFO:tasks.workunit.client.0.smithi076.stdout:6/125: getdents d1/d4/d10 0 2024-03-23T15:44:49.586 INFO:tasks.workunit.client.0.smithi076.stdout:0/166: mkdir d2/d8/d32 0 2024-03-23T15:44:49.589 INFO:tasks.workunit.client.0.smithi076.stdout:6/126: rename d1/d4/d10/d18/f1b to d1/f2c 0 2024-03-23T15:44:49.589 INFO:tasks.workunit.client.0.smithi076.stdout:6/127: truncate d1/f2b 698294 0 2024-03-23T15:44:49.617 INFO:tasks.workunit.client.0.smithi076.stdout:1/58: sync 2024-03-23T15:44:49.633 INFO:tasks.workunit.client.0.smithi076.stdout:1/59: dwrite d0/f8 [0,4194304] 0 2024-03-23T15:44:49.634 INFO:tasks.workunit.client.0.smithi076.stdout:1/60: chown d0/d2/c7 280792 1 2024-03-23T15:44:49.635 INFO:tasks.workunit.client.0.smithi076.stdout:1/61: link d0/d1/cb d0/d2/ce 0 2024-03-23T15:44:49.640 INFO:tasks.workunit.client.0.smithi076.stdout:1/62: dread d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:49.642 INFO:tasks.workunit.client.0.smithi076.stdout:1/63: mknod d0/d1/d6/cf 0 2024-03-23T15:44:49.643 INFO:tasks.workunit.client.0.smithi076.stdout:1/64: symlink d0/d1/l10 0 2024-03-23T15:44:49.645 INFO:tasks.workunit.client.0.smithi076.stdout:1/65: symlink d0/l11 0 2024-03-23T15:44:49.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/66: dwrite d0/f8 [0,4194304] 0 2024-03-23T15:44:49.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/67: truncate d0/f8 1082015 0 2024-03-23T15:44:49.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/68: write d0/d1/d6/f9 [4948273,78400] 0 2024-03-23T15:44:49.669 INFO:tasks.workunit.client.0.smithi076.stdout:1/69: symlink d0/d1/l12 0 2024-03-23T15:44:49.670 INFO:tasks.workunit.client.0.smithi076.stdout:1/70: write d0/f8 [2027408,48034] 0 2024-03-23T15:44:49.671 INFO:tasks.workunit.client.0.smithi076.stdout:1/71: readlink d0/l11 0 2024-03-23T15:44:49.671 INFO:tasks.workunit.client.0.smithi076.stdout:1/72: symlink d0/l13 0 2024-03-23T15:44:49.673 INFO:tasks.workunit.client.0.smithi076.stdout:1/73: fsync d0/d1/d6/f9 0 2024-03-23T15:44:49.673 INFO:tasks.workunit.client.0.smithi076.stdout:1/74: chown d0/l11 353507299 1 2024-03-23T15:44:49.676 INFO:tasks.workunit.client.0.smithi076.stdout:1/75: mknod d0/d2/c14 0 2024-03-23T15:44:49.693 INFO:tasks.workunit.client.0.smithi076.stdout:2/78: sync 2024-03-23T15:44:49.718 INFO:tasks.workunit.client.0.smithi076.stdout:2/79: dwrite d1/d4/d12/f18 [0,4194304] 0 2024-03-23T15:44:49.722 INFO:tasks.workunit.client.0.smithi076.stdout:2/80: getdents d1/d4 0 2024-03-23T15:44:49.722 INFO:tasks.workunit.client.0.smithi076.stdout:2/81: chown d1/lf 2012 1 2024-03-23T15:44:49.722 INFO:tasks.workunit.client.0.smithi076.stdout:2/82: chown d1/c3 2 1 2024-03-23T15:44:49.723 INFO:tasks.workunit.client.0.smithi076.stdout:2/83: creat d1/f19 x:0 0 0 2024-03-23T15:44:49.725 INFO:tasks.workunit.client.0.smithi076.stdout:2/84: truncate d1/d4/d12/f18 1257820 0 2024-03-23T15:44:49.726 INFO:tasks.workunit.client.0.smithi076.stdout:2/85: dread d1/d4/db/fc [0,4194304] 0 2024-03-23T15:44:49.727 INFO:tasks.workunit.client.0.smithi076.stdout:2/86: chown d1/d4/db 11621 1 2024-03-23T15:44:49.727 INFO:tasks.workunit.client.0.smithi076.stdout:2/87: dread d1/f14 [0,4194304] 0 2024-03-23T15:44:49.739 INFO:tasks.workunit.client.0.smithi076.stdout:2/88: dwrite d1/d4/f9 [0,4194304] 0 2024-03-23T15:44:49.739 INFO:tasks.workunit.client.0.smithi076.stdout:2/89: stat d1/d4/db/d15 0 2024-03-23T15:44:49.739 INFO:tasks.workunit.client.0.smithi076.stdout:2/90: readlink d1/d4/l11 0 2024-03-23T15:44:49.741 INFO:tasks.workunit.client.0.smithi076.stdout:2/91: creat d1/d4/db/d15/f1a x:0 0 0 2024-03-23T15:44:49.742 INFO:tasks.workunit.client.0.smithi076.stdout:2/92: mkdir d1/d4/d12/d1b 0 2024-03-23T15:44:49.747 INFO:tasks.workunit.client.0.smithi076.stdout:2/93: read d1/d4/f7 [2387855,77628] 0 2024-03-23T15:44:49.748 INFO:tasks.workunit.client.0.smithi076.stdout:2/94: creat d1/d4/d12/d1b/f1c x:0 0 0 2024-03-23T15:44:49.748 INFO:tasks.workunit.client.0.smithi076.stdout:2/95: write d1/d4/db/fc [1552301,57960] 0 2024-03-23T15:44:49.759 INFO:tasks.workunit.client.0.smithi076.stdout:2/96: rmdir d1 39 2024-03-23T15:44:49.759 INFO:tasks.workunit.client.0.smithi076.stdout:2/97: chown d1/lf 58 1 2024-03-23T15:44:49.760 INFO:tasks.workunit.client.0.smithi076.stdout:2/98: mkdir d1/d4/d12/d1d 0 2024-03-23T15:44:49.761 INFO:tasks.workunit.client.0.smithi076.stdout:2/99: creat d1/d4/f1e x:0 0 0 2024-03-23T15:44:49.761 INFO:tasks.workunit.client.0.smithi076.stdout:2/100: stat d1/lf 0 2024-03-23T15:44:49.762 INFO:tasks.workunit.client.0.smithi076.stdout:2/101: creat d1/d4/db/d15/f1f x:0 0 0 2024-03-23T15:44:49.762 INFO:tasks.workunit.client.0.smithi076.stdout:2/102: dread - d1/d4/db/d15/f1a zero size 2024-03-23T15:44:49.762 INFO:tasks.workunit.client.0.smithi076.stdout:2/103: truncate d1/d4/f1e 388328 0 2024-03-23T15:44:49.771 INFO:tasks.workunit.client.0.smithi076.stdout:8/97: sync 2024-03-23T15:44:49.771 INFO:tasks.workunit.client.0.smithi076.stdout:3/138: sync 2024-03-23T15:44:49.771 INFO:tasks.workunit.client.0.smithi076.stdout:7/100: sync 2024-03-23T15:44:49.771 INFO:tasks.workunit.client.0.smithi076.stdout:5/129: sync 2024-03-23T15:44:49.771 INFO:tasks.workunit.client.0.smithi076.stdout:9/134: sync 2024-03-23T15:44:49.771 INFO:tasks.workunit.client.0.smithi076.stdout:3/139: chown d5 687969936 1 2024-03-23T15:44:49.771 INFO:tasks.workunit.client.0.smithi076.stdout:4/163: sync 2024-03-23T15:44:49.772 INFO:tasks.workunit.client.0.smithi076.stdout:5/130: readlink lf 0 2024-03-23T15:44:49.772 INFO:tasks.workunit.client.0.smithi076.stdout:8/98: dread - d8/fe zero size 2024-03-23T15:44:49.772 INFO:tasks.workunit.client.0.smithi076.stdout:0/167: sync 2024-03-23T15:44:49.772 INFO:tasks.workunit.client.0.smithi076.stdout:9/135: chown d1/f18 26986 1 2024-03-23T15:44:49.773 INFO:tasks.workunit.client.0.smithi076.stdout:5/131: unlink d1a/d20/f32 0 2024-03-23T15:44:49.773 INFO:tasks.workunit.client.0.smithi076.stdout:5/132: write f4 [4550204,3943] 0 2024-03-23T15:44:49.774 INFO:tasks.workunit.client.0.smithi076.stdout:3/140: creat d5/f28 x:0 0 0 2024-03-23T15:44:49.775 INFO:tasks.workunit.client.0.smithi076.stdout:7/101: dread f3 [0,4194304] 0 2024-03-23T15:44:49.775 INFO:tasks.workunit.client.0.smithi076.stdout:2/104: dwrite d1/d4/f9 [0,4194304] 0 2024-03-23T15:44:49.775 INFO:tasks.workunit.client.0.smithi076.stdout:2/105: fdatasync d1/d4/db/d15/f1f 0 2024-03-23T15:44:49.776 INFO:tasks.workunit.client.0.smithi076.stdout:2/106: fsync d1/d4/db/fc 0 2024-03-23T15:44:49.776 INFO:tasks.workunit.client.0.smithi076.stdout:4/164: mknod d8/d9/d10/d15/c35 0 2024-03-23T15:44:49.776 INFO:tasks.workunit.client.0.smithi076.stdout:7/102: rmdir d8/d11 39 2024-03-23T15:44:49.776 INFO:tasks.workunit.client.0.smithi076.stdout:9/136: mknod d1/c27 0 2024-03-23T15:44:49.778 INFO:tasks.workunit.client.0.smithi076.stdout:5/133: creat d1a/d20/f34 x:0 0 0 2024-03-23T15:44:49.780 INFO:tasks.workunit.client.0.smithi076.stdout:4/165: truncate d8/ff 3896671 0 2024-03-23T15:44:49.780 INFO:tasks.workunit.client.0.smithi076.stdout:5/134: fdatasync f4 0 2024-03-23T15:44:49.781 INFO:tasks.workunit.client.0.smithi076.stdout:4/166: link d8/f30 d8/f36 0 2024-03-23T15:44:49.781 INFO:tasks.workunit.client.0.smithi076.stdout:4/167: chown d8/d9/l2e 352 1 2024-03-23T15:44:49.781 INFO:tasks.workunit.client.0.smithi076.stdout:4/168: fdatasync d8/ff 0 2024-03-23T15:44:49.783 INFO:tasks.workunit.client.0.smithi076.stdout:5/135: creat d1a/d33/f35 x:0 0 0 2024-03-23T15:44:49.786 INFO:tasks.workunit.client.0.smithi076.stdout:4/169: symlink d8/d9/d32/l37 0 2024-03-23T15:44:49.786 INFO:tasks.workunit.client.0.smithi076.stdout:4/170: truncate d8/d9/d10/d15/d18/f21 5125104 0 2024-03-23T15:44:49.786 INFO:tasks.workunit.client.0.smithi076.stdout:4/171: chown d8/d9/d10/d15/f34 1 1 2024-03-23T15:44:49.786 INFO:tasks.workunit.client.0.smithi076.stdout:4/172: chown d8/d9/d1d/l29 669325677 1 2024-03-23T15:44:49.786 INFO:tasks.workunit.client.0.smithi076.stdout:4/173: fdatasync d8/d9/fc 0 2024-03-23T15:44:49.789 INFO:tasks.workunit.client.0.smithi076.stdout:5/136: getdents d1a 0 2024-03-23T15:44:49.789 INFO:tasks.workunit.client.0.smithi076.stdout:5/137: chown d1a/d33 1 1 2024-03-23T15:44:49.798 INFO:tasks.workunit.client.0.smithi076.stdout:8/99: dwrite d8/fe [0,4194304] 0 2024-03-23T15:44:49.798 INFO:tasks.workunit.client.0.smithi076.stdout:0/168: dread d2/de/f1a [0,4194304] 0 2024-03-23T15:44:49.799 INFO:tasks.workunit.client.0.smithi076.stdout:7/103: write f0 [881946,103525] 0 2024-03-23T15:44:49.799 INFO:tasks.workunit.client.0.smithi076.stdout:7/104: dread - d8/fb zero size 2024-03-23T15:44:49.799 INFO:tasks.workunit.client.0.smithi076.stdout:4/174: write d8/d9/d10/f22 [211283,88889] 0 2024-03-23T15:44:49.801 INFO:tasks.workunit.client.0.smithi076.stdout:9/137: dwrite d1/f19 [0,4194304] 0 2024-03-23T15:44:49.801 INFO:tasks.workunit.client.0.smithi076.stdout:2/107: dwrite d1/f5 [0,4194304] 0 2024-03-23T15:44:49.801 INFO:tasks.workunit.client.0.smithi076.stdout:9/138: write d1/d1f/f21 [765997,74652] 0 2024-03-23T15:44:49.801 INFO:tasks.workunit.client.0.smithi076.stdout:8/100: rmdir d8 39 2024-03-23T15:44:49.803 INFO:tasks.workunit.client.0.smithi076.stdout:0/169: mknod d2/c33 0 2024-03-23T15:44:49.803 INFO:tasks.workunit.client.0.smithi076.stdout:2/108: creat d1/d4/d12/f20 x:0 0 0 2024-03-23T15:44:49.805 INFO:tasks.workunit.client.0.smithi076.stdout:4/175: mknod d8/d9/c38 0 2024-03-23T15:44:49.805 INFO:tasks.workunit.client.0.smithi076.stdout:4/176: fsync d8/d9/f2d 0 2024-03-23T15:44:49.805 INFO:tasks.workunit.client.0.smithi076.stdout:4/177: truncate d8/f28 1912562 0 2024-03-23T15:44:49.808 INFO:tasks.workunit.client.0.smithi076.stdout:9/139: link d1/f26 d1/d20/f28 0 2024-03-23T15:44:49.808 INFO:tasks.workunit.client.0.smithi076.stdout:5/138: dwrite f5 [0,4194304] 0 2024-03-23T15:44:49.808 INFO:tasks.workunit.client.0.smithi076.stdout:5/139: chown d1a 3753 1 2024-03-23T15:44:49.808 INFO:tasks.workunit.client.0.smithi076.stdout:5/140: fsync d1a/d20/f30 0 2024-03-23T15:44:49.809 INFO:tasks.workunit.client.0.smithi076.stdout:9/140: write d1/d1f/f21 [318908,32209] 0 2024-03-23T15:44:49.810 INFO:tasks.workunit.client.0.smithi076.stdout:0/170: rename d2/d8/d2d/c2e to d2/de/d10/c34 0 2024-03-23T15:44:49.810 INFO:tasks.workunit.client.0.smithi076.stdout:0/171: fsync d2/de/f13 0 2024-03-23T15:44:49.812 INFO:tasks.workunit.client.0.smithi076.stdout:5/141: mkdir d1a/d33/d36 0 2024-03-23T15:44:49.812 INFO:tasks.workunit.client.0.smithi076.stdout:5/142: dread - f19 zero size 2024-03-23T15:44:49.814 INFO:tasks.workunit.client.0.smithi076.stdout:5/143: mkdir d1a/d33/d37 0 2024-03-23T15:44:49.815 INFO:tasks.workunit.client.0.smithi076.stdout:3/141: dwrite d5/f11 [0,4194304] 0 2024-03-23T15:44:49.819 INFO:tasks.workunit.client.0.smithi076.stdout:9/141: dread d1/f1a [0,4194304] 0 2024-03-23T15:44:49.819 INFO:tasks.workunit.client.0.smithi076.stdout:9/142: fsync d1/f18 0 2024-03-23T15:44:49.819 INFO:tasks.workunit.client.0.smithi076.stdout:9/143: write d1/f18 [2040807,8790] 0 2024-03-23T15:44:49.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: pgmap v47: 97 pgs: 97 active+clean; 97 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 2.03k op/s 2024-03-23T15:44:49.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025df (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_child.c) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025e0 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025e1 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_parent.c) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025e2 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025f6 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/selinux_rxdir.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025f7 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025d8 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/Makefile) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025d9 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_noioctl.c) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025da (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.c) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: Scrub error on inode 0x100000025d7 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.sh) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.823 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.824 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.824 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: active paths [/] 2024-03-23T15:44:49.824 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:49 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.825 INFO:tasks.workunit.client.0.smithi076.stdout:8/101: dwrite f3 [0,4194304] 0 2024-03-23T15:44:49.825 INFO:tasks.workunit.client.0.smithi076.stdout:8/102: write d8/f19 [4805332,4145] 0 2024-03-23T15:44:49.825 INFO:tasks.workunit.client.0.smithi076.stdout:3/142: creat d5/d17/f29 x:0 0 0 2024-03-23T15:44:49.825 INFO:tasks.workunit.client.0.smithi076.stdout:7/105: truncate f0 1332269 0 2024-03-23T15:44:49.825 INFO:tasks.workunit.client.0.smithi076.stdout:9/144: mkdir d1/d1f/d29 0 2024-03-23T15:44:49.825 INFO:tasks.workunit.client.0.smithi076.stdout:2/109: dwrite d1/d4/f9 [0,4194304] 0 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: pgmap v47: 97 pgs: 97 active+clean; 97 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 2.03k op/s 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025df (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_child.c) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025e0 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025e1 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_parent.c) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025e2 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025f6 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/selinux_rxdir.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025f7 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025d8 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/Makefile) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025d9 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_noioctl.c) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025da (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.c) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: Scrub error on inode 0x100000025d7 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.sh) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.827 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.828 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: active paths [/] 2024-03-23T15:44:49.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:49 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.830 INFO:tasks.workunit.client.0.smithi076.stdout:0/172: dwrite d2/f4 [4194304,4194304] 0 2024-03-23T15:44:49.830 INFO:tasks.workunit.client.0.smithi076.stdout:0/173: fsync d2/de/f22 0 2024-03-23T15:44:49.831 INFO:tasks.workunit.client.0.smithi076.stdout:5/144: dread fd [0,4194304] 0 2024-03-23T15:44:49.831 INFO:tasks.workunit.client.0.smithi076.stdout:5/145: write d1a/d20/f30 [597023,15550] 0 2024-03-23T15:44:49.831 INFO:tasks.workunit.client.0.smithi076.stdout:5/146: truncate d1a/d2c/f2e 442948 0 2024-03-23T15:44:49.831 INFO:tasks.workunit.client.0.smithi076.stdout:2/110: truncate d1/d4/f9 1474032 0 2024-03-23T15:44:49.831 INFO:tasks.workunit.client.0.smithi076.stdout:8/103: creat d8/f23 x:0 0 0 2024-03-23T15:44:49.833 INFO:tasks.workunit.client.0.smithi076.stdout:3/143: creat d5/f2a x:0 0 0 2024-03-23T15:44:49.833 INFO:tasks.workunit.client.0.smithi076.stdout:3/144: truncate d5/f1f 18552 0 2024-03-23T15:44:49.833 INFO:tasks.workunit.client.0.smithi076.stdout:2/111: rename d1/d4/db/d15/f1a to d1/d4/db/d15/f21 0 2024-03-23T15:44:49.833 INFO:tasks.workunit.client.0.smithi076.stdout:2/112: chown d1/d4/d12 27 1 2024-03-23T15:44:49.833 INFO:tasks.workunit.client.0.smithi076.stdout:2/113: dread - d1/d4/d12/f20 zero size 2024-03-23T15:44:49.833 INFO:tasks.workunit.client.0.smithi076.stdout:4/178: dwrite d8/d9/f12 [0,4194304] 0 2024-03-23T15:44:49.835 INFO:tasks.workunit.client.0.smithi076.stdout:0/174: dread d2/de/f22 [0,4194304] 0 2024-03-23T15:44:49.835 INFO:tasks.workunit.client.0.smithi076.stdout:9/145: rename d1/cf to d1/d1f/d29/c2a 0 2024-03-23T15:44:49.838 INFO:tasks.workunit.client.0.smithi076.stdout:8/104: symlink d8/l24 0 2024-03-23T15:44:49.838 INFO:tasks.workunit.client.0.smithi076.stdout:8/105: write d8/f14 [662728,31058] 0 2024-03-23T15:44:49.838 INFO:tasks.workunit.client.0.smithi076.stdout:6/128: sync 2024-03-23T15:44:49.840 INFO:tasks.workunit.client.0.smithi076.stdout:3/145: creat d5/d17/d1c/f2b x:0 0 0 2024-03-23T15:44:49.844 INFO:tasks.workunit.client.0.smithi076.stdout:4/179: rename d8/d9/d10/f22 to d8/d9/d10/f39 0 2024-03-23T15:44:49.844 INFO:tasks.workunit.client.0.smithi076.stdout:1/76: sync 2024-03-23T15:44:49.846 INFO:tasks.workunit.client.0.smithi076.stdout:0/175: mknod d2/de/d10/c35 0 2024-03-23T15:44:49.846 INFO:tasks.workunit.client.0.smithi076.stdout:5/147: dwrite f11 [0,4194304] 0 2024-03-23T15:44:49.846 INFO:tasks.workunit.client.0.smithi076.stdout:5/148: write f18 [1167142,128446] 0 2024-03-23T15:44:49.846 INFO:tasks.workunit.client.0.smithi076.stdout:5/149: write f18 [2084809,10878] 0 2024-03-23T15:44:49.847 INFO:tasks.workunit.client.0.smithi076.stdout:9/146: rmdir d1/d20 39 2024-03-23T15:44:49.849 INFO:tasks.workunit.client.0.smithi076.stdout:7/106: truncate f1 1083142 0 2024-03-23T15:44:49.849 INFO:tasks.workunit.client.0.smithi076.stdout:3/146: symlink d5/d27/l2c 0 2024-03-23T15:44:49.851 INFO:tasks.workunit.client.0.smithi076.stdout:4/180: getdents d8/d9/d10/d15 0 2024-03-23T15:44:49.851 INFO:tasks.workunit.client.0.smithi076.stdout:4/181: fsync d8/d9/d10/d15/d18/f2b 0 2024-03-23T15:44:49.852 INFO:tasks.workunit.client.0.smithi076.stdout:2/114: dwrite d1/d4/f1e [0,4194304] 0 2024-03-23T15:44:49.853 INFO:tasks.workunit.client.0.smithi076.stdout:1/77: mknod d0/c15 0 2024-03-23T15:44:49.854 INFO:tasks.workunit.client.0.smithi076.stdout:8/106: dread d8/f1d [0,4194304] 0 2024-03-23T15:44:49.854 INFO:tasks.workunit.client.0.smithi076.stdout:8/107: readlink d8/l1f 0 2024-03-23T15:44:49.854 INFO:tasks.workunit.client.0.smithi076.stdout:8/108: fsync d8/f1d 0 2024-03-23T15:44:49.854 INFO:tasks.workunit.client.0.smithi076.stdout:8/109: stat d8/f1d 0 2024-03-23T15:44:49.855 INFO:tasks.workunit.client.0.smithi076.stdout:0/176: mknod d2/d1b/c36 0 2024-03-23T15:44:49.855 INFO:tasks.workunit.client.0.smithi076.stdout:2/115: mknod d1/d4/d12/c22 0 2024-03-23T15:44:49.855 INFO:tasks.workunit.client.0.smithi076.stdout:2/116: dread - d1/d4/d12/d1b/f1c zero size 2024-03-23T15:44:49.855 INFO:tasks.workunit.client.0.smithi076.stdout:2/117: rmdir d1/d4/db 39 2024-03-23T15:44:49.856 INFO:tasks.workunit.client.0.smithi076.stdout:5/150: symlink d1a/d2c/l38 0 2024-03-23T15:44:49.856 INFO:tasks.workunit.client.0.smithi076.stdout:9/147: rmdir d1/d20 39 2024-03-23T15:44:49.857 INFO:tasks.workunit.client.0.smithi076.stdout:2/118: creat d1/d4/d12/f23 x:0 0 0 2024-03-23T15:44:49.857 INFO:tasks.workunit.client.0.smithi076.stdout:8/110: write f7 [268810,105630] 0 2024-03-23T15:44:49.858 INFO:tasks.workunit.client.0.smithi076.stdout:3/147: creat d5/f2d x:0 0 0 2024-03-23T15:44:49.860 INFO:tasks.workunit.client.0.smithi076.stdout:6/129: dwrite d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:49.860 INFO:tasks.workunit.client.0.smithi076.stdout:2/119: write d1/d4/f9 [1450986,27071] 0 2024-03-23T15:44:49.861 INFO:tasks.workunit.client.0.smithi076.stdout:1/78: symlink d0/d1/l16 0 2024-03-23T15:44:49.861 INFO:tasks.workunit.client.0.smithi076.stdout:2/120: mknod d1/d4/db/c24 0 2024-03-23T15:44:49.863 INFO:tasks.workunit.client.0.smithi076.stdout:0/177: unlink d2/f11 0 2024-03-23T15:44:49.863 INFO:tasks.workunit.client.0.smithi076.stdout:0/178: chown d2/lc 1419 1 2024-03-23T15:44:49.863 INFO:tasks.workunit.client.0.smithi076.stdout:0/179: stat d2/d1b/c36 0 2024-03-23T15:44:49.863 INFO:tasks.workunit.client.0.smithi076.stdout:0/180: chown d2/de/d10/c15 384509 1 2024-03-23T15:44:49.864 INFO:tasks.workunit.client.0.smithi076.stdout:8/111: link d8/l22 d8/l25 0 2024-03-23T15:44:49.865 INFO:tasks.workunit.client.0.smithi076.stdout:3/148: stat d5/l20 0 2024-03-23T15:44:49.865 INFO:tasks.workunit.client.0.smithi076.stdout:3/149: chown d5/d17/d1c/f1e 22 1 2024-03-23T15:44:49.865 INFO:tasks.workunit.client.0.smithi076.stdout:3/150: truncate d5/f12 1013086 0 2024-03-23T15:44:49.866 INFO:tasks.workunit.client.0.smithi076.stdout:5/151: write f5 [1413129,121235] 0 2024-03-23T15:44:49.867 INFO:tasks.workunit.client.0.smithi076.stdout:4/182: rename d8/d9/d10/d15 to d8/d3a 0 2024-03-23T15:44:49.867 INFO:tasks.workunit.client.0.smithi076.stdout:4/183: dread - d8/d3a/d18/f20 zero size 2024-03-23T15:44:49.870 INFO:tasks.workunit.client.0.smithi076.stdout:0/181: symlink d2/de/d10/l37 0 2024-03-23T15:44:49.871 INFO:tasks.workunit.client.0.smithi076.stdout:8/112: mkdir d8/d26 0 2024-03-23T15:44:49.871 INFO:tasks.workunit.client.0.smithi076.stdout:8/113: chown d8/f13 1521051 1 2024-03-23T15:44:49.871 INFO:tasks.workunit.client.0.smithi076.stdout:8/114: chown f7 331 1 2024-03-23T15:44:49.872 INFO:tasks.workunit.client.0.smithi076.stdout:8/115: dread - d8/f1e zero size 2024-03-23T15:44:49.872 INFO:tasks.workunit.client.0.smithi076.stdout:8/116: getdents d8/d26 0 2024-03-23T15:44:49.872 INFO:tasks.workunit.client.0.smithi076.stdout:6/130: rmdir d1 39 2024-03-23T15:44:49.872 INFO:tasks.workunit.client.0.smithi076.stdout:1/79: write d0/f8 [1076551,8017] 0 2024-03-23T15:44:49.873 INFO:tasks.workunit.client.0.smithi076.stdout:7/107: truncate f0 803882 0 2024-03-23T15:44:49.873 INFO:tasks.workunit.client.0.smithi076.stdout:7/108: fdatasync f1 0 2024-03-23T15:44:49.873 INFO:tasks.workunit.client.0.smithi076.stdout:7/109: readlink d8/de/l1c 0 2024-03-23T15:44:49.873 INFO:tasks.workunit.client.0.smithi076.stdout:3/151: creat d5/d27/f2e x:0 0 0 2024-03-23T15:44:49.874 INFO:tasks.workunit.client.0.smithi076.stdout:7/110: creat d8/de/f1e x:0 0 0 2024-03-23T15:44:49.876 INFO:tasks.workunit.client.0.smithi076.stdout:0/182: symlink d2/d1b/l38 0 2024-03-23T15:44:49.877 INFO:tasks.workunit.client.0.smithi076.stdout:7/111: rmdir d8/de/d13 39 2024-03-23T15:44:49.878 INFO:tasks.workunit.client.0.smithi076.stdout:2/121: dread d1/f5 [0,4194304] 0 2024-03-23T15:44:49.878 INFO:tasks.workunit.client.0.smithi076.stdout:6/131: creat d1/dd/d2a/d23/f2d x:0 0 0 2024-03-23T15:44:49.880 INFO:tasks.workunit.client.0.smithi076.stdout:7/112: truncate d8/fa 931744 0 2024-03-23T15:44:49.882 INFO:tasks.workunit.client.0.smithi076.stdout:9/148: dwrite d1/f18 [0,4194304] 0 2024-03-23T15:44:49.882 INFO:tasks.workunit.client.0.smithi076.stdout:9/149: write d1/f19 [5082297,95448] 0 2024-03-23T15:44:49.882 INFO:tasks.workunit.client.0.smithi076.stdout:1/80: mkdir d0/d1/d6/d17 0 2024-03-23T15:44:49.882 INFO:tasks.workunit.client.0.smithi076.stdout:7/113: symlink d8/de/d13/l1f 0 2024-03-23T15:44:49.883 INFO:tasks.workunit.client.0.smithi076.stdout:7/114: fsync f1 0 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: pgmap v47: 97 pgs: 97 active+clean; 97 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 2.03k op/s 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025df (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_child.c) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025e0 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025e1 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_parent.c) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025e2 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025f6 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/selinux_rxdir.sh) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025f7 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/Makefile) see mds.i log and `damage ls` output for details 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025d8 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/Makefile) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025d9 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_noioctl.c) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025da (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.c) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: Scrub error on inode 0x100000025d7 (/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.sh) see mds.h log and `damage ls` output for details 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.886 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.887 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.887 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.887 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.887 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.887 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: active paths [/] 2024-03-23T15:44:49.887 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:49 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:49.888 INFO:tasks.workunit.client.0.smithi076.stdout:3/152: symlink d5/l2f 0 2024-03-23T15:44:49.888 INFO:tasks.workunit.client.0.smithi076.stdout:7/115: creat d8/de/d13/d18/d1a/f20 x:0 0 0 2024-03-23T15:44:49.888 INFO:tasks.workunit.client.0.smithi076.stdout:7/116: fsync f1 0 2024-03-23T15:44:49.888 INFO:tasks.workunit.client.0.smithi076.stdout:9/150: rename d1/l1d to d1/l2b 0 2024-03-23T15:44:49.888 INFO:tasks.workunit.client.0.smithi076.stdout:9/151: stat d1/d20/f28 0 2024-03-23T15:44:49.889 INFO:tasks.workunit.client.0.smithi076.stdout:1/81: rename d0/d2/c14 to d0/d2/c18 0 2024-03-23T15:44:49.890 INFO:tasks.workunit.client.0.smithi076.stdout:5/152: dwrite f7 [0,4194304] 0 2024-03-23T15:44:49.895 INFO:tasks.workunit.client.0.smithi076.stdout:9/152: creat d1/d20/f2c x:0 0 0 2024-03-23T15:44:49.896 INFO:tasks.workunit.client.0.smithi076.stdout:1/82: mkdir d0/d1/d19 0 2024-03-23T15:44:49.900 INFO:tasks.workunit.client.0.smithi076.stdout:4/184: rename d8/d9/d10 to d8/d3a/d3b 0 2024-03-23T15:44:49.901 INFO:tasks.workunit.client.0.smithi076.stdout:9/153: rmdir d1/d20 39 2024-03-23T15:44:49.902 INFO:tasks.workunit.client.0.smithi076.stdout:1/83: rmdir d0 39 2024-03-23T15:44:49.904 INFO:tasks.workunit.client.0.smithi076.stdout:4/185: symlink d8/d3a/d18/d1b/l3c 0 2024-03-23T15:44:49.905 INFO:tasks.workunit.client.0.smithi076.stdout:7/117: dwrite d8/de/f1e [0,4194304] 0 2024-03-23T15:44:49.907 INFO:tasks.workunit.client.0.smithi076.stdout:8/117: dwrite d8/f19 [4194304,4194304] 0 2024-03-23T15:44:49.909 INFO:tasks.workunit.client.0.smithi076.stdout:7/118: rename d8/de/f1e to d8/de/d13/f21 0 2024-03-23T15:44:49.911 INFO:tasks.workunit.client.0.smithi076.stdout:7/119: creat d8/de/d13/d18/f22 x:0 0 0 2024-03-23T15:44:49.911 INFO:tasks.workunit.client.0.smithi076.stdout:2/122: write d1/d4/f10 [4153001,74534] 0 2024-03-23T15:44:49.911 INFO:tasks.workunit.client.0.smithi076.stdout:0/183: dwrite d2/f18 [0,4194304] 0 2024-03-23T15:44:49.912 INFO:tasks.workunit.client.0.smithi076.stdout:2/123: dread - d1/d4/d12/f23 zero size 2024-03-23T15:44:49.912 INFO:tasks.workunit.client.0.smithi076.stdout:7/120: dread d8/fd [0,4194304] 0 2024-03-23T15:44:49.913 INFO:tasks.workunit.client.0.smithi076.stdout:9/154: mkdir d1/d1f/d29/d2d 0 2024-03-23T15:44:49.913 INFO:tasks.workunit.client.0.smithi076.stdout:2/124: symlink d1/d4/d12/d1b/l25 0 2024-03-23T15:44:49.914 INFO:tasks.workunit.client.0.smithi076.stdout:7/121: rename d8/d11/c1b to d8/de/d13/d18/c23 0 2024-03-23T15:44:49.915 INFO:tasks.workunit.client.0.smithi076.stdout:1/84: link d0/l11 d0/d1/d6/d17/l1a 0 2024-03-23T15:44:49.915 INFO:tasks.workunit.client.0.smithi076.stdout:2/125: creat d1/d4/d12/f26 x:0 0 0 2024-03-23T15:44:49.916 INFO:tasks.workunit.client.0.smithi076.stdout:2/126: read - d1/d4/db/d15/f1f zero size 2024-03-23T15:44:49.916 INFO:tasks.workunit.client.0.smithi076.stdout:7/122: mkdir d8/de/d13/d24 0 2024-03-23T15:44:49.917 INFO:tasks.workunit.client.0.smithi076.stdout:7/123: readlink d8/l9 0 2024-03-23T15:44:49.918 INFO:tasks.workunit.client.0.smithi076.stdout:2/127: link d1/d4/l11 d1/d4/d12/d1b/l27 0 2024-03-23T15:44:49.919 INFO:tasks.workunit.client.0.smithi076.stdout:7/124: creat d8/d11/f25 x:0 0 0 2024-03-23T15:44:49.919 INFO:tasks.workunit.client.0.smithi076.stdout:2/128: mknod d1/d4/d12/d1d/c28 0 2024-03-23T15:44:49.919 INFO:tasks.workunit.client.0.smithi076.stdout:2/129: read - d1/d4/d12/f20 zero size 2024-03-23T15:44:49.919 INFO:tasks.workunit.client.0.smithi076.stdout:2/130: readlink d1/lf 0 2024-03-23T15:44:49.921 INFO:tasks.workunit.client.0.smithi076.stdout:2/131: mkdir d1/d4/db/d15/d29 0 2024-03-23T15:44:49.921 INFO:tasks.workunit.client.0.smithi076.stdout:2/132: readlink d1/l13 0 2024-03-23T15:44:49.921 INFO:tasks.workunit.client.0.smithi076.stdout:2/133: chown d1/d4/l11 109635294 1 2024-03-23T15:44:49.922 INFO:tasks.workunit.client.0.smithi076.stdout:8/118: read f7 [6446163,36077] 0 2024-03-23T15:44:49.926 INFO:tasks.workunit.client.0.smithi076.stdout:8/119: mkdir d8/d26/d27 0 2024-03-23T15:44:49.926 INFO:tasks.workunit.client.0.smithi076.stdout:8/120: fsync d8/f1a 0 2024-03-23T15:44:49.926 INFO:tasks.workunit.client.0.smithi076.stdout:4/186: dwrite f5 [4194304,4194304] 0 2024-03-23T15:44:49.928 INFO:tasks.workunit.client.0.smithi076.stdout:5/153: dwrite f18 [0,4194304] 0 2024-03-23T15:44:49.928 INFO:tasks.workunit.client.0.smithi076.stdout:5/154: write f12 [4514726,938] 0 2024-03-23T15:44:49.928 INFO:tasks.workunit.client.0.smithi076.stdout:8/121: symlink d8/d26/d27/l28 0 2024-03-23T15:44:49.928 INFO:tasks.workunit.client.0.smithi076.stdout:8/122: write d8/f23 [932084,17683] 0 2024-03-23T15:44:49.929 INFO:tasks.workunit.client.0.smithi076.stdout:2/134: dread d1/d4/d12/f18 [0,4194304] 0 2024-03-23T15:44:49.929 INFO:tasks.workunit.client.0.smithi076.stdout:2/135: rename d1 to d1/d4/d12/d1b/d2a 22 2024-03-23T15:44:49.929 INFO:tasks.workunit.client.0.smithi076.stdout:2/136: write d1/d4/db/d15/f21 [431628,88239] 0 2024-03-23T15:44:49.929 INFO:tasks.workunit.client.0.smithi076.stdout:4/187: getdents d8/d3a 0 2024-03-23T15:44:49.929 INFO:tasks.workunit.client.0.smithi076.stdout:4/188: truncate d8/f36 369121 0 2024-03-23T15:44:49.930 INFO:tasks.workunit.client.0.smithi076.stdout:4/189: chown d8/d3a/c35 2073982 1 2024-03-23T15:44:49.930 INFO:tasks.workunit.client.0.smithi076.stdout:2/137: mknod d1/d4/db/c2b 0 2024-03-23T15:44:49.932 INFO:tasks.workunit.client.0.smithi076.stdout:5/155: mkdir d1a/d39 0 2024-03-23T15:44:49.934 INFO:tasks.workunit.client.0.smithi076.stdout:0/184: dwrite d2/de/f1a [0,4194304] 0 2024-03-23T15:44:49.934 INFO:tasks.workunit.client.0.smithi076.stdout:8/123: link d8/l1b d8/d26/d27/l29 0 2024-03-23T15:44:49.936 INFO:tasks.workunit.client.0.smithi076.stdout:1/85: dwrite d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:49.938 INFO:tasks.workunit.client.0.smithi076.stdout:5/156: mkdir d1a/d33/d3a 0 2024-03-23T15:44:49.938 INFO:tasks.workunit.client.0.smithi076.stdout:5/157: chown cb 64336 1 2024-03-23T15:44:49.938 INFO:tasks.workunit.client.0.smithi076.stdout:7/125: dwrite d8/fb [0,4194304] 0 2024-03-23T15:44:49.938 INFO:tasks.workunit.client.0.smithi076.stdout:7/126: write d8/de/d13/d18/d1a/f20 [40067,18001] 0 2024-03-23T15:44:49.940 INFO:tasks.workunit.client.0.smithi076.stdout:7/127: creat d8/de/f26 x:0 0 0 2024-03-23T15:44:49.941 INFO:tasks.workunit.client.0.smithi076.stdout:0/185: creat d2/d8/d32/f39 x:0 0 0 2024-03-23T15:44:49.941 INFO:tasks.workunit.client.0.smithi076.stdout:2/138: mknod d1/c2c 0 2024-03-23T15:44:49.942 INFO:tasks.workunit.client.0.smithi076.stdout:7/128: truncate d8/fa 790719 0 2024-03-23T15:44:49.943 INFO:tasks.workunit.client.0.smithi076.stdout:0/186: mkdir d2/de/d10/d3a 0 2024-03-23T15:44:49.943 INFO:tasks.workunit.client.0.smithi076.stdout:0/187: chown d2/de 63565 1 2024-03-23T15:44:49.944 INFO:tasks.workunit.client.0.smithi076.stdout:2/139: getdents d1/d4/d12/d1b 0 2024-03-23T15:44:49.944 INFO:tasks.workunit.client.0.smithi076.stdout:2/140: read - d1/d4/d12/f20 zero size 2024-03-23T15:44:49.945 INFO:tasks.workunit.client.0.smithi076.stdout:0/188: symlink d2/de/d10/d3a/l3b 0 2024-03-23T15:44:49.945 INFO:tasks.workunit.client.0.smithi076.stdout:5/158: dread f5 [0,4194304] 0 2024-03-23T15:44:49.945 INFO:tasks.workunit.client.0.smithi076.stdout:5/159: fdatasync fc 0 2024-03-23T15:44:49.945 INFO:tasks.workunit.client.0.smithi076.stdout:2/141: mkdir d1/d4/d12/d2d 0 2024-03-23T15:44:49.946 INFO:tasks.workunit.client.0.smithi076.stdout:0/189: mknod d2/d1b/c3c 0 2024-03-23T15:44:49.946 INFO:tasks.workunit.client.0.smithi076.stdout:0/190: dread - d2/d8/f27 zero size 2024-03-23T15:44:49.946 INFO:tasks.workunit.client.0.smithi076.stdout:4/190: dwrite d8/d9/d1d/f24 [0,4194304] 0 2024-03-23T15:44:49.947 INFO:tasks.workunit.client.0.smithi076.stdout:0/191: fdatasync d2/d8/f27 0 2024-03-23T15:44:49.948 INFO:tasks.workunit.client.0.smithi076.stdout:5/160: creat d1a/d33/f3b x:0 0 0 2024-03-23T15:44:49.950 INFO:tasks.workunit.client.0.smithi076.stdout:2/142: mkdir d1/d2e 0 2024-03-23T15:44:49.951 INFO:tasks.workunit.client.0.smithi076.stdout:5/161: creat d1a/f3c x:0 0 0 2024-03-23T15:44:49.954 INFO:tasks.workunit.client.0.smithi076.stdout:5/162: creat d1a/d33/d36/f3d x:0 0 0 2024-03-23T15:44:49.954 INFO:tasks.workunit.client.0.smithi076.stdout:5/163: chown d1a/d2c/f2d 1855131 1 2024-03-23T15:44:49.956 INFO:tasks.workunit.client.0.smithi076.stdout:5/164: symlink d1a/d33/d3a/l3e 0 2024-03-23T15:44:49.956 INFO:tasks.workunit.client.0.smithi076.stdout:8/124: dwrite d8/f1e [0,4194304] 0 2024-03-23T15:44:49.958 INFO:tasks.workunit.client.0.smithi076.stdout:5/165: rmdir d1a/d33/d37 0 2024-03-23T15:44:49.962 INFO:tasks.workunit.client.0.smithi076.stdout:8/125: creat d8/f2a x:0 0 0 2024-03-23T15:44:49.966 INFO:tasks.workunit.client.0.smithi076.stdout:5/166: unlink lf 0 2024-03-23T15:44:49.969 INFO:tasks.workunit.client.0.smithi076.stdout:5/167: creat d1a/f3f x:0 0 0 2024-03-23T15:44:49.969 INFO:tasks.workunit.client.0.smithi076.stdout:8/126: dread d8/ff [0,4194304] 0 2024-03-23T15:44:49.971 INFO:tasks.workunit.client.0.smithi076.stdout:5/168: creat d1a/d33/d36/f40 x:0 0 0 2024-03-23T15:44:49.971 INFO:tasks.workunit.client.0.smithi076.stdout:5/169: dread - d1a/d2c/f2d zero size 2024-03-23T15:44:49.973 INFO:tasks.workunit.client.0.smithi076.stdout:5/170: creat d1a/d33/d3a/f41 x:0 0 0 2024-03-23T15:44:49.975 INFO:tasks.workunit.client.0.smithi076.stdout:7/129: dwrite d8/de/d13/f15 [0,4194304] 0 2024-03-23T15:44:49.975 INFO:tasks.workunit.client.0.smithi076.stdout:5/171: creat d1a/f42 x:0 0 0 2024-03-23T15:44:49.977 INFO:tasks.workunit.client.0.smithi076.stdout:5/172: creat d1a/d33/d3a/f43 x:0 0 0 2024-03-23T15:44:49.977 INFO:tasks.workunit.client.0.smithi076.stdout:5/173: readlink le 0 2024-03-23T15:44:49.979 INFO:tasks.workunit.client.0.smithi076.stdout:0/192: dwrite d2/f9 [0,4194304] 0 2024-03-23T15:44:49.979 INFO:tasks.workunit.client.0.smithi076.stdout:7/130: dread d8/fc [0,4194304] 0 2024-03-23T15:44:49.979 INFO:tasks.workunit.client.0.smithi076.stdout:5/174: creat d1a/d39/f44 x:0 0 0 2024-03-23T15:44:49.980 INFO:tasks.workunit.client.0.smithi076.stdout:5/175: dread d1a/d2c/f2e [0,4194304] 0 2024-03-23T15:44:49.980 INFO:tasks.workunit.client.0.smithi076.stdout:4/191: dwrite f2 [0,4194304] 0 2024-03-23T15:44:49.982 INFO:tasks.workunit.client.0.smithi076.stdout:8/127: write f3 [184767,50175] 0 2024-03-23T15:44:49.982 INFO:tasks.workunit.client.0.smithi076.stdout:2/143: dwrite d1/f19 [0,4194304] 0 2024-03-23T15:44:49.982 INFO:tasks.workunit.client.0.smithi076.stdout:4/192: link d8/f30 d8/f3d 0 2024-03-23T15:44:49.982 INFO:tasks.workunit.client.0.smithi076.stdout:4/193: dread - d8/d9/f2d zero size 2024-03-23T15:44:49.983 INFO:tasks.workunit.client.0.smithi076.stdout:4/194: chown d8/c23 10 1 2024-03-23T15:44:49.983 INFO:tasks.workunit.client.0.smithi076.stdout:4/195: chown d8/d9/c38 2117903 1 2024-03-23T15:44:49.983 INFO:tasks.workunit.client.0.smithi076.stdout:7/131: dread d8/fa [0,4194304] 0 2024-03-23T15:44:49.983 INFO:tasks.workunit.client.0.smithi076.stdout:8/128: mknod d8/d26/d27/c2b 0 2024-03-23T15:44:49.983 INFO:tasks.workunit.client.0.smithi076.stdout:8/129: write f7 [6971422,17363] 0 2024-03-23T15:44:49.984 INFO:tasks.workunit.client.0.smithi076.stdout:2/144: dread d1/d4/f9 [0,4194304] 0 2024-03-23T15:44:49.985 INFO:tasks.workunit.client.0.smithi076.stdout:4/196: mknod d8/d3a/c3e 0 2024-03-23T15:44:49.985 INFO:tasks.workunit.client.0.smithi076.stdout:7/132: symlink d8/l27 0 2024-03-23T15:44:49.986 INFO:tasks.workunit.client.0.smithi076.stdout:8/130: rename d8/c1c to d8/d26/d27/c2c 0 2024-03-23T15:44:49.988 INFO:tasks.workunit.client.0.smithi076.stdout:4/197: creat d8/f3f x:0 0 0 2024-03-23T15:44:49.989 INFO:tasks.workunit.client.0.smithi076.stdout:2/145: creat d1/f2f x:0 0 0 2024-03-23T15:44:49.989 INFO:tasks.workunit.client.0.smithi076.stdout:8/131: mknod d8/d26/c2d 0 2024-03-23T15:44:49.990 INFO:tasks.workunit.client.0.smithi076.stdout:8/132: chown d8/f11 31714011 1 2024-03-23T15:44:49.991 INFO:tasks.workunit.client.0.smithi076.stdout:2/146: rename d1/d4/f7 to d1/d4/db/d15/d29/f30 0 2024-03-23T15:44:49.991 INFO:tasks.workunit.client.0.smithi076.stdout:8/133: mkdir d8/d2e 0 2024-03-23T15:44:49.993 INFO:tasks.workunit.client.0.smithi076.stdout:2/147: readlink d1/ld 0 2024-03-23T15:44:49.993 INFO:tasks.workunit.client.0.smithi076.stdout:8/134: mknod d8/d2e/c2f 0 2024-03-23T15:44:49.993 INFO:tasks.workunit.client.0.smithi076.stdout:2/148: mknod d1/d4/d12/d1b/c31 0 2024-03-23T15:44:49.994 INFO:tasks.workunit.client.0.smithi076.stdout:8/135: mkdir d8/d2e/d30 0 2024-03-23T15:44:49.995 INFO:tasks.workunit.client.0.smithi076.stdout:2/149: dread d1/f14 [0,4194304] 0 2024-03-23T15:44:49.995 INFO:tasks.workunit.client.0.smithi076.stdout:8/136: mknod d8/d2e/d30/c31 0 2024-03-23T15:44:49.997 INFO:tasks.workunit.client.0.smithi076.stdout:4/198: dread f6 [0,4194304] 0 2024-03-23T15:44:49.997 INFO:tasks.workunit.client.0.smithi076.stdout:4/199: dread - d8/d9/f2d zero size 2024-03-23T15:44:49.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/133: write f1 [1106679,97841] 0 2024-03-23T15:44:49.998 INFO:tasks.workunit.client.0.smithi076.stdout:2/150: read d1/d4/d12/f18 [81299,2635] 0 2024-03-23T15:44:49.998 INFO:tasks.workunit.client.0.smithi076.stdout:4/200: link d8/d3a/c3e d8/d3a/d3b/c40 0 2024-03-23T15:44:49.998 INFO:tasks.workunit.client.0.smithi076.stdout:7/134: write d8/fc [661511,6641] 0 2024-03-23T15:44:49.999 INFO:tasks.workunit.client.0.smithi076.stdout:7/135: chown d8/de/d13/l1f 495896687 1 2024-03-23T15:44:49.999 INFO:tasks.workunit.client.0.smithi076.stdout:2/151: mkdir d1/d4/d12/d1b/d32 0 2024-03-23T15:44:49.999 INFO:tasks.workunit.client.0.smithi076.stdout:7/136: chown d8/de/c10 856455 1 2024-03-23T15:44:49.999 INFO:tasks.workunit.client.0.smithi076.stdout:4/201: mkdir d8/d41 0 2024-03-23T15:44:50.000 INFO:tasks.workunit.client.0.smithi076.stdout:4/202: chown d8/d9/d1d 9265822 1 2024-03-23T15:44:50.000 INFO:tasks.workunit.client.0.smithi076.stdout:7/137: creat d8/de/d13/d18/d1a/f28 x:0 0 0 2024-03-23T15:44:50.001 INFO:tasks.workunit.client.0.smithi076.stdout:0/193: dwrite d2/f18 [0,4194304] 0 2024-03-23T15:44:50.001 INFO:tasks.workunit.client.0.smithi076.stdout:0/194: truncate d2/d8/f25 4998154 0 2024-03-23T15:44:50.002 INFO:tasks.workunit.client.0.smithi076.stdout:7/138: creat d8/de/d13/d18/d1a/f29 x:0 0 0 2024-03-23T15:44:50.002 INFO:tasks.workunit.client.0.smithi076.stdout:7/139: dread - d8/d11/f19 zero size 2024-03-23T15:44:50.005 INFO:tasks.workunit.client.0.smithi076.stdout:0/195: unlink d2/de/d10/l37 0 2024-03-23T15:44:50.011 INFO:tasks.workunit.client.0.smithi076.stdout:0/196: link d2/de/d10/c15 d2/d8/c3d 0 2024-03-23T15:44:50.014 INFO:tasks.workunit.client.0.smithi076.stdout:2/152: dwrite d1/f2f [0,4194304] 0 2024-03-23T15:44:50.015 INFO:tasks.workunit.client.0.smithi076.stdout:2/153: mknod d1/d4/db/d15/d29/c33 0 2024-03-23T15:44:50.021 INFO:tasks.workunit.client.0.smithi076.stdout:8/137: truncate f3 2100877 0 2024-03-23T15:44:50.021 INFO:tasks.workunit.client.0.smithi076.stdout:8/138: fsync d8/f13 0 2024-03-23T15:44:50.023 INFO:tasks.workunit.client.0.smithi076.stdout:8/139: creat d8/d2e/d30/f32 x:0 0 0 2024-03-23T15:44:50.023 INFO:tasks.workunit.client.0.smithi076.stdout:8/140: dread - d8/f2a zero size 2024-03-23T15:44:50.023 INFO:tasks.workunit.client.0.smithi076.stdout:8/141: stat l6 0 2024-03-23T15:44:50.023 INFO:tasks.workunit.client.0.smithi076.stdout:8/142: write f2 [12869462,125240] 0 2024-03-23T15:44:50.023 INFO:tasks.workunit.client.0.smithi076.stdout:4/203: dwrite d8/d9/f17 [0,4194304] 0 2024-03-23T15:44:50.025 INFO:tasks.workunit.client.0.smithi076.stdout:0/197: getdents d2/de/d10 0 2024-03-23T15:44:50.028 INFO:tasks.workunit.client.0.smithi076.stdout:7/140: dwrite d8/fc [0,4194304] 0 2024-03-23T15:44:50.029 INFO:tasks.workunit.client.0.smithi076.stdout:2/154: dwrite d1/d4/d12/f20 [0,4194304] 0 2024-03-23T15:44:50.029 INFO:tasks.workunit.client.0.smithi076.stdout:7/141: rmdir d8/de/d13/d18 39 2024-03-23T15:44:50.032 INFO:tasks.workunit.client.0.smithi076.stdout:2/155: mknod d1/d4/d12/c34 0 2024-03-23T15:44:50.034 INFO:tasks.workunit.client.0.smithi076.stdout:7/142: creat d8/de/d13/d18/d1a/f2a x:0 0 0 2024-03-23T15:44:50.034 INFO:tasks.workunit.client.0.smithi076.stdout:7/143: chown d8/de/d13/d18 3 1 2024-03-23T15:44:50.035 INFO:tasks.workunit.client.0.smithi076.stdout:2/156: mknod d1/d4/c35 0 2024-03-23T15:44:50.035 INFO:tasks.workunit.client.0.smithi076.stdout:2/157: readlink d1/lf 0 2024-03-23T15:44:50.036 INFO:tasks.workunit.client.0.smithi076.stdout:0/198: read d2/de/f22 [2748051,7824] 0 2024-03-23T15:44:50.036 INFO:tasks.workunit.client.0.smithi076.stdout:0/199: chown d2/l16 270248 1 2024-03-23T15:44:50.036 INFO:tasks.workunit.client.0.smithi076.stdout:0/200: write d2/de/f22 [4574242,77907] 0 2024-03-23T15:44:50.041 INFO:tasks.workunit.client.0.smithi076.stdout:0/201: link d2/de/d10/d12/l26 d2/l3e 0 2024-03-23T15:44:50.042 INFO:tasks.workunit.client.0.smithi076.stdout:0/202: creat d2/d8/d2d/f3f x:0 0 0 2024-03-23T15:44:50.042 INFO:tasks.workunit.client.0.smithi076.stdout:4/204: dwrite d8/d9/f17 [4194304,4194304] 0 2024-03-23T15:44:50.045 INFO:tasks.workunit.client.0.smithi076.stdout:4/205: link d8/f28 d8/f42 0 2024-03-23T15:44:50.052 INFO:tasks.workunit.client.0.smithi076.stdout:0/203: dread d2/f18 [0,4194304] 0 2024-03-23T15:44:50.052 INFO:tasks.workunit.client.0.smithi076.stdout:0/204: chown d2/d8/d2d/f3f 8377 1 2024-03-23T15:44:50.052 INFO:tasks.workunit.client.0.smithi076.stdout:0/205: chown d2/de/d10/l24 1 1 2024-03-23T15:44:50.054 INFO:tasks.workunit.client.0.smithi076.stdout:0/206: creat d2/d8/f40 x:0 0 0 2024-03-23T15:44:50.054 INFO:tasks.workunit.client.0.smithi076.stdout:0/207: dread - d2/d8/f27 zero size 2024-03-23T15:44:50.054 INFO:tasks.workunit.client.0.smithi076.stdout:2/158: dwrite d1/d4/d12/f20 [0,4194304] 0 2024-03-23T15:44:50.059 INFO:tasks.workunit.client.0.smithi076.stdout:0/208: dread d2/f7 [0,4194304] 0 2024-03-23T15:44:50.060 INFO:tasks.workunit.client.0.smithi076.stdout:0/209: write d2/de/f22 [1502024,45342] 0 2024-03-23T15:44:50.063 INFO:tasks.workunit.client.0.smithi076.stdout:0/210: creat d2/d1b/f41 x:0 0 0 2024-03-23T15:44:50.063 INFO:tasks.workunit.client.0.smithi076.stdout:0/211: chown d2/de/d10/c15 39569 1 2024-03-23T15:44:50.063 INFO:tasks.workunit.client.0.smithi076.stdout:0/212: fdatasync d2/f7 0 2024-03-23T15:44:50.063 INFO:tasks.workunit.client.0.smithi076.stdout:7/144: dwrite f1 [0,4194304] 0 2024-03-23T15:44:50.066 INFO:tasks.workunit.client.0.smithi076.stdout:7/145: creat d8/de/d13/d18/f2b x:0 0 0 2024-03-23T15:44:50.069 INFO:tasks.workunit.client.0.smithi076.stdout:7/146: creat d8/d11/f2c x:0 0 0 2024-03-23T15:44:50.071 INFO:tasks.workunit.client.0.smithi076.stdout:0/213: write d2/f7 [758055,58182] 0 2024-03-23T15:44:50.071 INFO:tasks.workunit.client.0.smithi076.stdout:0/214: fsync d2/de/f13 0 2024-03-23T15:44:50.072 INFO:tasks.workunit.client.0.smithi076.stdout:0/215: rmdir d2/d1b 39 2024-03-23T15:44:50.072 INFO:tasks.workunit.client.0.smithi076.stdout:0/216: readlink d2/de/d10/l24 0 2024-03-23T15:44:50.072 INFO:tasks.workunit.client.0.smithi076.stdout:0/217: stat d2/de/d10/d12/l2b 0 2024-03-23T15:44:50.073 INFO:tasks.workunit.client.0.smithi076.stdout:0/218: write d2/d8/f40 [554807,10197] 0 2024-03-23T15:44:50.073 INFO:tasks.workunit.client.0.smithi076.stdout:7/147: unlink d8/de/d13/d18/f22 0 2024-03-23T15:44:50.073 INFO:tasks.workunit.client.0.smithi076.stdout:0/219: creat d2/d1b/f42 x:0 0 0 2024-03-23T15:44:50.073 INFO:tasks.workunit.client.0.smithi076.stdout:0/220: stat d2/d1b/f1e 0 2024-03-23T15:44:50.074 INFO:tasks.workunit.client.0.smithi076.stdout:0/221: mkdir d2/de/d10/d43 0 2024-03-23T15:44:50.075 INFO:tasks.workunit.client.0.smithi076.stdout:4/206: dwrite d8/d9/f11 [0,4194304] 0 2024-03-23T15:44:50.079 INFO:tasks.workunit.client.0.smithi076.stdout:2/159: dwrite d1/d4/db/d15/d29/f30 [4194304,4194304] 0 2024-03-23T15:44:50.087 INFO:tasks.workunit.client.0.smithi076.stdout:4/207: write f5 [1779032,50340] 0 2024-03-23T15:44:50.087 INFO:tasks.workunit.client.0.smithi076.stdout:4/208: fsync d8/d9/f17 0 2024-03-23T15:44:50.087 INFO:tasks.workunit.client.0.smithi076.stdout:2/160: dread d1/d4/db/d15/d29/f30 [4194304,4194304] 0 2024-03-23T15:44:50.088 INFO:tasks.workunit.client.0.smithi076.stdout:2/161: symlink d1/d4/db/d15/l36 0 2024-03-23T15:44:50.091 INFO:tasks.workunit.client.0.smithi076.stdout:4/209: write d8/f28 [385774,105606] 0 2024-03-23T15:44:50.092 INFO:tasks.workunit.client.0.smithi076.stdout:0/222: dwrite d2/d1b/f42 [0,4194304] 0 2024-03-23T15:44:50.094 INFO:tasks.workunit.client.0.smithi076.stdout:7/148: dwrite d8/de/d13/d18/d1a/f28 [0,4194304] 0 2024-03-23T15:44:50.095 INFO:tasks.workunit.client.0.smithi076.stdout:7/149: dread - d8/de/f26 zero size 2024-03-23T15:44:50.095 INFO:tasks.workunit.client.0.smithi076.stdout:7/150: chown d8/d11/c16 3435179 1 2024-03-23T15:44:50.095 INFO:tasks.workunit.client.0.smithi076.stdout:7/151: chown d8/de/d13/d18/c1d 455013 1 2024-03-23T15:44:50.097 INFO:tasks.workunit.client.0.smithi076.stdout:0/223: dread d2/d1b/f1e [0,4194304] 0 2024-03-23T15:44:50.097 INFO:tasks.workunit.client.0.smithi076.stdout:0/224: dread - d2/de/f2c zero size 2024-03-23T15:44:50.097 INFO:tasks.workunit.client.0.smithi076.stdout:0/225: stat d2/l16 0 2024-03-23T15:44:50.097 INFO:tasks.workunit.client.0.smithi076.stdout:7/152: creat d8/d11/f2d x:0 0 0 2024-03-23T15:44:50.098 INFO:tasks.workunit.client.0.smithi076.stdout:7/153: stat d8/d11/f2d 0 2024-03-23T15:44:50.099 INFO:tasks.workunit.client.0.smithi076.stdout:7/154: symlink d8/de/d13/d18/d1a/l2e 0 2024-03-23T15:44:50.101 INFO:tasks.workunit.client.0.smithi076.stdout:0/226: unlink d2/f18 0 2024-03-23T15:44:50.101 INFO:tasks.workunit.client.0.smithi076.stdout:7/155: creat d8/de/d13/f2f x:0 0 0 2024-03-23T15:44:50.103 INFO:tasks.workunit.client.0.smithi076.stdout:7/156: symlink d8/de/d13/l30 0 2024-03-23T15:44:50.103 INFO:tasks.workunit.client.0.smithi076.stdout:0/227: creat d2/de/d10/d3a/f44 x:0 0 0 2024-03-23T15:44:50.104 INFO:tasks.workunit.client.0.smithi076.stdout:3/153: sync 2024-03-23T15:44:50.104 INFO:tasks.workunit.client.0.smithi076.stdout:6/132: sync 2024-03-23T15:44:50.105 INFO:tasks.workunit.client.0.smithi076.stdout:7/157: mknod d8/de/d13/d18/d1a/c31 0 2024-03-23T15:44:50.105 INFO:tasks.workunit.client.0.smithi076.stdout:4/210: dwrite d8/d9/d1d/f24 [0,4194304] 0 2024-03-23T15:44:50.106 INFO:tasks.workunit.client.0.smithi076.stdout:0/228: rename d2/de/d10/d3a/f44 to d2/d8/d2d/f45 0 2024-03-23T15:44:50.106 INFO:tasks.workunit.client.0.smithi076.stdout:0/229: write d2/de/f13 [1361340,51397] 0 2024-03-23T15:44:50.106 INFO:tasks.workunit.client.0.smithi076.stdout:0/230: dread - d2/d8/d2d/f3f zero size 2024-03-23T15:44:50.106 INFO:tasks.workunit.client.0.smithi076.stdout:6/133: symlink d1/d2/d27/d28/l2e 0 2024-03-23T15:44:50.107 INFO:tasks.workunit.client.0.smithi076.stdout:2/162: dwrite d1/f19 [0,4194304] 0 2024-03-23T15:44:50.107 INFO:tasks.workunit.client.0.smithi076.stdout:6/134: read d1/d4/d10/d18/f1e [220296,77264] 0 2024-03-23T15:44:50.107 INFO:tasks.workunit.client.0.smithi076.stdout:4/211: mknod d8/d3a/d18/d1b/c43 0 2024-03-23T15:44:50.108 INFO:tasks.workunit.client.0.smithi076.stdout:2/163: rmdir d1/d4/d12/d2d 0 2024-03-23T15:44:50.108 INFO:tasks.workunit.client.0.smithi076.stdout:2/164: truncate d1/f19 5055233 0 2024-03-23T15:44:50.108 INFO:tasks.workunit.client.0.smithi076.stdout:2/165: chown d1 18780514 1 2024-03-23T15:44:50.108 INFO:tasks.workunit.client.0.smithi076.stdout:2/166: chown d1/f14 3 1 2024-03-23T15:44:50.108 INFO:tasks.workunit.client.0.smithi076.stdout:2/167: write d1/f19 [5176493,120411] 0 2024-03-23T15:44:50.108 INFO:tasks.workunit.client.0.smithi076.stdout:2/168: dread - d1/d4/d12/f23 zero size 2024-03-23T15:44:50.109 INFO:tasks.workunit.client.0.smithi076.stdout:3/154: rename d5/lc to d5/d17/d1c/l30 0 2024-03-23T15:44:50.111 INFO:tasks.workunit.client.0.smithi076.stdout:0/231: mknod d2/de/d10/c46 0 2024-03-23T15:44:50.112 INFO:tasks.workunit.client.0.smithi076.stdout:7/158: rename d8/fb to d8/de/d13/d18/d1a/f32 0 2024-03-23T15:44:50.113 INFO:tasks.workunit.client.0.smithi076.stdout:4/212: read d8/d3a/d18/f21 [78295,38634] 0 2024-03-23T15:44:50.113 INFO:tasks.workunit.client.0.smithi076.stdout:4/213: write d8/d3a/d3b/f39 [284709,60950] 0 2024-03-23T15:44:50.114 INFO:tasks.workunit.client.0.smithi076.stdout:7/159: creat d8/d11/f33 x:0 0 0 2024-03-23T15:44:50.116 INFO:tasks.workunit.client.0.smithi076.stdout:7/160: creat d8/de/d13/d18/d1a/f34 x:0 0 0 2024-03-23T15:44:50.116 INFO:tasks.workunit.client.0.smithi076.stdout:3/155: mkdir d5/d17/d1c/d31 0 2024-03-23T15:44:50.116 INFO:tasks.workunit.client.0.smithi076.stdout:3/156: chown d5 412 1 2024-03-23T15:44:50.116 INFO:tasks.workunit.client.0.smithi076.stdout:3/157: chown d5/f14 38 1 2024-03-23T15:44:50.117 INFO:tasks.workunit.client.0.smithi076.stdout:0/232: unlink d2/d8/f27 0 2024-03-23T15:44:50.118 INFO:tasks.workunit.client.0.smithi076.stdout:0/233: fdatasync d2/d1b/f41 0 2024-03-23T15:44:50.119 INFO:tasks.workunit.client.0.smithi076.stdout:7/161: dread f3 [0,4194304] 0 2024-03-23T15:44:50.119 INFO:tasks.workunit.client.0.smithi076.stdout:7/162: readlink d8/de/l1c 0 2024-03-23T15:44:50.119 INFO:tasks.workunit.client.0.smithi076.stdout:7/163: stat d8/de/d13/d18/d1a 0 2024-03-23T15:44:50.119 INFO:tasks.workunit.client.0.smithi076.stdout:7/164: read - d8/de/d13/f2f zero size 2024-03-23T15:44:50.119 INFO:tasks.workunit.client.0.smithi076.stdout:7/165: chown d8/de 21 1 2024-03-23T15:44:50.119 INFO:tasks.workunit.client.0.smithi076.stdout:4/214: creat d8/d3a/d18/d1b/f44 x:0 0 0 2024-03-23T15:44:50.120 INFO:tasks.workunit.client.0.smithi076.stdout:7/166: mknod d8/de/d13/c35 0 2024-03-23T15:44:50.120 INFO:tasks.workunit.client.0.smithi076.stdout:7/167: readlink d8/de/d13/d18/d1a/l2e 0 2024-03-23T15:44:50.121 INFO:tasks.workunit.client.0.smithi076.stdout:2/169: getdents d1/d4/d12 0 2024-03-23T15:44:50.121 INFO:tasks.workunit.client.0.smithi076.stdout:2/170: chown d1/d4/db/fc 2020 1 2024-03-23T15:44:50.122 INFO:tasks.workunit.client.0.smithi076.stdout:3/158: rename d5/d27 to d5/d17/d1c/d32 0 2024-03-23T15:44:50.124 INFO:tasks.workunit.client.0.smithi076.stdout:6/135: rename d1/d2/l16 to d1/d2/l2f 0 2024-03-23T15:44:50.125 INFO:tasks.workunit.client.0.smithi076.stdout:0/234: mkdir d2/de/d10/d3a/d47 0 2024-03-23T15:44:50.127 INFO:tasks.workunit.client.0.smithi076.stdout:4/215: mknod d8/d41/c45 0 2024-03-23T15:44:50.127 INFO:tasks.workunit.client.0.smithi076.stdout:4/216: chown d8/d3a/d18 318034310 1 2024-03-23T15:44:50.129 INFO:tasks.workunit.client.0.smithi076.stdout:3/159: rename d5/l20 to d5/d17/d1c/l33 0 2024-03-23T15:44:50.131 INFO:tasks.workunit.client.0.smithi076.stdout:0/235: truncate d2/d1b/f1e 698786 0 2024-03-23T15:44:50.131 INFO:tasks.workunit.client.0.smithi076.stdout:0/236: fsync d2/d8/d2d/f3f 0 2024-03-23T15:44:50.131 INFO:tasks.workunit.client.0.smithi076.stdout:4/217: symlink d8/d9/l46 0 2024-03-23T15:44:50.132 INFO:tasks.workunit.client.0.smithi076.stdout:2/171: rename d1/d4/d12/d1b/l25 to d1/d2e/l37 0 2024-03-23T15:44:50.132 INFO:tasks.workunit.client.0.smithi076.stdout:2/172: rmdir d1/d4/db/d15/d29 39 2024-03-23T15:44:50.134 INFO:tasks.workunit.client.0.smithi076.stdout:3/160: creat d5/d17/d1c/d32/f34 x:0 0 0 2024-03-23T15:44:50.136 INFO:tasks.workunit.client.0.smithi076.stdout:0/237: creat d2/d8/d32/f48 x:0 0 0 2024-03-23T15:44:50.136 INFO:tasks.workunit.client.0.smithi076.stdout:0/238: chown d2/d8/d32/f48 914 1 2024-03-23T15:44:50.136 INFO:tasks.workunit.client.0.smithi076.stdout:0/239: dread - d2/d8/d2d/f45 zero size 2024-03-23T15:44:50.136 INFO:tasks.workunit.client.0.smithi076.stdout:0/240: write d2/d8/d32/f39 [433183,16403] 0 2024-03-23T15:44:50.136 INFO:tasks.workunit.client.0.smithi076.stdout:0/241: write d2/de/f2c [873872,65173] 0 2024-03-23T15:44:50.137 INFO:tasks.workunit.client.0.smithi076.stdout:7/168: dwrite d8/de/d13/d18/d1a/f20 [0,4194304] 0 2024-03-23T15:44:50.138 INFO:tasks.workunit.client.0.smithi076.stdout:3/161: mkdir d5/d35 0 2024-03-23T15:44:50.141 INFO:tasks.workunit.client.0.smithi076.stdout:7/169: mknod d8/de/d13/d18/d1a/c36 0 2024-03-23T15:44:50.144 INFO:tasks.workunit.client.0.smithi076.stdout:6/136: dwrite d1/d4/f5 [0,4194304] 0 2024-03-23T15:44:50.144 INFO:tasks.workunit.client.0.smithi076.stdout:6/137: chown d1/d4/d10 6006 1 2024-03-23T15:44:50.145 INFO:tasks.workunit.client.0.smithi076.stdout:6/138: write d1/d2/f9 [980131,6945] 0 2024-03-23T15:44:50.145 INFO:tasks.workunit.client.0.smithi076.stdout:6/139: dread - d1/dd/df/f13 zero size 2024-03-23T15:44:50.149 INFO:tasks.workunit.client.0.smithi076.stdout:6/140: creat d1/d4/d10/f30 x:0 0 0 2024-03-23T15:44:50.152 INFO:tasks.workunit.client.0.smithi076.stdout:4/218: dwrite d8/d3a/d18/d1b/f44 [0,4194304] 0 2024-03-23T15:44:50.154 INFO:tasks.workunit.client.0.smithi076.stdout:2/173: dwrite d1/f19 [4194304,4194304] 0 2024-03-23T15:44:50.157 INFO:tasks.workunit.client.0.smithi076.stdout:2/174: link d1/d4/db/fc d1/d4/d12/d1b/d32/f38 0 2024-03-23T15:44:50.158 INFO:tasks.workunit.client.0.smithi076.stdout:9/155: sync 2024-03-23T15:44:50.158 INFO:tasks.workunit.client.0.smithi076.stdout:9/156: fsync d1/f1a 0 2024-03-23T15:44:50.158 INFO:tasks.workunit.client.0.smithi076.stdout:2/175: symlink d1/d2e/l39 0 2024-03-23T15:44:50.160 INFO:tasks.workunit.client.0.smithi076.stdout:2/176: symlink d1/d4/l3a 0 2024-03-23T15:44:50.161 INFO:tasks.workunit.client.0.smithi076.stdout:7/170: dwrite f3 [0,4194304] 0 2024-03-23T15:44:50.161 INFO:tasks.workunit.client.0.smithi076.stdout:7/171: dread - d8/de/f26 zero size 2024-03-23T15:44:50.161 INFO:tasks.workunit.client.0.smithi076.stdout:7/172: truncate d8/de/d13/f2f 283002 0 2024-03-23T15:44:50.163 INFO:tasks.workunit.client.0.smithi076.stdout:3/162: dwrite d5/d17/f21 [0,4194304] 0 2024-03-23T15:44:50.163 INFO:tasks.workunit.client.0.smithi076.stdout:3/163: stat d5/c10 0 2024-03-23T15:44:50.163 INFO:tasks.workunit.client.0.smithi076.stdout:3/164: chown d5/l2f 51 1 2024-03-23T15:44:50.163 INFO:tasks.workunit.client.0.smithi076.stdout:3/165: readlink d5/l25 0 2024-03-23T15:44:50.163 INFO:tasks.workunit.client.0.smithi076.stdout:3/166: truncate d5/f16 1592090 0 2024-03-23T15:44:50.167 INFO:tasks.workunit.client.0.smithi076.stdout:3/167: rename d5/d17/f21 to d5/d35/f36 0 2024-03-23T15:44:50.169 INFO:tasks.workunit.client.0.smithi076.stdout:3/168: mkdir d5/d17/d37 0 2024-03-23T15:44:50.169 INFO:tasks.workunit.client.0.smithi076.stdout:3/169: read d5/f6 [918014,96971] 0 2024-03-23T15:44:50.169 INFO:tasks.workunit.client.0.smithi076.stdout:5/176: sync 2024-03-23T15:44:50.169 INFO:tasks.workunit.client.0.smithi076.stdout:8/143: sync 2024-03-23T15:44:50.169 INFO:tasks.workunit.client.0.smithi076.stdout:1/86: sync 2024-03-23T15:44:50.170 INFO:tasks.workunit.client.0.smithi076.stdout:3/170: mknod d5/d35/c38 0 2024-03-23T15:44:50.173 INFO:tasks.workunit.client.0.smithi076.stdout:3/171: mkdir d5/d17/d1c/d32/d39 0 2024-03-23T15:44:50.174 INFO:tasks.workunit.client.0.smithi076.stdout:3/172: creat d5/d17/d1c/d32/d39/f3a x:0 0 0 2024-03-23T15:44:50.174 INFO:tasks.workunit.client.0.smithi076.stdout:3/173: rmdir d5/d35 39 2024-03-23T15:44:50.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/174: mknod d5/d17/d1c/d32/d39/c3b 0 2024-03-23T15:44:50.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/175: fdatasync d5/f1f 0 2024-03-23T15:44:50.177 INFO:tasks.workunit.client.0.smithi076.stdout:6/141: dwrite d1/f21 [0,4194304] 0 2024-03-23T15:44:50.177 INFO:tasks.workunit.client.0.smithi076.stdout:3/176: mkdir d5/d35/d3c 0 2024-03-23T15:44:50.177 INFO:tasks.workunit.client.0.smithi076.stdout:3/177: chown d5/c18 15725186 1 2024-03-23T15:44:50.178 INFO:tasks.workunit.client.0.smithi076.stdout:7/173: dread d8/de/d13/f14 [0,4194304] 0 2024-03-23T15:44:50.178 INFO:tasks.workunit.client.0.smithi076.stdout:0/242: dwrite d2/de/f22 [0,4194304] 0 2024-03-23T15:44:50.179 INFO:tasks.workunit.client.0.smithi076.stdout:7/174: dread - d8/d11/f25 zero size 2024-03-23T15:44:50.179 INFO:tasks.workunit.client.0.smithi076.stdout:1/87: dread d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:50.180 INFO:tasks.workunit.client.0.smithi076.stdout:7/175: mknod d8/de/d13/d24/c37 0 2024-03-23T15:44:50.182 INFO:tasks.workunit.client.0.smithi076.stdout:7/176: truncate d8/de/d13/f21 1863378 0 2024-03-23T15:44:50.182 INFO:tasks.workunit.client.0.smithi076.stdout:7/177: fsync d8/de/d13/f2f 0 2024-03-23T15:44:50.182 INFO:tasks.workunit.client.0.smithi076.stdout:7/178: chown d8/de/d13/d18/d1a/c36 20842544 1 2024-03-23T15:44:50.182 INFO:tasks.workunit.client.0.smithi076.stdout:7/179: fdatasync d8/d11/f2c 0 2024-03-23T15:44:50.182 INFO:tasks.workunit.client.0.smithi076.stdout:3/178: symlink d5/d17/d1c/d32/d39/l3d 0 2024-03-23T15:44:50.182 INFO:tasks.workunit.client.0.smithi076.stdout:3/179: fdatasync d5/d17/d1c/d32/f34 0 2024-03-23T15:44:50.182 INFO:tasks.workunit.client.0.smithi076.stdout:3/180: chown d5/d17/d1c/f1e 192030 1 2024-03-23T15:44:50.183 INFO:tasks.workunit.client.0.smithi076.stdout:3/181: chown d5/d17/d1c/d32/l2c 32951153 1 2024-03-23T15:44:50.183 INFO:tasks.workunit.client.0.smithi076.stdout:1/88: symlink d0/d1/d6/l1b 0 2024-03-23T15:44:50.183 INFO:tasks.workunit.client.0.smithi076.stdout:1/89: chown d0 34 1 2024-03-23T15:44:50.183 INFO:tasks.workunit.client.0.smithi076.stdout:0/243: dread d2/d8/f25 [4194304,4194304] 0 2024-03-23T15:44:50.183 INFO:tasks.workunit.client.0.smithi076.stdout:0/244: readlink d2/d1b/l38 0 2024-03-23T15:44:50.183 INFO:tasks.workunit.client.0.smithi076.stdout:1/90: symlink d0/l1c 0 2024-03-23T15:44:50.184 INFO:tasks.workunit.client.0.smithi076.stdout:0/245: dread d2/d8/f28 [0,4194304] 0 2024-03-23T15:44:50.184 INFO:tasks.workunit.client.0.smithi076.stdout:0/246: fsync d2/f9 0 2024-03-23T15:44:50.185 INFO:tasks.workunit.client.0.smithi076.stdout:0/247: chown d2/d8/d32/f39 5972 1 2024-03-23T15:44:50.185 INFO:tasks.workunit.client.0.smithi076.stdout:9/157: dwrite d1/d20/f2c [0,4194304] 0 2024-03-23T15:44:50.186 INFO:tasks.workunit.client.0.smithi076.stdout:0/248: rename d2/de/d10/d12/l19 to d2/de/d10/d12/l49 0 2024-03-23T15:44:50.186 INFO:tasks.workunit.client.0.smithi076.stdout:0/249: readlink d2/l3e 0 2024-03-23T15:44:50.187 INFO:tasks.workunit.client.0.smithi076.stdout:0/250: chown d2/d1b/f42 192878315 1 2024-03-23T15:44:50.188 INFO:tasks.workunit.client.0.smithi076.stdout:7/180: write f0 [3831843,120883] 0 2024-03-23T15:44:50.188 INFO:tasks.workunit.client.0.smithi076.stdout:7/181: dread - d8/de/f26 zero size 2024-03-23T15:44:50.189 INFO:tasks.workunit.client.0.smithi076.stdout:9/158: symlink d1/d20/l2e 0 2024-03-23T15:44:50.189 INFO:tasks.workunit.client.0.smithi076.stdout:7/182: symlink d8/de/d13/d18/d1a/l38 0 2024-03-23T15:44:50.190 INFO:tasks.workunit.client.0.smithi076.stdout:7/183: write d8/d11/f2c [921706,45990] 0 2024-03-23T15:44:50.190 INFO:tasks.workunit.client.0.smithi076.stdout:0/251: truncate d2/de/f1a 2937587 0 2024-03-23T15:44:50.190 INFO:tasks.workunit.client.0.smithi076.stdout:5/177: dread f11 [0,4194304] 0 2024-03-23T15:44:50.191 INFO:tasks.workunit.client.0.smithi076.stdout:9/159: truncate d1/f15 369477 0 2024-03-23T15:44:50.193 INFO:tasks.workunit.client.0.smithi076.stdout:9/160: truncate d1/f26 2518066 0 2024-03-23T15:44:50.193 INFO:tasks.workunit.client.0.smithi076.stdout:9/161: fsync d1/d1f/f22 0 2024-03-23T15:44:50.195 INFO:tasks.workunit.client.0.smithi076.stdout:5/178: unlink d1a/d33/d36/f40 0 2024-03-23T15:44:50.196 INFO:tasks.workunit.client.0.smithi076.stdout:9/162: rmdir d1 39 2024-03-23T15:44:50.197 INFO:tasks.workunit.client.0.smithi076.stdout:9/163: dread d1/f12 [0,4194304] 0 2024-03-23T15:44:50.198 INFO:tasks.workunit.client.0.smithi076.stdout:6/142: dwrite d1/dd/d2a/d23/f2d [0,4194304] 0 2024-03-23T15:44:50.198 INFO:tasks.workunit.client.0.smithi076.stdout:6/143: write d1/f2b [966264,16825] 0 2024-03-23T15:44:50.198 INFO:tasks.workunit.client.0.smithi076.stdout:6/144: rename d1 to d1/dd/d31 22 2024-03-23T15:44:50.199 INFO:tasks.workunit.client.0.smithi076.stdout:8/144: dwrite d8/f10 [0,4194304] 0 2024-03-23T15:44:50.199 INFO:tasks.workunit.client.0.smithi076.stdout:8/145: chown d8/c20 393868 1 2024-03-23T15:44:50.199 INFO:tasks.workunit.client.0.smithi076.stdout:9/164: mkdir d1/d2f 0 2024-03-23T15:44:50.200 INFO:tasks.workunit.client.0.smithi076.stdout:8/146: dread d8/f23 [0,4194304] 0 2024-03-23T15:44:50.200 INFO:tasks.workunit.client.0.smithi076.stdout:8/147: readlink d8/l25 0 2024-03-23T15:44:50.200 INFO:tasks.workunit.client.0.smithi076.stdout:8/148: stat d8/fa 0 2024-03-23T15:44:50.200 INFO:tasks.workunit.client.0.smithi076.stdout:8/149: chown l4 29 1 2024-03-23T15:44:50.201 INFO:tasks.workunit.client.0.smithi076.stdout:5/179: write fc [4651178,42071] 0 2024-03-23T15:44:50.203 INFO:tasks.workunit.client.0.smithi076.stdout:3/182: dwrite d5/d17/d1c/f2b [0,4194304] 0 2024-03-23T15:44:50.205 INFO:tasks.workunit.client.0.smithi076.stdout:8/150: unlink l4 0 2024-03-23T15:44:50.206 INFO:tasks.workunit.client.0.smithi076.stdout:5/180: mknod d1a/d33/d3a/c45 0 2024-03-23T15:44:50.212 INFO:tasks.workunit.client.0.smithi076.stdout:3/183: creat d5/d17/d1c/d32/d39/f3e x:0 0 0 2024-03-23T15:44:50.212 INFO:tasks.workunit.client.0.smithi076.stdout:3/184: write d5/f13 [569123,97921] 0 2024-03-23T15:44:50.214 INFO:tasks.workunit.client.0.smithi076.stdout:8/151: mkdir d8/d26/d33 0 2024-03-23T15:44:50.216 INFO:tasks.workunit.client.0.smithi076.stdout:9/165: dwrite d1/d1f/f22 [0,4194304] 0 2024-03-23T15:44:50.217 INFO:tasks.workunit.client.0.smithi076.stdout:5/181: stat c16 0 2024-03-23T15:44:50.217 INFO:tasks.workunit.client.0.smithi076.stdout:5/182: dread - d1a/f31 zero size 2024-03-23T15:44:50.218 INFO:tasks.workunit.client.0.smithi076.stdout:0/252: dread d2/de/f1a [0,4194304] 0 2024-03-23T15:44:50.218 INFO:tasks.workunit.client.0.smithi076.stdout:8/152: symlink d8/d26/l34 0 2024-03-23T15:44:50.224 INFO:tasks.workunit.client.0.smithi076.stdout:0/253: creat d2/d8/d32/f4a x:0 0 0 2024-03-23T15:44:50.225 INFO:tasks.workunit.client.0.smithi076.stdout:5/183: truncate fd 2519877 0 2024-03-23T15:44:50.225 INFO:tasks.workunit.client.0.smithi076.stdout:5/184: fsync d1a/d20/f30 0 2024-03-23T15:44:50.226 INFO:tasks.workunit.client.0.smithi076.stdout:0/254: getdents d2/d1b 0 2024-03-23T15:44:50.229 INFO:tasks.workunit.client.0.smithi076.stdout:5/185: mknod d1a/d33/c46 0 2024-03-23T15:44:50.229 INFO:tasks.workunit.client.0.smithi076.stdout:5/186: write d1a/d33/d36/f3d [26430,41495] 0 2024-03-23T15:44:50.229 INFO:tasks.workunit.client.0.smithi076.stdout:0/255: link d2/c1c d2/de/d10/c4b 0 2024-03-23T15:44:50.230 INFO:tasks.workunit.client.0.smithi076.stdout:0/256: read d2/d8/f28 [144796,124710] 0 2024-03-23T15:44:50.231 INFO:tasks.workunit.client.0.smithi076.stdout:0/257: getdents d2/de 0 2024-03-23T15:44:50.232 INFO:tasks.workunit.client.0.smithi076.stdout:3/185: dwrite d5/f9 [0,4194304] 0 2024-03-23T15:44:50.232 INFO:tasks.workunit.client.0.smithi076.stdout:3/186: chown d5/d17/l22 350277 1 2024-03-23T15:44:50.236 INFO:tasks.workunit.client.0.smithi076.stdout:0/258: link d2/f1d d2/de/d10/d43/f4c 0 2024-03-23T15:44:50.236 INFO:tasks.workunit.client.0.smithi076.stdout:8/153: dwrite d8/f2a [0,4194304] 0 2024-03-23T15:44:50.252 INFO:tasks.workunit.client.0.smithi076.stdout:6/145: rename d1/d2/ce to d1/dd/df/c32 0 2024-03-23T15:44:50.255 INFO:tasks.workunit.client.0.smithi076.stdout:9/166: rename d1/d20/f2c to d1/d1f/f30 0 2024-03-23T15:44:50.255 INFO:tasks.workunit.client.0.smithi076.stdout:9/167: fsync d1/f15 0 2024-03-23T15:44:50.255 INFO:tasks.workunit.client.0.smithi076.stdout:8/154: dwrite f2 [0,4194304] 0 2024-03-23T15:44:50.255 INFO:tasks.workunit.client.0.smithi076.stdout:5/187: dwrite d1a/d33/d3a/f43 [0,4194304] 0 2024-03-23T15:44:50.257 INFO:tasks.workunit.client.0.smithi076.stdout:6/146: rename d1/d4/d10/l29 to d1/dd/df/l33 0 2024-03-23T15:44:50.257 INFO:tasks.workunit.client.0.smithi076.stdout:6/147: dread - d1/f2c zero size 2024-03-23T15:44:50.257 INFO:tasks.workunit.client.0.smithi076.stdout:0/259: dwrite d2/d8/f28 [0,4194304] 0 2024-03-23T15:44:50.259 INFO:tasks.workunit.client.0.smithi076.stdout:6/148: creat d1/d4/d10/f34 x:0 0 0 2024-03-23T15:44:50.260 INFO:tasks.workunit.client.0.smithi076.stdout:8/155: write d8/f1d [2739040,129887] 0 2024-03-23T15:44:50.262 INFO:tasks.workunit.client.0.smithi076.stdout:6/149: creat d1/d4/f35 x:0 0 0 2024-03-23T15:44:50.274 INFO:tasks.workunit.client.0.smithi076.stdout:9/168: dwrite d1/f18 [0,4194304] 0 2024-03-23T15:44:50.276 INFO:tasks.workunit.client.0.smithi076.stdout:9/169: dread d1/f1a [0,4194304] 0 2024-03-23T15:44:50.278 INFO:tasks.workunit.client.0.smithi076.stdout:0/260: dwrite d2/de/f14 [0,4194304] 0 2024-03-23T15:44:50.279 INFO:tasks.workunit.client.0.smithi076.stdout:6/150: dwrite d1/d4/d10/d18/f1e [0,4194304] 0 2024-03-23T15:44:50.281 INFO:tasks.workunit.client.0.smithi076.stdout:0/261: unlink d2/de/d10/d12/l49 0 2024-03-23T15:44:50.283 INFO:tasks.workunit.client.0.smithi076.stdout:6/151: dread d1/f11 [0,4194304] 0 2024-03-23T15:44:50.283 INFO:tasks.workunit.client.0.smithi076.stdout:6/152: truncate d1/d2/f22 402973 0 2024-03-23T15:44:50.293 INFO:tasks.workunit.client.0.smithi076.stdout:9/170: dwrite d1/f12 [0,4194304] 0 2024-03-23T15:44:50.293 INFO:tasks.workunit.client.0.smithi076.stdout:0/262: read d2/f9 [1842773,29494] 0 2024-03-23T15:44:50.293 INFO:tasks.workunit.client.0.smithi076.stdout:0/263: chown d2/de/d10/d3a 391574249 1 2024-03-23T15:44:50.294 INFO:tasks.workunit.client.0.smithi076.stdout:6/153: dread d1/d4/d10/d18/f1e [0,4194304] 0 2024-03-23T15:44:50.296 INFO:tasks.workunit.client.0.smithi076.stdout:9/171: creat d1/d20/f31 x:0 0 0 2024-03-23T15:44:50.297 INFO:tasks.workunit.client.0.smithi076.stdout:0/264: link d2/d8/c3d d2/de/d10/d12/c4d 0 2024-03-23T15:44:50.299 INFO:tasks.workunit.client.0.smithi076.stdout:6/154: rename d1/d4/d10/d18 to d1/dd/d2a/d23/d36 0 2024-03-23T15:44:50.299 INFO:tasks.workunit.client.0.smithi076.stdout:6/155: chown d1/f21 4454 1 2024-03-23T15:44:50.299 INFO:tasks.workunit.client.0.smithi076.stdout:9/172: creat d1/f32 x:0 0 0 2024-03-23T15:44:50.305 INFO:tasks.workunit.client.0.smithi076.stdout:9/173: getdents d1 0 2024-03-23T15:44:50.307 INFO:tasks.workunit.client.0.smithi076.stdout:9/174: dread d1/f15 [0,4194304] 0 2024-03-23T15:44:50.308 INFO:tasks.workunit.client.0.smithi076.stdout:9/175: fdatasync d1/f18 0 2024-03-23T15:44:50.310 INFO:tasks.workunit.client.0.smithi076.stdout:0/265: dwrite d2/de/f2c [0,4194304] 0 2024-03-23T15:44:50.314 INFO:tasks.workunit.client.0.smithi076.stdout:6/156: rename d1/dd/df/c32 to d1/d2/d27/d28/c37 0 2024-03-23T15:44:50.315 INFO:tasks.workunit.client.0.smithi076.stdout:0/266: creat d2/de/d10/d3a/d47/f4e x:0 0 0 2024-03-23T15:44:50.316 INFO:tasks.workunit.client.0.smithi076.stdout:9/176: dread d1/f19 [0,4194304] 0 2024-03-23T15:44:50.316 INFO:tasks.workunit.client.0.smithi076.stdout:9/177: chown d1/c5 32 1 2024-03-23T15:44:50.317 INFO:tasks.workunit.client.0.smithi076.stdout:0/267: creat d2/d8/f4f x:0 0 0 2024-03-23T15:44:50.317 INFO:tasks.workunit.client.0.smithi076.stdout:4/219: sync 2024-03-23T15:44:50.318 INFO:tasks.workunit.client.0.smithi076.stdout:9/178: creat d1/d1f/d29/d2d/f33 x:0 0 0 2024-03-23T15:44:50.320 INFO:tasks.workunit.client.0.smithi076.stdout:0/268: creat d2/de/d10/d43/f50 x:0 0 0 2024-03-23T15:44:50.321 INFO:tasks.workunit.client.0.smithi076.stdout:4/220: mknod d8/d3a/c47 0 2024-03-23T15:44:50.325 INFO:tasks.workunit.client.0.smithi076.stdout:0/269: unlink d2/d8/c3d 0 2024-03-23T15:44:50.326 INFO:tasks.workunit.client.0.smithi076.stdout:0/270: getdents d2/de/d10/d12 0 2024-03-23T15:44:50.328 INFO:tasks.workunit.client.0.smithi076.stdout:0/271: unlink d2/de/f2c 0 2024-03-23T15:44:50.330 INFO:tasks.workunit.client.0.smithi076.stdout:0/272: unlink d2/d1b/f41 0 2024-03-23T15:44:50.330 INFO:tasks.workunit.client.0.smithi076.stdout:0/273: chown d2/d1b/f42 32409 1 2024-03-23T15:44:50.331 INFO:tasks.workunit.client.0.smithi076.stdout:0/274: write d2/f1d [761253,100288] 0 2024-03-23T15:44:50.331 INFO:tasks.workunit.client.0.smithi076.stdout:0/275: dread - d2/de/d10/d43/f50 zero size 2024-03-23T15:44:50.338 INFO:tasks.workunit.client.0.smithi076.stdout:9/179: dwrite d1/d20/f31 [0,4194304] 0 2024-03-23T15:44:50.340 INFO:tasks.workunit.client.0.smithi076.stdout:4/221: dwrite f5 [4194304,4194304] 0 2024-03-23T15:44:50.341 INFO:tasks.workunit.client.0.smithi076.stdout:9/180: mknod d1/d1f/d29/c34 0 2024-03-23T15:44:50.344 INFO:tasks.workunit.client.0.smithi076.stdout:9/181: rename d1/d20/f31 to d1/f35 0 2024-03-23T15:44:50.346 INFO:tasks.workunit.client.0.smithi076.stdout:9/182: mknod d1/d1f/c36 0 2024-03-23T15:44:50.347 INFO:tasks.workunit.client.0.smithi076.stdout:9/183: write d1/f32 [484987,95876] 0 2024-03-23T15:44:50.349 INFO:tasks.workunit.client.0.smithi076.stdout:9/184: mkdir d1/d1f/d29/d2d/d37 0 2024-03-23T15:44:50.351 INFO:tasks.workunit.client.0.smithi076.stdout:0/276: dwrite d2/de/f1a [0,4194304] 0 2024-03-23T15:44:50.352 INFO:tasks.workunit.client.0.smithi076.stdout:9/185: truncate d1/d20/f28 1145059 0 2024-03-23T15:44:50.352 INFO:tasks.workunit.client.0.smithi076.stdout:9/186: write d1/f15 [888523,66231] 0 2024-03-23T15:44:50.353 INFO:tasks.workunit.client.0.smithi076.stdout:0/277: dread d2/d8/d32/f39 [0,4194304] 0 2024-03-23T15:44:50.355 INFO:tasks.workunit.client.0.smithi076.stdout:4/222: dwrite d8/d9/d1d/f24 [4194304,4194304] 0 2024-03-23T15:44:50.356 INFO:tasks.workunit.client.0.smithi076.stdout:9/187: symlink d1/d20/l38 0 2024-03-23T15:44:50.357 INFO:tasks.workunit.client.0.smithi076.stdout:0/278: unlink d2/d8/d2d/f2f 0 2024-03-23T15:44:50.359 INFO:tasks.workunit.client.0.smithi076.stdout:4/223: creat d8/d3a/f48 x:0 0 0 2024-03-23T15:44:50.360 INFO:tasks.workunit.client.0.smithi076.stdout:4/224: chown d8/d3a/f34 403 1 2024-03-23T15:44:50.361 INFO:tasks.workunit.client.0.smithi076.stdout:9/188: symlink d1/d1f/d29/d2d/d37/l39 0 2024-03-23T15:44:50.364 INFO:tasks.workunit.client.0.smithi076.stdout:4/225: rename f2 to d8/f49 0 2024-03-23T15:44:50.364 INFO:tasks.workunit.client.0.smithi076.stdout:4/226: fdatasync d8/f3d 0 2024-03-23T15:44:50.364 INFO:tasks.workunit.client.0.smithi076.stdout:4/227: chown d8/d3a/c47 1405014 1 2024-03-23T15:44:50.368 INFO:tasks.workunit.client.0.smithi076.stdout:4/228: link d8/d9/d1d/f24 d8/d3a/d3b/f4a 0 2024-03-23T15:44:50.368 INFO:tasks.workunit.client.0.smithi076.stdout:0/279: truncate d2/de/f1a 1559027 0 2024-03-23T15:44:50.371 INFO:tasks.workunit.client.0.smithi076.stdout:4/229: rename d8/d9/f11 to d8/d3a/d3b/f4b 0 2024-03-23T15:44:50.371 INFO:tasks.workunit.client.0.smithi076.stdout:4/230: stat d8/d9/f17 0 2024-03-23T15:44:50.371 INFO:tasks.workunit.client.0.smithi076.stdout:4/231: chown d8/d3a/d3b/c2f 1348 1 2024-03-23T15:44:50.371 INFO:tasks.workunit.client.0.smithi076.stdout:4/232: readlink d8/d3a/d18/d1b/l3c 0 2024-03-23T15:44:50.373 INFO:tasks.workunit.client.0.smithi076.stdout:0/280: rename d2/de/d10/c4b to d2/de/d10/d12/c51 0 2024-03-23T15:44:50.375 INFO:tasks.workunit.client.0.smithi076.stdout:0/281: mkdir d2/d8/d52 0 2024-03-23T15:44:50.378 INFO:tasks.workunit.client.0.smithi076.stdout:9/189: dwrite d1/f18 [0,4194304] 0 2024-03-23T15:44:50.381 INFO:tasks.workunit.client.0.smithi076.stdout:0/282: creat d2/d8/f53 x:0 0 0 2024-03-23T15:44:50.385 INFO:tasks.workunit.client.0.smithi076.stdout:9/190: write d1/d1f/f22 [3197618,123798] 0 2024-03-23T15:44:50.387 INFO:tasks.workunit.client.0.smithi076.stdout:9/191: truncate d1/d1f/f21 582363 0 2024-03-23T15:44:50.394 INFO:tasks.workunit.client.0.smithi076.stdout:9/192: dread d1/f35 [0,4194304] 0 2024-03-23T15:44:50.397 INFO:tasks.workunit.client.0.smithi076.stdout:0/283: dwrite d2/f7 [0,4194304] 0 2024-03-23T15:44:50.400 INFO:tasks.workunit.client.0.smithi076.stdout:0/284: link d2/d8/c17 d2/de/d10/d12/c54 0 2024-03-23T15:44:50.402 INFO:tasks.workunit.client.0.smithi076.stdout:0/285: symlink d2/d1b/l55 0 2024-03-23T15:44:50.403 INFO:tasks.workunit.client.0.smithi076.stdout:2/177: sync 2024-03-23T15:44:50.416 INFO:tasks.workunit.client.0.smithi076.stdout:2/178: dread d1/d4/d12/f20 [0,4194304] 0 2024-03-23T15:44:50.421 INFO:tasks.workunit.client.0.smithi076.stdout:2/179: dread d1/d4/db/d15/d29/f30 [0,4194304] 0 2024-03-23T15:44:50.421 INFO:tasks.workunit.client.0.smithi076.stdout:0/286: dwrite d2/d1b/f1e [0,4194304] 0 2024-03-23T15:44:50.421 INFO:tasks.workunit.client.0.smithi076.stdout:0/287: write d2/de/d10/d12/f23 [1766472,35537] 0 2024-03-23T15:44:50.421 INFO:tasks.workunit.client.0.smithi076.stdout:0/288: fdatasync d2/de/d10/d43/f50 0 2024-03-23T15:44:50.422 INFO:tasks.workunit.client.0.smithi076.stdout:0/289: mkdir d2/de/d10/d43/d56 0 2024-03-23T15:44:50.422 INFO:tasks.workunit.client.0.smithi076.stdout:0/290: write d2/d8/fa [550167,31445] 0 2024-03-23T15:44:50.423 INFO:tasks.workunit.client.0.smithi076.stdout:2/180: creat d1/d4/db/f3b x:0 0 0 2024-03-23T15:44:50.423 INFO:tasks.workunit.client.0.smithi076.stdout:2/181: chown d1/d4/d12/d1d 30476184 1 2024-03-23T15:44:50.424 INFO:tasks.workunit.client.0.smithi076.stdout:2/182: creat d1/d4/d12/d1d/f3c x:0 0 0 2024-03-23T15:44:50.424 INFO:tasks.workunit.client.0.smithi076.stdout:2/183: read - d1/d4/d12/d1b/f1c zero size 2024-03-23T15:44:50.424 INFO:tasks.workunit.client.0.smithi076.stdout:2/184: write d1/d4/d12/d1b/f1c [669340,40131] 0 2024-03-23T15:44:50.426 INFO:tasks.workunit.client.0.smithi076.stdout:2/185: creat d1/d4/db/f3d x:0 0 0 2024-03-23T15:44:50.426 INFO:tasks.workunit.client.0.smithi076.stdout:2/186: read d1/d4/f10 [1095001,44209] 0 2024-03-23T15:44:50.426 INFO:tasks.workunit.client.0.smithi076.stdout:2/187: mkdir d1/d4/db/d3e 0 2024-03-23T15:44:50.427 INFO:tasks.workunit.client.0.smithi076.stdout:2/188: stat d1/d4/l11 0 2024-03-23T15:44:50.427 INFO:tasks.workunit.client.0.smithi076.stdout:2/189: creat d1/d2e/f3f x:0 0 0 2024-03-23T15:44:50.428 INFO:tasks.workunit.client.0.smithi076.stdout:2/190: fdatasync d1/d4/d12/f20 0 2024-03-23T15:44:50.428 INFO:tasks.workunit.client.0.smithi076.stdout:2/191: creat d1/d4/d12/d1b/f40 x:0 0 0 2024-03-23T15:44:50.430 INFO:tasks.workunit.client.0.smithi076.stdout:2/192: mknod d1/c41 0 2024-03-23T15:44:50.430 INFO:tasks.workunit.client.0.smithi076.stdout:2/193: chown d1/d4/d12/d1b/f40 43267 1 2024-03-23T15:44:50.432 INFO:tasks.workunit.client.0.smithi076.stdout:2/194: creat d1/d2e/f42 x:0 0 0 2024-03-23T15:44:50.432 INFO:tasks.workunit.client.0.smithi076.stdout:2/195: chown d1/d4/d12/c34 1046143825 1 2024-03-23T15:44:50.433 INFO:tasks.workunit.client.0.smithi076.stdout:2/196: mkdir d1/d4/d12/d43 0 2024-03-23T15:44:50.433 INFO:tasks.workunit.client.0.smithi076.stdout:2/197: write d1/d4/db/d15/f1f [477842,53121] 0 2024-03-23T15:44:50.446 INFO:tasks.workunit.client.0.smithi076.stdout:2/198: dwrite d1/d4/db/d15/d29/f30 [0,4194304] 0 2024-03-23T15:44:50.447 INFO:tasks.workunit.client.0.smithi076.stdout:2/199: creat d1/d4/d12/d1b/f44 x:0 0 0 2024-03-23T15:44:50.498 INFO:tasks.workunit.client.0.smithi076.stdout:1/91: sync 2024-03-23T15:44:50.499 INFO:tasks.workunit.client.0.smithi076.stdout:1/92: creat d0/d1/d6/d17/f1d x:0 0 0 2024-03-23T15:44:50.500 INFO:tasks.workunit.client.0.smithi076.stdout:1/93: link d0/d1/d6/cf d0/d1/d6/d17/c1e 0 2024-03-23T15:44:50.502 INFO:tasks.workunit.client.0.smithi076.stdout:1/94: unlink d0/f8 0 2024-03-23T15:44:50.502 INFO:tasks.workunit.client.0.smithi076.stdout:1/95: rmdir d0/d2 39 2024-03-23T15:44:50.503 INFO:tasks.workunit.client.0.smithi076.stdout:1/96: mkdir d0/d2/d1f 0 2024-03-23T15:44:50.504 INFO:tasks.workunit.client.0.smithi076.stdout:1/97: mkdir d0/d1/d6/d17/d20 0 2024-03-23T15:44:50.505 INFO:tasks.workunit.client.0.smithi076.stdout:1/98: rename d0/l1c to d0/d1/l21 0 2024-03-23T15:44:50.507 INFO:tasks.workunit.client.0.smithi076.stdout:1/99: dread d0/d1/d6/f9 [4194304,4194304] 0 2024-03-23T15:44:50.519 INFO:tasks.workunit.client.0.smithi076.stdout:1/100: dwrite d0/d1/d6/f9 [4194304,4194304] 0 2024-03-23T15:44:50.535 INFO:tasks.workunit.client.0.smithi076.stdout:1/101: dwrite d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:50.535 INFO:tasks.workunit.client.0.smithi076.stdout:1/102: fdatasync d0/d1/d6/f9 0 2024-03-23T15:44:50.549 INFO:tasks.workunit.client.0.smithi076.stdout:1/103: dwrite d0/d1/d6/d17/f1d [0,4194304] 0 2024-03-23T15:44:50.549 INFO:tasks.workunit.client.0.smithi076.stdout:1/104: fdatasync d0/d1/d6/f9 0 2024-03-23T15:44:50.549 INFO:tasks.workunit.client.0.smithi076.stdout:1/105: stat d0/l13 0 2024-03-23T15:44:50.555 INFO:tasks.workunit.client.0.smithi076.stdout:1/106: dread d0/d1/d6/d17/f1d [0,4194304] 0 2024-03-23T15:44:50.555 INFO:tasks.workunit.client.0.smithi076.stdout:1/107: write d0/d1/d6/d17/f1d [5182514,72080] 0 2024-03-23T15:44:50.559 INFO:tasks.workunit.client.0.smithi076.stdout:7/184: sync 2024-03-23T15:44:50.559 INFO:tasks.workunit.client.0.smithi076.stdout:7/185: chown d8/de/f26 349 1 2024-03-23T15:44:50.567 INFO:tasks.workunit.client.0.smithi076.stdout:7/186: dread f3 [0,4194304] 0 2024-03-23T15:44:50.567 INFO:tasks.workunit.client.0.smithi076.stdout:7/187: chown d8/de/d13/l1f 598 1 2024-03-23T15:44:50.568 INFO:tasks.workunit.client.0.smithi076.stdout:7/188: mknod d8/de/d13/d24/c39 0 2024-03-23T15:44:50.572 INFO:tasks.workunit.client.0.smithi076.stdout:7/189: write d8/de/d13/f21 [740041,70235] 0 2024-03-23T15:44:50.573 INFO:tasks.workunit.client.0.smithi076.stdout:7/190: dread - d8/de/d13/d18/d1a/f2a zero size 2024-03-23T15:44:50.573 INFO:tasks.workunit.client.0.smithi076.stdout:7/191: dread - d8/de/d13/d18/d1a/f34 zero size 2024-03-23T15:44:50.573 INFO:tasks.workunit.client.0.smithi076.stdout:1/108: dwrite d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:50.579 INFO:tasks.workunit.client.0.smithi076.stdout:1/109: dread d0/d1/d6/f9 [4194304,4194304] 0 2024-03-23T15:44:50.580 INFO:tasks.workunit.client.0.smithi076.stdout:1/110: mknod d0/d1/d6/c22 0 2024-03-23T15:44:50.582 INFO:tasks.workunit.client.0.smithi076.stdout:1/111: creat d0/d1/d6/f23 x:0 0 0 2024-03-23T15:44:50.583 INFO:tasks.workunit.client.0.smithi076.stdout:1/112: symlink d0/d2/d1f/l24 0 2024-03-23T15:44:50.584 INFO:tasks.workunit.client.0.smithi076.stdout:1/113: creat d0/d1/d6/d17/d20/f25 x:0 0 0 2024-03-23T15:44:50.588 INFO:tasks.workunit.client.0.smithi076.stdout:7/192: dwrite d8/de/d13/d18/d1a/f2a [0,4194304] 0 2024-03-23T15:44:50.591 INFO:tasks.workunit.client.0.smithi076.stdout:1/114: dread d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:50.604 INFO:tasks.workunit.client.0.smithi076.stdout:7/193: dwrite d8/fd [0,4194304] 0 2024-03-23T15:44:50.613 INFO:tasks.workunit.client.0.smithi076.stdout:1/115: dwrite d0/d1/d6/f23 [0,4194304] 0 2024-03-23T15:44:50.617 INFO:tasks.workunit.client.0.smithi076.stdout:1/116: unlink d0/c3 0 2024-03-23T15:44:50.619 INFO:tasks.workunit.client.0.smithi076.stdout:7/194: dwrite d8/de/d13/d18/d1a/f29 [0,4194304] 0 2024-03-23T15:44:50.619 INFO:tasks.workunit.client.0.smithi076.stdout:7/195: read - d8/de/d13/d18/f2b zero size 2024-03-23T15:44:50.621 INFO:tasks.workunit.client.0.smithi076.stdout:1/117: write d0/d1/d6/f9 [5047412,95162] 0 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:1/118: chown d0/d1/d6/d17/d20 58907 1 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:1/119: write d0/d1/d6/d17/d20/f25 [485375,73847] 0 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:3/187: sync 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:8/156: sync 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:2/200: sync 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:4/233: sync 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:5/188: sync 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:6/157: sync 2024-03-23T15:44:50.622 INFO:tasks.workunit.client.0.smithi076.stdout:9/193: sync 2024-03-23T15:44:50.623 INFO:tasks.workunit.client.0.smithi076.stdout:0/291: sync 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:5/189: write d1a/f1d [154805,79561] 0 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:5/190: readlink d1a/l2f 0 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:2/201: mknod d1/d4/d12/d43/c45 0 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:2/202: stat d1/d4/db/f3b 0 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:2/203: truncate d1/d4/db/d15/d29/f30 8884617 0 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:4/234: creat d8/d3a/d3b/f4c x:0 0 0 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:4/235: chown d8/d9/d1d 13 1 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:4/236: dread - d8/d3a/d3b/f4c zero size 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:3/188: dread d5/d17/f1a [0,4194304] 0 2024-03-23T15:44:50.624 INFO:tasks.workunit.client.0.smithi076.stdout:3/189: truncate d5/f1f 912862 0 2024-03-23T15:44:50.625 INFO:tasks.workunit.client.0.smithi076.stdout:5/191: symlink d1a/d39/l47 0 2024-03-23T15:44:50.626 INFO:tasks.workunit.client.0.smithi076.stdout:4/237: unlink d8/d3a/d3b/f4a 0 2024-03-23T15:44:50.626 INFO:tasks.workunit.client.0.smithi076.stdout:5/192: getdents d1a 0 2024-03-23T15:44:50.628 INFO:tasks.workunit.client.0.smithi076.stdout:7/196: unlink f0 0 2024-03-23T15:44:50.628 INFO:tasks.workunit.client.0.smithi076.stdout:4/238: creat d8/d3a/d18/f4d x:0 0 0 2024-03-23T15:44:50.628 INFO:tasks.workunit.client.0.smithi076.stdout:5/193: mkdir d1a/d39/d48 0 2024-03-23T15:44:50.629 INFO:tasks.workunit.client.0.smithi076.stdout:7/197: mkdir d8/de/d13/d3a 0 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:7/198: chown d8/de 37638 1 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:7/199: chown d8/d11/f25 22656 1 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:9/194: write d1/f12 [2796520,113818] 0 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:9/195: chown d1/d1f/f21 339187470 1 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:9/196: readlink d1/l2b 0 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:9/197: chown d1/l2b 17 1 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:9/198: write d1/f1a [1428349,27246] 0 2024-03-23T15:44:50.630 INFO:tasks.workunit.client.0.smithi076.stdout:4/239: rename d8/d3a/c3e to d8/d3a/d3b/c4e 0 2024-03-23T15:44:50.631 INFO:tasks.workunit.client.0.smithi076.stdout:5/194: mknod d1a/d33/d36/c49 0 2024-03-23T15:44:50.631 INFO:tasks.workunit.client.0.smithi076.stdout:7/200: creat d8/de/d13/d24/f3b x:0 0 0 2024-03-23T15:44:50.631 INFO:tasks.workunit.client.0.smithi076.stdout:9/199: mkdir d1/d2f/d3a 0 2024-03-23T15:44:50.631 INFO:tasks.workunit.client.0.smithi076.stdout:9/200: stat d1/d1f/c36 0 2024-03-23T15:44:50.633 INFO:tasks.workunit.client.0.smithi076.stdout:7/201: mkdir d8/de/d13/d18/d1a/d3c 0 2024-03-23T15:44:50.634 INFO:tasks.workunit.client.0.smithi076.stdout:4/240: unlink d8/d9/l1f 0 2024-03-23T15:44:50.635 INFO:tasks.workunit.client.0.smithi076.stdout:5/195: truncate f13 628588 0 2024-03-23T15:44:50.635 INFO:tasks.workunit.client.0.smithi076.stdout:7/202: symlink d8/de/d13/d3a/l3d 0 2024-03-23T15:44:50.636 INFO:tasks.workunit.client.0.smithi076.stdout:9/201: mknod d1/c3b 0 2024-03-23T15:44:50.636 INFO:tasks.workunit.client.0.smithi076.stdout:9/202: chown d1/c1e 5 1 2024-03-23T15:44:50.636 INFO:tasks.workunit.client.0.smithi076.stdout:9/203: truncate d1/d1f/f21 2447817 0 2024-03-23T15:44:50.638 INFO:tasks.workunit.client.0.smithi076.stdout:0/292: read d2/d1b/f1e [2870151,18324] 0 2024-03-23T15:44:50.638 INFO:tasks.workunit.client.0.smithi076.stdout:4/241: creat d8/d9/d1d/f4f x:0 0 0 2024-03-23T15:44:50.638 INFO:tasks.workunit.client.0.smithi076.stdout:7/203: creat d8/de/d13/d18/d1a/f3e x:0 0 0 2024-03-23T15:44:50.638 INFO:tasks.workunit.client.0.smithi076.stdout:7/204: readlink d8/l27 0 2024-03-23T15:44:50.638 INFO:tasks.workunit.client.0.smithi076.stdout:7/205: write d8/d11/f33 [951062,57724] 0 2024-03-23T15:44:50.638 INFO:tasks.workunit.client.0.smithi076.stdout:7/206: chown d8/de/d13/d18/d1a/c36 252 1 2024-03-23T15:44:50.638 INFO:tasks.workunit.client.0.smithi076.stdout:5/196: creat d1a/d2c/f4a x:0 0 0 2024-03-23T15:44:50.639 INFO:tasks.workunit.client.0.smithi076.stdout:9/204: creat d1/d1f/d29/d2d/d37/f3c x:0 0 0 2024-03-23T15:44:50.640 INFO:tasks.workunit.client.0.smithi076.stdout:7/207: creat d8/f3f x:0 0 0 2024-03-23T15:44:50.640 INFO:tasks.workunit.client.0.smithi076.stdout:7/208: chown d8/d11/f19 712045472 1 2024-03-23T15:44:50.640 INFO:tasks.workunit.client.0.smithi076.stdout:0/293: creat d2/de/d10/d43/f57 x:0 0 0 2024-03-23T15:44:50.641 INFO:tasks.workunit.client.0.smithi076.stdout:4/242: rmdir d8/d3a 39 2024-03-23T15:44:50.641 INFO:tasks.workunit.client.0.smithi076.stdout:4/243: chown d8/d9/ld 22924533 1 2024-03-23T15:44:50.642 INFO:tasks.workunit.client.0.smithi076.stdout:8/157: write d8/fc [12570938,68626] 0 2024-03-23T15:44:50.642 INFO:tasks.workunit.client.0.smithi076.stdout:2/204: write d1/d4/db/d15/d29/f30 [1264103,62203] 0 2024-03-23T15:44:50.642 INFO:tasks.workunit.client.0.smithi076.stdout:8/158: readlink d8/l15 0 2024-03-23T15:44:50.642 INFO:tasks.workunit.client.0.smithi076.stdout:8/159: chown d8/c20 0 1 2024-03-23T15:44:50.642 INFO:tasks.workunit.client.0.smithi076.stdout:7/209: truncate d8/de/d13/d18/d1a/f2a 1286323 0 2024-03-23T15:44:50.643 INFO:tasks.workunit.client.0.smithi076.stdout:7/210: chown d8/de/d13/d3a 376772785 1 2024-03-23T15:44:50.643 INFO:tasks.workunit.client.0.smithi076.stdout:9/205: mkdir d1/d2f/d3a/d3d 0 2024-03-23T15:44:50.643 INFO:tasks.workunit.client.0.smithi076.stdout:9/206: readlink d1/l1b 0 2024-03-23T15:44:50.644 INFO:tasks.workunit.client.0.smithi076.stdout:9/207: link d1/c13 d1/d1f/c3e 0 2024-03-23T15:44:50.645 INFO:tasks.workunit.client.0.smithi076.stdout:9/208: dread - d1/d1f/d29/d2d/f33 zero size 2024-03-23T15:44:50.646 INFO:tasks.workunit.client.0.smithi076.stdout:7/211: dread d8/de/d13/f2f [0,4194304] 0 2024-03-23T15:44:50.646 INFO:tasks.workunit.client.0.smithi076.stdout:4/244: dread d8/f42 [0,4194304] 0 2024-03-23T15:44:50.646 INFO:tasks.workunit.client.0.smithi076.stdout:9/209: mkdir d1/d2f/d3f 0 2024-03-23T15:44:50.647 INFO:tasks.workunit.client.0.smithi076.stdout:1/120: dwrite d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:50.647 INFO:tasks.workunit.client.0.smithi076.stdout:6/158: dwrite d1/dd/df/f13 [0,4194304] 0 2024-03-23T15:44:50.649 INFO:tasks.workunit.client.0.smithi076.stdout:4/245: rename d8/d9/l1e to d8/d3a/d3b/l50 0 2024-03-23T15:44:50.650 INFO:tasks.workunit.client.0.smithi076.stdout:9/210: creat d1/d1f/d29/f40 x:0 0 0 2024-03-23T15:44:50.653 INFO:tasks.workunit.client.0.smithi076.stdout:1/121: link d0/d1/d6/d17/f1d d0/f26 0 2024-03-23T15:44:50.653 INFO:tasks.workunit.client.0.smithi076.stdout:1/122: fsync d0/d1/d6/f23 0 2024-03-23T15:44:50.653 INFO:tasks.workunit.client.0.smithi076.stdout:5/197: dwrite d1a/d33/d3a/f41 [0,4194304] 0 2024-03-23T15:44:50.656 INFO:tasks.workunit.client.0.smithi076.stdout:4/246: rename d8/ce to d8/d3a/d18/c51 0 2024-03-23T15:44:50.657 INFO:tasks.workunit.client.0.smithi076.stdout:3/190: dwrite d5/d35/f36 [0,4194304] 0 2024-03-23T15:44:50.658 INFO:tasks.workunit.client.0.smithi076.stdout:1/123: symlink d0/l27 0 2024-03-23T15:44:50.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/124: dread d0/d1/d6/f9 [4194304,4194304] 0 2024-03-23T15:44:50.665 INFO:tasks.workunit.client.0.smithi076.stdout:5/198: mkdir d1a/d4b 0 2024-03-23T15:44:50.669 INFO:tasks.workunit.client.0.smithi076.stdout:1/125: dread d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:50.669 INFO:tasks.workunit.client.0.smithi076.stdout:4/247: mkdir d8/d3a/d18/d52 0 2024-03-23T15:44:50.669 INFO:tasks.workunit.client.0.smithi076.stdout:3/191: read d5/f9 [4129066,110340] 0 2024-03-23T15:44:50.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/205: dwrite d1/d4/db/d15/f21 [0,4194304] 0 2024-03-23T15:44:50.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/206: write d1/f19 [8398618,90603] 0 2024-03-23T15:44:50.671 INFO:tasks.workunit.client.0.smithi076.stdout:2/207: unlink d1/d4/db/fc 0 2024-03-23T15:44:50.671 INFO:tasks.workunit.client.0.smithi076.stdout:1/126: read d0/d1/d6/d17/f1d [3174544,47581] 0 2024-03-23T15:44:50.672 INFO:tasks.workunit.client.0.smithi076.stdout:0/294: dwrite d2/de/f22 [0,4194304] 0 2024-03-23T15:44:50.673 INFO:tasks.workunit.client.0.smithi076.stdout:2/208: dread d1/f14 [0,4194304] 0 2024-03-23T15:44:50.674 INFO:tasks.workunit.client.0.smithi076.stdout:2/209: mknod d1/d4/db/d15/c46 0 2024-03-23T15:44:50.674 INFO:tasks.workunit.client.0.smithi076.stdout:2/210: dread - d1/d2e/f42 zero size 2024-03-23T15:44:50.676 INFO:tasks.workunit.client.0.smithi076.stdout:7/212: dwrite d8/de/d13/d18/d1a/f29 [0,4194304] 0 2024-03-23T15:44:50.681 INFO:tasks.workunit.client.0.smithi076.stdout:2/211: truncate d1/f19 5416939 0 2024-03-23T15:44:50.683 INFO:tasks.workunit.client.0.smithi076.stdout:2/212: write d1/d4/d12/d1b/d32/f38 [1372499,69590] 0 2024-03-23T15:44:50.684 INFO:tasks.workunit.client.0.smithi076.stdout:5/199: rmdir d1a/d39/d48 0 2024-03-23T15:44:50.687 INFO:tasks.workunit.client.0.smithi076.stdout:3/192: link d5/d17/d1c/l33 d5/d17/d37/l3f 0 2024-03-23T15:44:50.688 INFO:tasks.workunit.client.0.smithi076.stdout:4/248: dwrite d8/d3a/d18/f20 [0,4194304] 0 2024-03-23T15:44:50.690 INFO:tasks.workunit.client.0.smithi076.stdout:1/127: unlink d0/d2/ce 0 2024-03-23T15:44:50.690 INFO:tasks.workunit.client.0.smithi076.stdout:1/128: stat d0/d1/d6/d17/d20 0 2024-03-23T15:44:50.693 INFO:tasks.workunit.client.0.smithi076.stdout:2/213: write d1/d4/d12/d1b/d32/f38 [2362651,1280] 0 2024-03-23T15:44:50.693 INFO:tasks.workunit.client.0.smithi076.stdout:1/129: write d0/d1/d6/f9 [3994043,92768] 0 2024-03-23T15:44:50.693 INFO:tasks.workunit.client.0.smithi076.stdout:0/295: symlink d2/d8/d2d/l58 0 2024-03-23T15:44:50.694 INFO:tasks.workunit.client.0.smithi076.stdout:0/296: getdents d2/d8/d52 0 2024-03-23T15:44:50.695 INFO:tasks.workunit.client.0.smithi076.stdout:2/214: symlink d1/d4/d12/l47 0 2024-03-23T15:44:50.696 INFO:tasks.workunit.client.0.smithi076.stdout:2/215: write d1/f14 [634258,47002] 0 2024-03-23T15:44:50.698 INFO:tasks.workunit.client.0.smithi076.stdout:7/213: dwrite d8/f3f [0,4194304] 0 2024-03-23T15:44:50.704 INFO:tasks.workunit.client.0.smithi076.stdout:3/193: unlink d5/d17/d1c/d32/f2e 0 2024-03-23T15:44:50.704 INFO:tasks.workunit.client.0.smithi076.stdout:5/200: chown la 311567 1 2024-03-23T15:44:50.704 INFO:tasks.workunit.client.0.smithi076.stdout:3/194: truncate d5/f16 2498400 0 2024-03-23T15:44:50.704 INFO:tasks.workunit.client.0.smithi076.stdout:5/201: write d1a/d33/f3b [881020,51398] 0 2024-03-23T15:44:50.708 INFO:tasks.workunit.client.0.smithi076.stdout:1/130: dwrite d0/f26 [4194304,4194304] 0 2024-03-23T15:44:50.708 INFO:tasks.workunit.client.0.smithi076.stdout:2/216: dwrite d1/d2e/f3f [0,4194304] 0 2024-03-23T15:44:50.709 INFO:tasks.workunit.client.0.smithi076.stdout:4/249: unlink d8/d3a/d18/d1b/f44 0 2024-03-23T15:44:50.709 INFO:tasks.workunit.client.0.smithi076.stdout:4/250: stat d8/f1a 0 2024-03-23T15:44:50.711 INFO:tasks.workunit.client.0.smithi076.stdout:3/195: truncate d5/d35/f36 3289015 0 2024-03-23T15:44:50.711 INFO:tasks.workunit.client.0.smithi076.stdout:3/196: fdatasync d5/f2a 0 2024-03-23T15:44:50.711 INFO:tasks.workunit.client.0.smithi076.stdout:3/197: write d5/f2a [48432,639] 0 2024-03-23T15:44:50.711 INFO:tasks.workunit.client.0.smithi076.stdout:3/198: fsync d5/f11 0 2024-03-23T15:44:50.711 INFO:tasks.workunit.client.0.smithi076.stdout:3/199: chown d5/d35/f36 113 1 2024-03-23T15:44:50.712 INFO:tasks.workunit.client.0.smithi076.stdout:2/217: creat d1/d4/d12/f48 x:0 0 0 2024-03-23T15:44:50.715 INFO:tasks.workunit.client.0.smithi076.stdout:4/251: dread d8/d9/fc [0,4194304] 0 2024-03-23T15:44:50.715 INFO:tasks.workunit.client.0.smithi076.stdout:4/252: stat d8/d3a/c47 0 2024-03-23T15:44:50.721 INFO:tasks.workunit.client.0.smithi076.stdout:2/218: dread d1/d4/db/d15/d29/f30 [0,4194304] 0 2024-03-23T15:44:50.723 INFO:tasks.workunit.client.0.smithi076.stdout:3/200: write d5/fd [7725630,99798] 0 2024-03-23T15:44:50.723 INFO:tasks.workunit.client.0.smithi076.stdout:7/214: dwrite d8/de/d13/d18/d1a/f32 [0,4194304] 0 2024-03-23T15:44:50.723 INFO:tasks.workunit.client.0.smithi076.stdout:3/201: stat d5/d17/d1c/d32/d39/l3d 0 2024-03-23T15:44:50.723 INFO:tasks.workunit.client.0.smithi076.stdout:7/215: chown d8/d11/c16 172 1 2024-03-23T15:44:50.724 INFO:tasks.workunit.client.0.smithi076.stdout:7/216: stat d8/de/d13/d18/f2b 0 2024-03-23T15:44:50.724 INFO:tasks.workunit.client.0.smithi076.stdout:7/217: truncate d8/de/d13/f15 3021932 0 2024-03-23T15:44:50.724 INFO:tasks.workunit.client.0.smithi076.stdout:7/218: chown d8/d11/f25 43662 1 2024-03-23T15:44:50.724 INFO:tasks.workunit.client.0.smithi076.stdout:7/219: fdatasync d8/d11/f33 0 2024-03-23T15:44:50.725 INFO:tasks.workunit.client.0.smithi076.stdout:5/202: dwrite f11 [0,4194304] 0 2024-03-23T15:44:50.726 INFO:tasks.workunit.client.0.smithi076.stdout:3/202: truncate d5/f1f 127567 0 2024-03-23T15:44:50.727 INFO:tasks.workunit.client.0.smithi076.stdout:1/131: dwrite d0/d1/d6/d17/f1d [4194304,4194304] 0 2024-03-23T15:44:50.728 INFO:tasks.workunit.client.0.smithi076.stdout:1/132: getdents d0/d1/d6/d17 0 2024-03-23T15:44:50.732 INFO:tasks.workunit.client.0.smithi076.stdout:6/159: rename d1/dd/d2a/d23/d36/f1e to d1/d2/f38 0 2024-03-23T15:44:50.734 INFO:tasks.workunit.client.0.smithi076.stdout:4/253: rename d8/d3a to d8/d3a/d53 22 2024-03-23T15:44:50.734 INFO:tasks.workunit.client.0.smithi076.stdout:0/297: rename d2/f4 to d2/de/d10/d3a/d47/f59 0 2024-03-23T15:44:50.735 INFO:tasks.workunit.client.0.smithi076.stdout:3/203: write d5/d17/d1c/f2b [1486574,110493] 0 2024-03-23T15:44:50.735 INFO:tasks.workunit.client.0.smithi076.stdout:0/298: symlink d2/d8/d2d/l5a 0 2024-03-23T15:44:50.737 INFO:tasks.workunit.client.0.smithi076.stdout:3/204: link d5/d17/d37/l3f d5/d17/d37/l40 0 2024-03-23T15:44:50.738 INFO:tasks.workunit.client.0.smithi076.stdout:0/299: mknod d2/de/d10/d12/c5b 0 2024-03-23T15:44:50.739 INFO:tasks.workunit.client.0.smithi076.stdout:4/254: write d8/d3a/d3b/f39 [7565835,101204] 0 2024-03-23T15:44:50.739 INFO:tasks.workunit.client.0.smithi076.stdout:4/255: write d8/f28 [1289156,25986] 0 2024-03-23T15:44:50.741 INFO:tasks.workunit.client.0.smithi076.stdout:0/300: creat d2/de/d10/d3a/f5c x:0 0 0 2024-03-23T15:44:50.741 INFO:tasks.workunit.client.0.smithi076.stdout:0/301: dread - d2/d8/f4f zero size 2024-03-23T15:44:50.743 INFO:tasks.workunit.client.0.smithi076.stdout:6/160: unlink d1/dd/c1a 0 2024-03-23T15:44:50.744 INFO:tasks.workunit.client.0.smithi076.stdout:4/256: creat d8/d41/f54 x:0 0 0 2024-03-23T15:44:50.744 INFO:tasks.workunit.client.0.smithi076.stdout:4/257: stat d8/d9/c38 0 2024-03-23T15:44:50.745 INFO:tasks.workunit.client.0.smithi076.stdout:4/258: write d8/d3a/d18/f2b [814335,22365] 0 2024-03-23T15:44:50.745 INFO:tasks.workunit.client.0.smithi076.stdout:4/259: write d8/d3a/d18/f21 [4124702,7619] 0 2024-03-23T15:44:50.745 INFO:tasks.workunit.client.0.smithi076.stdout:8/160: dwrite d8/fc [4194304,4194304] 0 2024-03-23T15:44:50.745 INFO:tasks.workunit.client.0.smithi076.stdout:3/205: dread d5/f2a [0,4194304] 0 2024-03-23T15:44:50.745 INFO:tasks.workunit.client.0.smithi076.stdout:3/206: write d5/f11 [4429593,74992] 0 2024-03-23T15:44:50.745 INFO:tasks.workunit.client.0.smithi076.stdout:3/207: dread d5/f12 [0,4194304] 0 2024-03-23T15:44:50.747 INFO:tasks.workunit.client.0.smithi076.stdout:2/219: dwrite d1/f14 [0,4194304] 0 2024-03-23T15:44:50.747 INFO:tasks.workunit.client.0.smithi076.stdout:6/161: mkdir d1/d4/d10/d39 0 2024-03-23T15:44:50.748 INFO:tasks.workunit.client.0.smithi076.stdout:4/260: unlink f5 0 2024-03-23T15:44:50.748 INFO:tasks.workunit.client.0.smithi076.stdout:3/208: dread d5/d17/f1a [0,4194304] 0 2024-03-23T15:44:50.752 INFO:tasks.workunit.client.0.smithi076.stdout:2/220: rename d1/d4/d12/c34 to d1/d4/d12/d1b/d32/c49 0 2024-03-23T15:44:50.752 INFO:tasks.workunit.client.0.smithi076.stdout:2/221: fdatasync d1/f5 0 2024-03-23T15:44:50.752 INFO:tasks.workunit.client.0.smithi076.stdout:2/222: chown d1/d4/d12/d43 3633215 1 2024-03-23T15:44:50.754 INFO:tasks.workunit.client.0.smithi076.stdout:7/220: dwrite f1 [0,4194304] 0 2024-03-23T15:44:50.754 INFO:tasks.workunit.client.0.smithi076.stdout:7/221: write d8/d11/f19 [1028672,65494] 0 2024-03-23T15:44:50.754 INFO:tasks.workunit.client.0.smithi076.stdout:5/203: dwrite d1a/f29 [0,4194304] 0 2024-03-23T15:44:50.754 INFO:tasks.workunit.client.0.smithi076.stdout:5/204: stat d1a/d2c/f2d 0 2024-03-23T15:44:50.755 INFO:tasks.workunit.client.0.smithi076.stdout:7/222: getdents d8/de 0 2024-03-23T15:44:50.756 INFO:tasks.workunit.client.0.smithi076.stdout:4/261: dread d8/d9/fc [0,4194304] 0 2024-03-23T15:44:50.756 INFO:tasks.workunit.client.0.smithi076.stdout:6/162: link d1/dd/f26 d1/dd/d2a/d23/f3a 0 2024-03-23T15:44:50.757 INFO:tasks.workunit.client.0.smithi076.stdout:4/262: write d8/f28 [2511695,117547] 0 2024-03-23T15:44:50.760 INFO:tasks.workunit.client.0.smithi076.stdout:3/209: creat d5/d17/d37/f41 x:0 0 0 2024-03-23T15:44:50.761 INFO:tasks.workunit.client.0.smithi076.stdout:3/210: stat d5/d17 0 2024-03-23T15:44:50.766 INFO:tasks.workunit.client.0.smithi076.stdout:0/302: dwrite d2/de/d10/d43/f50 [0,4194304] 0 2024-03-23T15:44:50.766 INFO:tasks.workunit.client.0.smithi076.stdout:8/161: dwrite d8/f1a [0,4194304] 0 2024-03-23T15:44:50.766 INFO:tasks.workunit.client.0.smithi076.stdout:8/162: fsync d8/f16 0 2024-03-23T15:44:50.768 INFO:tasks.workunit.client.0.smithi076.stdout:0/303: read d2/d8/d32/f39 [308047,4091] 0 2024-03-23T15:44:50.768 INFO:tasks.workunit.client.0.smithi076.stdout:6/163: mknod d1/dd/d2a/d23/c3b 0 2024-03-23T15:44:50.768 INFO:tasks.workunit.client.0.smithi076.stdout:6/164: write d1/f2c [765393,64631] 0 2024-03-23T15:44:50.771 INFO:tasks.workunit.client.0.smithi076.stdout:2/223: symlink d1/l4a 0 2024-03-23T15:44:50.771 INFO:tasks.workunit.client.0.smithi076.stdout:2/224: chown d1/d4/db/d3e 101 1 2024-03-23T15:44:50.771 INFO:tasks.workunit.client.0.smithi076.stdout:6/165: fsync d1/f2c 0 2024-03-23T15:44:50.772 INFO:tasks.workunit.client.0.smithi076.stdout:2/225: getdents d1/d4/db/d15 0 2024-03-23T15:44:50.772 INFO:tasks.workunit.client.0.smithi076.stdout:3/211: rename d5/l2f to d5/d17/d1c/d32/l42 0 2024-03-23T15:44:50.773 INFO:tasks.workunit.client.0.smithi076.stdout:6/166: mknod d1/d2/d27/c3c 0 2024-03-23T15:44:50.773 INFO:tasks.workunit.client.0.smithi076.stdout:6/167: write d1/d2/f38 [4194944,12057] 0 2024-03-23T15:44:50.777 INFO:tasks.workunit.client.0.smithi076.stdout:7/223: dwrite d8/de/d13/f14 [0,4194304] 0 2024-03-23T15:44:50.777 INFO:tasks.workunit.client.0.smithi076.stdout:8/163: dread d8/f1d [0,4194304] 0 2024-03-23T15:44:50.777 INFO:tasks.workunit.client.0.smithi076.stdout:8/164: fdatasync d8/fa 0 2024-03-23T15:44:50.778 INFO:tasks.workunit.client.0.smithi076.stdout:7/224: mkdir d8/de/d13/d40 0 2024-03-23T15:44:50.780 INFO:tasks.workunit.client.0.smithi076.stdout:7/225: mknod d8/de/d13/d18/d1a/d3c/c41 0 2024-03-23T15:44:50.781 INFO:tasks.workunit.client.0.smithi076.stdout:7/226: readlink d8/de/d13/l30 0 2024-03-23T15:44:50.781 INFO:tasks.workunit.client.0.smithi076.stdout:5/205: dwrite d1a/d2c/f2e [0,4194304] 0 2024-03-23T15:44:50.785 INFO:tasks.workunit.client.0.smithi076.stdout:4/263: dwrite d8/f28 [0,4194304] 0 2024-03-23T15:44:50.790 INFO:tasks.workunit.client.0.smithi076.stdout:3/212: creat d5/f43 x:0 0 0 2024-03-23T15:44:50.791 INFO:tasks.workunit.client.0.smithi076.stdout:6/168: getdents d1/dd 0 2024-03-23T15:44:50.792 INFO:tasks.workunit.client.0.smithi076.stdout:6/169: fdatasync d1/dd/f26 0 2024-03-23T15:44:50.792 INFO:tasks.workunit.client.0.smithi076.stdout:4/264: dread d8/d3a/d18/f20 [0,4194304] 0 2024-03-23T15:44:50.792 INFO:tasks.workunit.client.0.smithi076.stdout:0/304: dwrite d2/d1b/f42 [0,4194304] 0 2024-03-23T15:44:50.795 INFO:tasks.workunit.client.0.smithi076.stdout:9/211: sync 2024-03-23T15:44:50.796 INFO:tasks.workunit.client.0.smithi076.stdout:7/227: dread d8/f3f [0,4194304] 0 2024-03-23T15:44:50.796 INFO:tasks.workunit.client.0.smithi076.stdout:8/165: dwrite f3 [0,4194304] 0 2024-03-23T15:44:50.797 INFO:tasks.workunit.client.0.smithi076.stdout:7/228: symlink d8/de/d13/d18/d1a/d3c/l42 0 2024-03-23T15:44:50.797 INFO:tasks.workunit.client.0.smithi076.stdout:7/229: truncate d8/fa 1695836 0 2024-03-23T15:44:50.797 INFO:tasks.workunit.client.0.smithi076.stdout:7/230: stat d8/de/d13/d18/d1a/f34 0 2024-03-23T15:44:50.800 INFO:tasks.workunit.client.0.smithi076.stdout:5/206: rename d1a/d2c/f2d to d1a/d4b/f4c 0 2024-03-23T15:44:50.800 INFO:tasks.workunit.client.0.smithi076.stdout:5/207: chown d1a/d4b 67611 1 2024-03-23T15:44:50.800 INFO:tasks.workunit.client.0.smithi076.stdout:6/170: mknod d1/d4/d10/c3d 0 2024-03-23T15:44:50.800 INFO:tasks.workunit.client.0.smithi076.stdout:6/171: chown d1/f11 7 1 2024-03-23T15:44:50.801 INFO:tasks.workunit.client.0.smithi076.stdout:2/226: dwrite d1/d4/f10 [0,4194304] 0 2024-03-23T15:44:50.802 INFO:tasks.workunit.client.0.smithi076.stdout:7/231: read d8/d11/f19 [854612,99276] 0 2024-03-23T15:44:50.802 INFO:tasks.workunit.client.0.smithi076.stdout:2/227: creat d1/d4/db/d15/d29/f4b x:0 0 0 2024-03-23T15:44:50.807 INFO:tasks.workunit.client.0.smithi076.stdout:6/172: rmdir d1/d2/d27/d28 39 2024-03-23T15:44:50.809 INFO:tasks.workunit.client.0.smithi076.stdout:8/166: mknod d8/d26/d33/c35 0 2024-03-23T15:44:50.810 INFO:tasks.workunit.client.0.smithi076.stdout:3/213: dwrite d5/f13 [0,4194304] 0 2024-03-23T15:44:50.810 INFO:tasks.workunit.client.0.smithi076.stdout:7/232: write d8/de/d13/d18/d1a/f32 [1136033,103537] 0 2024-03-23T15:44:50.811 INFO:tasks.workunit.client.0.smithi076.stdout:7/233: creat d8/de/d13/d18/d1a/d3c/f43 x:0 0 0 2024-03-23T15:44:50.811 INFO:tasks.workunit.client.0.smithi076.stdout:5/208: rename fd to d1a/d33/f4d 0 2024-03-23T15:44:50.812 INFO:tasks.workunit.client.0.smithi076.stdout:7/234: symlink d8/l44 0 2024-03-23T15:44:50.812 INFO:tasks.workunit.client.0.smithi076.stdout:7/235: creat d8/de/d13/d40/f45 x:0 0 0 2024-03-23T15:44:50.813 INFO:tasks.workunit.client.0.smithi076.stdout:4/265: getdents d8/d9/d1d 0 2024-03-23T15:44:50.813 INFO:tasks.workunit.client.0.smithi076.stdout:4/266: write d8/d41/f54 [80644,84590] 0 2024-03-23T15:44:50.814 INFO:tasks.workunit.client.0.smithi076.stdout:9/212: dwrite d1/d1f/d29/f40 [0,4194304] 0 2024-03-23T15:44:50.815 INFO:tasks.workunit.client.0.smithi076.stdout:6/173: creat d1/dd/d2a/d23/d36/f3e x:0 0 0 2024-03-23T15:44:50.816 INFO:tasks.workunit.client.0.smithi076.stdout:8/167: getdents d8 0 2024-03-23T15:44:50.817 INFO:tasks.workunit.client.0.smithi076.stdout:2/228: link d1/d4/l3a d1/l4c 0 2024-03-23T15:44:50.818 INFO:tasks.workunit.client.0.smithi076.stdout:2/229: chown d1/d4/db/d15/f21 15 1 2024-03-23T15:44:50.818 INFO:tasks.workunit.client.0.smithi076.stdout:2/230: fdatasync d1/d4/db/f3b 0 2024-03-23T15:44:50.818 INFO:tasks.workunit.client.0.smithi076.stdout:2/231: truncate d1/d4/db/d15/d29/f4b 783984 0 2024-03-23T15:44:50.818 INFO:tasks.workunit.client.0.smithi076.stdout:2/232: mknod d1/d4/d12/d1b/c4d 0 2024-03-23T15:44:50.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:50 smithi192 ceph-mon[36653]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:50.820 INFO:tasks.workunit.client.0.smithi076.stdout:3/214: creat d5/d17/d1c/d32/d39/f44 x:0 0 0 2024-03-23T15:44:50.820 INFO:tasks.workunit.client.0.smithi076.stdout:7/236: getdents d8 0 2024-03-23T15:44:50.820 INFO:tasks.workunit.client.0.smithi076.stdout:8/168: dread d8/f19 [0,4194304] 0 2024-03-23T15:44:50.821 INFO:tasks.workunit.client.0.smithi076.stdout:5/209: fsync f18 0 2024-03-23T15:44:50.821 INFO:tasks.workunit.client.0.smithi076.stdout:7/237: mkdir d8/de/d13/d18/d1a/d3c/d46 0 2024-03-23T15:44:50.821 INFO:tasks.workunit.client.0.smithi076.stdout:7/238: fdatasync d8/de/d13/d40/f45 0 2024-03-23T15:44:50.825 INFO:tasks.workunit.client.0.smithi076.stdout:7/239: symlink d8/l47 0 2024-03-23T15:44:50.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:50 smithi076 ceph-mon[32176]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:50.827 INFO:tasks.workunit.client.0.smithi076.stdout:4/267: truncate d8/d3a/d3b/f26 4103613 0 2024-03-23T15:44:50.827 INFO:tasks.workunit.client.0.smithi076.stdout:4/268: fdatasync d8/d3a/f48 0 2024-03-23T15:44:50.827 INFO:tasks.workunit.client.0.smithi076.stdout:7/240: mkdir d8/de/d48 0 2024-03-23T15:44:50.828 INFO:tasks.workunit.client.0.smithi076.stdout:2/233: dread d1/d4/f10 [0,4194304] 0 2024-03-23T15:44:50.829 INFO:tasks.workunit.client.0.smithi076.stdout:9/213: symlink d1/d1f/d29/l41 0 2024-03-23T15:44:50.829 INFO:tasks.workunit.client.0.smithi076.stdout:7/241: mknod d8/de/d13/d18/c49 0 2024-03-23T15:44:50.831 INFO:tasks.workunit.client.0.smithi076.stdout:7/242: write d8/d11/f2d [392314,36312] 0 2024-03-23T15:44:50.831 INFO:tasks.workunit.client.0.smithi076.stdout:7/243: fsync d8/de/d13/d40/f45 0 2024-03-23T15:44:50.831 INFO:tasks.workunit.client.0.smithi076.stdout:7/244: dread - d8/de/d13/d18/d1a/f34 zero size 2024-03-23T15:44:50.831 INFO:tasks.workunit.client.0.smithi076.stdout:7/245: truncate d8/de/d13/d18/f2b 812297 0 2024-03-23T15:44:50.831 INFO:tasks.workunit.client.0.smithi076.stdout:6/174: symlink d1/l3f 0 2024-03-23T15:44:50.832 INFO:tasks.workunit.client.0.smithi076.stdout:3/215: link d5/d17/c19 d5/d35/d3c/c45 0 2024-03-23T15:44:50.833 INFO:tasks.workunit.client.0.smithi076.stdout:5/210: symlink d1a/d4b/l4e 0 2024-03-23T15:44:50.833 INFO:tasks.workunit.client.0.smithi076.stdout:5/211: fdatasync d1a/f1b 0 2024-03-23T15:44:50.833 INFO:tasks.workunit.client.0.smithi076.stdout:7/246: rename d8/de/d13/d18/d1a/d3c/l42 to d8/de/d13/d3a/l4a 0 2024-03-23T15:44:50.834 INFO:tasks.workunit.client.0.smithi076.stdout:3/216: dread d5/f14 [0,4194304] 0 2024-03-23T15:44:50.834 INFO:tasks.workunit.client.0.smithi076.stdout:7/247: symlink d8/de/d13/d3a/l4b 0 2024-03-23T15:44:50.835 INFO:tasks.workunit.client.0.smithi076.stdout:4/269: creat d8/d3a/d18/d1b/f55 x:0 0 0 2024-03-23T15:44:50.835 INFO:tasks.workunit.client.0.smithi076.stdout:4/270: chown d8/d3a/c35 0 1 2024-03-23T15:44:50.835 INFO:tasks.workunit.client.0.smithi076.stdout:4/271: chown d8/f1a 2257519 1 2024-03-23T15:44:50.838 INFO:tasks.workunit.client.0.smithi076.stdout:7/248: dread d8/d11/f2c [0,4194304] 0 2024-03-23T15:44:50.839 INFO:tasks.workunit.client.0.smithi076.stdout:6/175: link d1/l14 d1/dd/d2a/d23/d36/l40 0 2024-03-23T15:44:50.841 INFO:tasks.workunit.client.0.smithi076.stdout:2/234: write d1/f14 [3982887,47498] 0 2024-03-23T15:44:50.841 INFO:tasks.workunit.client.0.smithi076.stdout:3/217: symlink d5/d17/d37/l46 0 2024-03-23T15:44:50.841 INFO:tasks.workunit.client.0.smithi076.stdout:7/249: dread d8/d11/f2c [0,4194304] 0 2024-03-23T15:44:50.842 INFO:tasks.workunit.client.0.smithi076.stdout:4/272: chown d8/d9/d32 17961983 1 2024-03-23T15:44:50.843 INFO:tasks.workunit.client.0.smithi076.stdout:2/235: creat d1/d4/d12/f4e x:0 0 0 2024-03-23T15:44:50.845 INFO:tasks.workunit.client.0.smithi076.stdout:7/250: creat d8/de/d13/f4c x:0 0 0 2024-03-23T15:44:50.845 INFO:tasks.workunit.client.0.smithi076.stdout:3/218: mkdir d5/d17/d1c/d31/d47 0 2024-03-23T15:44:50.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/214: dwrite d1/d20/f25 [0,4194304] 0 2024-03-23T15:44:50.846 INFO:tasks.workunit.client.0.smithi076.stdout:2/236: mknod d1/d4/c4f 0 2024-03-23T15:44:50.846 INFO:tasks.workunit.client.0.smithi076.stdout:2/237: truncate d1/d4/d12/d1b/f1c 1534397 0 2024-03-23T15:44:50.847 INFO:tasks.workunit.client.0.smithi076.stdout:4/273: link d8/d9/f2d d8/f56 0 2024-03-23T15:44:50.847 INFO:tasks.workunit.client.0.smithi076.stdout:4/274: read - d8/d9/f2d zero size 2024-03-23T15:44:50.848 INFO:tasks.workunit.client.0.smithi076.stdout:8/169: dwrite d8/f19 [0,4194304] 0 2024-03-23T15:44:50.848 INFO:tasks.workunit.client.0.smithi076.stdout:8/170: stat d8/d2e/d30 0 2024-03-23T15:44:50.850 INFO:tasks.workunit.client.0.smithi076.stdout:7/251: truncate d8/de/d13/d18/d1a/f20 497234 0 2024-03-23T15:44:50.853 INFO:tasks.workunit.client.0.smithi076.stdout:5/212: write f12 [635596,93974] 0 2024-03-23T15:44:50.855 INFO:tasks.workunit.client.0.smithi076.stdout:9/215: read d1/f32 [403697,16820] 0 2024-03-23T15:44:50.855 INFO:tasks.workunit.client.0.smithi076.stdout:6/176: read d1/dd/d2a/d23/f3a [1711617,54417] 0 2024-03-23T15:44:50.855 INFO:tasks.workunit.client.0.smithi076.stdout:6/177: stat d1/d4/d10 0 2024-03-23T15:44:50.855 INFO:tasks.workunit.client.0.smithi076.stdout:5/213: mknod d1a/d33/d3a/c4f 0 2024-03-23T15:44:50.858 INFO:tasks.workunit.client.0.smithi076.stdout:5/214: write d1a/f2a [122685,76443] 0 2024-03-23T15:44:50.858 INFO:tasks.workunit.client.0.smithi076.stdout:5/215: write fc [5534305,14006] 0 2024-03-23T15:44:50.858 INFO:tasks.workunit.client.0.smithi076.stdout:9/216: creat d1/d2f/f42 x:0 0 0 2024-03-23T15:44:50.858 INFO:tasks.workunit.client.0.smithi076.stdout:5/216: chown cb 96288465 1 2024-03-23T15:44:50.858 INFO:tasks.workunit.client.0.smithi076.stdout:5/217: fsync d1a/d33/f3b 0 2024-03-23T15:44:50.858 INFO:tasks.workunit.client.0.smithi076.stdout:5/218: write d1a/d20/f34 [733118,102240] 0 2024-03-23T15:44:50.859 INFO:tasks.workunit.client.0.smithi076.stdout:9/217: creat d1/d20/f43 x:0 0 0 2024-03-23T15:44:50.862 INFO:tasks.workunit.client.0.smithi076.stdout:6/178: symlink d1/d2/l41 0 2024-03-23T15:44:50.862 INFO:tasks.workunit.client.0.smithi076.stdout:6/179: stat d1/d4/d10/f30 0 2024-03-23T15:44:50.864 INFO:tasks.workunit.client.0.smithi076.stdout:5/219: write d1a/d33/d3a/f43 [1414787,62276] 0 2024-03-23T15:44:50.866 INFO:tasks.workunit.client.0.smithi076.stdout:6/180: rename d1/d2/d27/c3c to d1/d2/c42 0 2024-03-23T15:44:50.866 INFO:tasks.workunit.client.0.smithi076.stdout:6/181: dread - d1/dd/d2a/d23/d36/f3e zero size 2024-03-23T15:44:50.866 INFO:tasks.workunit.client.0.smithi076.stdout:2/238: dwrite d1/d4/db/d15/f1f [0,4194304] 0 2024-03-23T15:44:50.868 INFO:tasks.workunit.client.0.smithi076.stdout:9/218: dread d1/d1f/f30 [0,4194304] 0 2024-03-23T15:44:50.868 INFO:tasks.workunit.client.0.smithi076.stdout:9/219: rename d1/d2f/d3a to d1/d2f/d3a/d3d/d44 22 2024-03-23T15:44:50.870 INFO:tasks.workunit.client.0.smithi076.stdout:6/182: mknod d1/dd/d2a/d23/c43 0 2024-03-23T15:44:50.870 INFO:tasks.workunit.client.0.smithi076.stdout:7/252: dwrite d8/de/d13/d18/d1a/d3c/f43 [0,4194304] 0 2024-03-23T15:44:50.871 INFO:tasks.workunit.client.0.smithi076.stdout:9/220: creat d1/d20/f45 x:0 0 0 2024-03-23T15:44:50.873 INFO:tasks.workunit.client.0.smithi076.stdout:9/221: unlink d1/d1f/c3e 0 2024-03-23T15:44:50.875 INFO:tasks.workunit.client.0.smithi076.stdout:2/239: creat d1/d4/d12/d43/f50 x:0 0 0 2024-03-23T15:44:50.875 INFO:tasks.workunit.client.0.smithi076.stdout:2/240: write d1/d4/db/f3b [431200,85831] 0 2024-03-23T15:44:50.875 INFO:tasks.workunit.client.0.smithi076.stdout:2/241: truncate d1/d4/d12/d43/f50 469085 0 2024-03-23T15:44:50.878 INFO:tasks.workunit.client.0.smithi076.stdout:7/253: mkdir d8/de/d13/d18/d1a/d3c/d4d 0 2024-03-23T15:44:50.879 INFO:tasks.workunit.client.0.smithi076.stdout:6/183: dread d1/f21 [0,4194304] 0 2024-03-23T15:44:50.879 INFO:tasks.workunit.client.0.smithi076.stdout:2/242: creat d1/d4/d12/d43/f51 x:0 0 0 2024-03-23T15:44:50.882 INFO:tasks.workunit.client.0.smithi076.stdout:7/254: mkdir d8/d4e 0 2024-03-23T15:44:50.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:50 smithi183 ceph-mon[36296]: scrub summary: idle+waiting paths [/] 2024-03-23T15:44:50.884 INFO:tasks.workunit.client.0.smithi076.stdout:7/255: creat d8/de/d13/d18/d1a/d3c/f4f x:0 0 0 2024-03-23T15:44:50.884 INFO:tasks.workunit.client.0.smithi076.stdout:7/256: stat d8/de/f26 0 2024-03-23T15:44:50.885 INFO:tasks.workunit.client.0.smithi076.stdout:2/243: dread d1/d4/db/d15/f21 [0,4194304] 0 2024-03-23T15:44:50.901 INFO:tasks.workunit.client.0.smithi076.stdout:2/244: dwrite d1/f19 [0,4194304] 0 2024-03-23T15:44:50.902 INFO:tasks.workunit.client.0.smithi076.stdout:6/184: rename d1/d2/d27/d28/c37 to d1/d4/d10/c44 0 2024-03-23T15:44:50.902 INFO:tasks.workunit.client.0.smithi076.stdout:6/185: write d1/d4/f35 [171618,115688] 0 2024-03-23T15:44:50.902 INFO:tasks.workunit.client.0.smithi076.stdout:2/245: mkdir d1/d4/d52 0 2024-03-23T15:44:50.904 INFO:tasks.workunit.client.0.smithi076.stdout:7/257: rename d8/de/d13/d18/d1a/f32 to d8/de/d13/d3a/f50 0 2024-03-23T15:44:50.905 INFO:tasks.workunit.client.0.smithi076.stdout:2/246: mknod d1/d4/d12/d1d/c53 0 2024-03-23T15:44:50.906 INFO:tasks.workunit.client.0.smithi076.stdout:7/258: rmdir d8/de/d13 39 2024-03-23T15:44:50.907 INFO:tasks.workunit.client.0.smithi076.stdout:7/259: creat d8/de/d13/d18/d1a/d3c/d46/f51 x:0 0 0 2024-03-23T15:44:50.908 INFO:tasks.workunit.client.0.smithi076.stdout:7/260: mknod d8/de/d13/d18/d1a/d3c/d46/c52 0 2024-03-23T15:44:50.909 INFO:tasks.workunit.client.0.smithi076.stdout:7/261: dread d8/de/d13/d18/f2b [0,4194304] 0 2024-03-23T15:44:50.909 INFO:tasks.workunit.client.0.smithi076.stdout:7/262: write d8/de/d13/d18/d1a/d3c/f4f [858865,130044] 0 2024-03-23T15:44:50.909 INFO:tasks.workunit.client.0.smithi076.stdout:7/263: write d8/de/f26 [810102,94241] 0 2024-03-23T15:44:50.909 INFO:tasks.workunit.client.0.smithi076.stdout:7/264: getdents d8/de/d13/d40 0 2024-03-23T15:44:50.911 INFO:tasks.workunit.client.0.smithi076.stdout:7/265: unlink d8/de/d13/d18/d1a/d3c/c41 0 2024-03-23T15:44:50.914 INFO:tasks.workunit.client.0.smithi076.stdout:2/247: write d1/f19 [2998933,54911] 0 2024-03-23T15:44:50.914 INFO:tasks.workunit.client.0.smithi076.stdout:2/248: read - d1/d4/d12/d43/f51 zero size 2024-03-23T15:44:50.915 INFO:tasks.workunit.client.0.smithi076.stdout:2/249: getdents d1/d4/d12 0 2024-03-23T15:44:50.917 INFO:tasks.workunit.client.0.smithi076.stdout:2/250: dread d1/d4/d12/f18 [0,4194304] 0 2024-03-23T15:44:50.921 INFO:tasks.workunit.client.0.smithi076.stdout:7/266: dread d8/de/d13/f14 [0,4194304] 0 2024-03-23T15:44:50.931 INFO:tasks.workunit.client.0.smithi076.stdout:7/267: dread d8/fd [0,4194304] 0 2024-03-23T15:44:50.932 INFO:tasks.workunit.client.0.smithi076.stdout:7/268: mknod d8/d4e/c53 0 2024-03-23T15:44:50.932 INFO:tasks.workunit.client.0.smithi076.stdout:7/269: chown d8/de/d13/d18 732748064 1 2024-03-23T15:44:50.932 INFO:tasks.workunit.client.0.smithi076.stdout:7/270: chown d8/d11/f19 1026649 1 2024-03-23T15:44:50.933 INFO:tasks.workunit.client.0.smithi076.stdout:7/271: read d8/d11/f33 [682553,44220] 0 2024-03-23T15:44:50.935 INFO:tasks.workunit.client.0.smithi076.stdout:7/272: link d8/de/d13/d18/d1a/l2e d8/de/d13/d18/d1a/l54 0 2024-03-23T15:44:50.937 INFO:tasks.workunit.client.0.smithi076.stdout:7/273: creat d8/de/d13/d24/f55 x:0 0 0 2024-03-23T15:44:50.949 INFO:tasks.workunit.client.0.smithi076.stdout:7/274: dwrite d8/d11/f2c [0,4194304] 0 2024-03-23T15:44:50.949 INFO:tasks.workunit.client.0.smithi076.stdout:7/275: chown d8/de/d13/d18/d1a/l38 1098 1 2024-03-23T15:44:50.949 INFO:tasks.workunit.client.0.smithi076.stdout:7/276: truncate d8/de/d13/d18/d1a/f3e 775006 0 2024-03-23T15:44:50.949 INFO:tasks.workunit.client.0.smithi076.stdout:7/277: write d8/de/d13/d18/f2b [140047,91920] 0 2024-03-23T15:44:50.952 INFO:tasks.workunit.client.0.smithi076.stdout:7/278: symlink d8/de/d13/d18/d1a/d3c/d46/l56 0 2024-03-23T15:44:50.952 INFO:tasks.workunit.client.0.smithi076.stdout:7/279: fdatasync f1 0 2024-03-23T15:44:50.954 INFO:tasks.workunit.client.0.smithi076.stdout:7/280: link d8/d11/f33 d8/d11/f57 0 2024-03-23T15:44:50.956 INFO:tasks.workunit.client.0.smithi076.stdout:7/281: write d8/fc [4714467,22271] 0 2024-03-23T15:44:50.958 INFO:tasks.workunit.client.0.smithi076.stdout:7/282: link d8/de/d13/d40/f45 d8/f58 0 2024-03-23T15:44:50.958 INFO:tasks.workunit.client.0.smithi076.stdout:7/283: chown d8/de/d13 62 1 2024-03-23T15:44:50.960 INFO:tasks.workunit.client.0.smithi076.stdout:7/284: creat d8/de/d13/d40/f59 x:0 0 0 2024-03-23T15:44:50.976 INFO:tasks.workunit.client.0.smithi076.stdout:7/285: dwrite d8/de/d13/d18/d1a/f2a [0,4194304] 0 2024-03-23T15:44:50.977 INFO:tasks.workunit.client.0.smithi076.stdout:7/286: mkdir d8/d11/d5a 0 2024-03-23T15:44:50.978 INFO:tasks.workunit.client.0.smithi076.stdout:7/287: link d8/fc d8/de/d13/d18/d1a/f5b 0 2024-03-23T15:44:50.994 INFO:tasks.workunit.client.0.smithi076.stdout:7/288: dwrite d8/d11/f2c [4194304,4194304] 0 2024-03-23T15:44:50.994 INFO:tasks.workunit.client.0.smithi076.stdout:7/289: chown d8/de/d13/d18/d1a/f34 201 1 2024-03-23T15:44:50.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/290: link f3 d8/de/d13/f5c 0 2024-03-23T15:44:51.000 INFO:tasks.workunit.client.0.smithi076.stdout:7/291: rename d8/de/d13/d18/d1a to d8/de/d13/d40/d5d 0 2024-03-23T15:44:51.003 INFO:tasks.workunit.client.0.smithi076.stdout:7/292: rename d8/de/d13/f4c to d8/d11/d5a/f5e 0 2024-03-23T15:44:51.005 INFO:tasks.workunit.client.0.smithi076.stdout:7/293: link d8/de/c10 d8/de/c5f 0 2024-03-23T15:44:51.081 INFO:tasks.workunit.client.0.smithi076.stdout:0/305: sync 2024-03-23T15:44:51.081 INFO:tasks.workunit.client.0.smithi076.stdout:1/133: sync 2024-03-23T15:44:51.081 INFO:tasks.workunit.client.0.smithi076.stdout:0/306: write d2/de/d10/d3a/d47/f4e [823312,25697] 0 2024-03-23T15:44:51.082 INFO:tasks.workunit.client.0.smithi076.stdout:0/307: creat d2/de/d10/f5d x:0 0 0 2024-03-23T15:44:51.082 INFO:tasks.workunit.client.0.smithi076.stdout:0/308: write d2/de/f1a [2411738,33490] 0 2024-03-23T15:44:51.082 INFO:tasks.workunit.client.0.smithi076.stdout:0/309: symlink d2/d8/d52/l5e 0 2024-03-23T15:44:51.083 INFO:tasks.workunit.client.0.smithi076.stdout:0/310: write d2/d8/f40 [105961,83470] 0 2024-03-23T15:44:51.098 INFO:tasks.workunit.client.0.smithi076.stdout:0/311: dwrite d2/f9 [0,4194304] 0 2024-03-23T15:44:51.098 INFO:tasks.workunit.client.0.smithi076.stdout:0/312: chown d2/de/d10/d43/d56 108136 1 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:6/186: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:5/220: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:8/171: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:8/172: rename d8 to d8/d36 22 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:3/219: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:9/222: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:7/294: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:2/251: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:4/275: sync 2024-03-23T15:44:51.223 INFO:tasks.workunit.client.0.smithi076.stdout:8/173: write d8/fa [63543,17905] 0 2024-03-23T15:44:51.224 INFO:tasks.workunit.client.0.smithi076.stdout:6/187: symlink d1/dd/d2a/d23/l45 0 2024-03-23T15:44:51.224 INFO:tasks.workunit.client.0.smithi076.stdout:7/295: rename d8/de/d13/d3a/l3d to d8/de/d13/d3a/l60 0 2024-03-23T15:44:51.224 INFO:tasks.workunit.client.0.smithi076.stdout:9/223: getdents d1 0 2024-03-23T15:44:51.225 INFO:tasks.workunit.client.0.smithi076.stdout:4/276: mknod d8/d9/d32/c57 0 2024-03-23T15:44:51.225 INFO:tasks.workunit.client.0.smithi076.stdout:4/277: chown d8/d3a/d18/c51 3450 1 2024-03-23T15:44:51.225 INFO:tasks.workunit.client.0.smithi076.stdout:2/252: creat d1/d4/f54 x:0 0 0 2024-03-23T15:44:51.226 INFO:tasks.workunit.client.0.smithi076.stdout:9/224: mkdir d1/d1f/d46 0 2024-03-23T15:44:51.227 INFO:tasks.workunit.client.0.smithi076.stdout:3/220: dread d5/f9 [0,4194304] 0 2024-03-23T15:44:51.227 INFO:tasks.workunit.client.0.smithi076.stdout:5/221: unlink d1a/f1d 0 2024-03-23T15:44:51.229 INFO:tasks.workunit.client.0.smithi076.stdout:4/278: mknod d8/d3a/d18/d1b/c58 0 2024-03-23T15:44:51.229 INFO:tasks.workunit.client.0.smithi076.stdout:6/188: unlink d1/l14 0 2024-03-23T15:44:51.230 INFO:tasks.workunit.client.0.smithi076.stdout:9/225: mkdir d1/d1f/d29/d2d/d37/d47 0 2024-03-23T15:44:51.230 INFO:tasks.workunit.client.0.smithi076.stdout:7/296: unlink d8/d11/d5a/f5e 0 2024-03-23T15:44:51.230 INFO:tasks.workunit.client.0.smithi076.stdout:7/297: dread - d8/de/d13/d40/d5d/d3c/d46/f51 zero size 2024-03-23T15:44:51.231 INFO:tasks.workunit.client.0.smithi076.stdout:3/221: mkdir d5/d17/d37/d48 0 2024-03-23T15:44:51.235 INFO:tasks.workunit.client.0.smithi076.stdout:6/189: dread d1/f2b [0,4194304] 0 2024-03-23T15:44:51.235 INFO:tasks.workunit.client.0.smithi076.stdout:7/298: mknod d8/d11/c61 0 2024-03-23T15:44:51.235 INFO:tasks.workunit.client.0.smithi076.stdout:7/299: dread - d8/d11/f25 zero size 2024-03-23T15:44:51.235 INFO:tasks.workunit.client.0.smithi076.stdout:9/226: rmdir d1/d2f/d3a/d3d 0 2024-03-23T15:44:51.235 INFO:tasks.workunit.client.0.smithi076.stdout:7/300: chown d8/l27 156 1 2024-03-23T15:44:51.235 INFO:tasks.workunit.client.0.smithi076.stdout:3/222: creat d5/d17/d37/f49 x:0 0 0 2024-03-23T15:44:51.235 INFO:tasks.workunit.client.0.smithi076.stdout:3/223: write d5/f9 [4578510,52350] 0 2024-03-23T15:44:51.236 INFO:tasks.workunit.client.0.smithi076.stdout:9/227: rename d1/d20/f28 to d1/f48 0 2024-03-23T15:44:51.236 INFO:tasks.workunit.client.0.smithi076.stdout:9/228: dread - d1/d20/f45 zero size 2024-03-23T15:44:51.236 INFO:tasks.workunit.client.0.smithi076.stdout:7/301: unlink d8/fa 0 2024-03-23T15:44:51.238 INFO:tasks.workunit.client.0.smithi076.stdout:7/302: truncate d8/de/d13/f21 438124 0 2024-03-23T15:44:51.238 INFO:tasks.workunit.client.0.smithi076.stdout:7/303: write d8/de/d13/d40/d5d/f34 [266896,88206] 0 2024-03-23T15:44:51.240 INFO:tasks.workunit.client.0.smithi076.stdout:4/279: read d8/d9/f17 [2915936,81183] 0 2024-03-23T15:44:51.242 INFO:tasks.workunit.client.0.smithi076.stdout:4/280: rename d8/d41/c45 to d8/d9/d32/c59 0 2024-03-23T15:44:51.242 INFO:tasks.workunit.client.0.smithi076.stdout:4/281: getdents d8/d3a/d18/d52 0 2024-03-23T15:44:51.243 INFO:tasks.workunit.client.0.smithi076.stdout:4/282: fsync d8/f42 0 2024-03-23T15:44:51.243 INFO:tasks.workunit.client.0.smithi076.stdout:8/174: dwrite d8/f14 [0,4194304] 0 2024-03-23T15:44:51.244 INFO:tasks.workunit.client.0.smithi076.stdout:9/229: dread d1/d1f/f21 [0,4194304] 0 2024-03-23T15:44:51.244 INFO:tasks.workunit.client.0.smithi076.stdout:4/283: creat d8/d9/f5a x:0 0 0 2024-03-23T15:44:51.244 INFO:tasks.workunit.client.0.smithi076.stdout:4/284: chown d8/d9/d1d/f4f 2532 1 2024-03-23T15:44:51.246 INFO:tasks.workunit.client.0.smithi076.stdout:9/230: creat d1/d2f/d3f/f49 x:0 0 0 2024-03-23T15:44:51.248 INFO:tasks.workunit.client.0.smithi076.stdout:4/285: creat d8/d9/d32/f5b x:0 0 0 2024-03-23T15:44:51.250 INFO:tasks.workunit.client.0.smithi076.stdout:9/231: mkdir d1/d1f/d46/d4a 0 2024-03-23T15:44:51.253 INFO:tasks.workunit.client.0.smithi076.stdout:5/222: dwrite f18 [0,4194304] 0 2024-03-23T15:44:51.253 INFO:tasks.workunit.client.0.smithi076.stdout:5/223: write d1a/f1b [450415,46108] 0 2024-03-23T15:44:51.259 INFO:tasks.workunit.client.0.smithi076.stdout:5/224: rename d1a/d2c/f2e to d1a/d4b/f50 0 2024-03-23T15:44:51.262 INFO:tasks.workunit.client.0.smithi076.stdout:3/224: dwrite d5/d17/d37/f49 [0,4194304] 0 2024-03-23T15:44:51.264 INFO:tasks.workunit.client.0.smithi076.stdout:7/304: dwrite d8/de/d13/f14 [0,4194304] 0 2024-03-23T15:44:51.264 INFO:tasks.workunit.client.0.smithi076.stdout:5/225: getdents d1a/d33/d3a 0 2024-03-23T15:44:51.266 INFO:tasks.workunit.client.0.smithi076.stdout:5/226: dread d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:51.266 INFO:tasks.workunit.client.0.smithi076.stdout:6/190: dwrite d1/f2c [0,4194304] 0 2024-03-23T15:44:51.270 INFO:tasks.workunit.client.0.smithi076.stdout:3/225: stat d5/d17/d1c/d32/l42 0 2024-03-23T15:44:51.270 INFO:tasks.workunit.client.0.smithi076.stdout:3/226: readlink d5/l23 0 2024-03-23T15:44:51.270 INFO:tasks.workunit.client.0.smithi076.stdout:3/227: chown d5/f9 2 1 2024-03-23T15:44:51.271 INFO:tasks.workunit.client.0.smithi076.stdout:3/228: readlink d5/d17/d37/l46 0 2024-03-23T15:44:51.271 INFO:tasks.workunit.client.0.smithi076.stdout:4/286: dwrite d8/f36 [0,4194304] 0 2024-03-23T15:44:51.274 INFO:tasks.workunit.client.0.smithi076.stdout:5/227: rename d1a/d20/f30 to d1a/d33/f51 0 2024-03-23T15:44:51.275 INFO:tasks.workunit.client.0.smithi076.stdout:4/287: write d8/d3a/d18/f20 [2015376,79917] 0 2024-03-23T15:44:51.276 INFO:tasks.workunit.client.0.smithi076.stdout:2/253: rmdir d1 39 2024-03-23T15:44:51.277 INFO:tasks.workunit.client.0.smithi076.stdout:2/254: creat d1/d4/db/d15/f55 x:0 0 0 2024-03-23T15:44:51.278 INFO:tasks.workunit.client.0.smithi076.stdout:9/232: dwrite d1/d1f/f30 [4194304,4194304] 0 2024-03-23T15:44:51.278 INFO:tasks.workunit.client.0.smithi076.stdout:9/233: chown d1/c14 12994351 1 2024-03-23T15:44:51.279 INFO:tasks.workunit.client.0.smithi076.stdout:2/255: rename d1/d2e/l37 to d1/d4/l56 0 2024-03-23T15:44:51.279 INFO:tasks.workunit.client.0.smithi076.stdout:2/256: fsync d1/d4/d12/f26 0 2024-03-23T15:44:51.280 INFO:tasks.workunit.client.0.smithi076.stdout:6/191: getdents d1/dd/df 0 2024-03-23T15:44:51.281 INFO:tasks.workunit.client.0.smithi076.stdout:7/305: dwrite d8/de/d13/d24/f3b [0,4194304] 0 2024-03-23T15:44:51.283 INFO:tasks.workunit.client.0.smithi076.stdout:3/229: mkdir d5/d17/d1c/d32/d4a 0 2024-03-23T15:44:51.283 INFO:tasks.workunit.client.0.smithi076.stdout:3/230: fsync d5/f2d 0 2024-03-23T15:44:51.283 INFO:tasks.workunit.client.0.smithi076.stdout:2/257: unlink d1/d4/d12/d1b/l27 0 2024-03-23T15:44:51.284 INFO:tasks.workunit.client.0.smithi076.stdout:7/306: read d8/de/d13/d40/d5d/f34 [87204,86808] 0 2024-03-23T15:44:51.284 INFO:tasks.workunit.client.0.smithi076.stdout:7/307: chown d8/de/d13/d3a/l60 1694148 1 2024-03-23T15:44:51.287 INFO:tasks.workunit.client.0.smithi076.stdout:2/258: mknod d1/d4/d12/c57 0 2024-03-23T15:44:51.287 INFO:tasks.workunit.client.0.smithi076.stdout:2/259: write d1/d4/d12/f18 [1400020,33469] 0 2024-03-23T15:44:51.290 INFO:tasks.workunit.client.0.smithi076.stdout:6/192: dread d1/dd/df/f13 [0,4194304] 0 2024-03-23T15:44:51.290 INFO:tasks.workunit.client.0.smithi076.stdout:5/228: link d1a/f3c d1a/d39/f52 0 2024-03-23T15:44:51.290 INFO:tasks.workunit.client.0.smithi076.stdout:2/260: symlink d1/d4/db/d3e/l58 0 2024-03-23T15:44:51.290 INFO:tasks.workunit.client.0.smithi076.stdout:2/261: dread - d1/d4/d12/f23 zero size 2024-03-23T15:44:51.290 INFO:tasks.workunit.client.0.smithi076.stdout:2/262: read - d1/d4/d12/d43/f51 zero size 2024-03-23T15:44:51.292 INFO:tasks.workunit.client.0.smithi076.stdout:4/288: creat d8/d9/f5c x:0 0 0 2024-03-23T15:44:51.292 INFO:tasks.workunit.client.0.smithi076.stdout:4/289: fsync d8/d3a/d18/d1b/f55 0 2024-03-23T15:44:51.292 INFO:tasks.workunit.client.0.smithi076.stdout:6/193: creat d1/d2/d27/f46 x:0 0 0 2024-03-23T15:44:51.294 INFO:tasks.workunit.client.0.smithi076.stdout:2/263: rename d1/d4/d12/d1b/d32/c49 to d1/d4/db/c59 0 2024-03-23T15:44:51.294 INFO:tasks.workunit.client.0.smithi076.stdout:2/264: chown d1/d4/d12/d1b/d32/f38 504515 1 2024-03-23T15:44:51.294 INFO:tasks.workunit.client.0.smithi076.stdout:7/308: rmdir d8/de 39 2024-03-23T15:44:51.295 INFO:tasks.workunit.client.0.smithi076.stdout:2/265: mknod d1/d2e/c5a 0 2024-03-23T15:44:51.296 INFO:tasks.workunit.client.0.smithi076.stdout:9/234: dwrite d1/f15 [0,4194304] 0 2024-03-23T15:44:51.297 INFO:tasks.workunit.client.0.smithi076.stdout:3/231: symlink d5/d17/d37/l4b 0 2024-03-23T15:44:51.297 INFO:tasks.workunit.client.0.smithi076.stdout:3/232: write d5/f2a [563636,94418] 0 2024-03-23T15:44:51.297 INFO:tasks.workunit.client.0.smithi076.stdout:7/309: creat d8/f62 x:0 0 0 2024-03-23T15:44:51.297 INFO:tasks.workunit.client.0.smithi076.stdout:7/310: chown d8/de/d13/l1f 1844943 1 2024-03-23T15:44:51.299 INFO:tasks.workunit.client.0.smithi076.stdout:2/266: link d1/d4/db/d15/f1f d1/d4/db/f5b 0 2024-03-23T15:44:51.299 INFO:tasks.workunit.client.0.smithi076.stdout:5/229: creat d1a/d20/f53 x:0 0 0 2024-03-23T15:44:51.301 INFO:tasks.workunit.client.0.smithi076.stdout:2/267: mknod d1/d4/c5c 0 2024-03-23T15:44:51.301 INFO:tasks.workunit.client.0.smithi076.stdout:2/268: chown d1/d2e/c5a 12 1 2024-03-23T15:44:51.302 INFO:tasks.workunit.client.0.smithi076.stdout:9/235: mknod d1/d2f/c4b 0 2024-03-23T15:44:51.304 INFO:tasks.workunit.client.0.smithi076.stdout:5/230: dread d1a/d33/f3b [0,4194304] 0 2024-03-23T15:44:51.304 INFO:tasks.workunit.client.0.smithi076.stdout:5/231: stat d1a/d33/d36/f3d 0 2024-03-23T15:44:51.305 INFO:tasks.workunit.client.0.smithi076.stdout:7/311: unlink d8/de/d13/d40/d5d/f28 0 2024-03-23T15:44:51.305 INFO:tasks.workunit.client.0.smithi076.stdout:7/312: stat d8/de/d13/d40/d5d/f29 0 2024-03-23T15:44:51.306 INFO:tasks.workunit.client.0.smithi076.stdout:9/236: dread d1/f19 [4194304,4194304] 0 2024-03-23T15:44:51.309 INFO:tasks.workunit.client.0.smithi076.stdout:2/269: read d1/d4/db/f5b [3502792,102141] 0 2024-03-23T15:44:51.309 INFO:tasks.workunit.client.0.smithi076.stdout:2/270: read d1/d4/f9 [93529,3599] 0 2024-03-23T15:44:51.310 INFO:tasks.workunit.client.0.smithi076.stdout:1/134: sync 2024-03-23T15:44:51.311 INFO:tasks.workunit.client.0.smithi076.stdout:2/271: mknod d1/d4/c5d 0 2024-03-23T15:44:51.313 INFO:tasks.workunit.client.0.smithi076.stdout:4/290: write d8/f1a [1866631,82903] 0 2024-03-23T15:44:51.313 INFO:tasks.workunit.client.0.smithi076.stdout:9/237: rename d1/d1f/c36 to d1/d1f/d29/d2d/d37/d47/c4c 0 2024-03-23T15:44:51.313 INFO:tasks.workunit.client.0.smithi076.stdout:9/238: fsync d1/d20/f43 0 2024-03-23T15:44:51.316 INFO:tasks.workunit.client.0.smithi076.stdout:2/272: dread d1/d4/db/d15/f21 [0,4194304] 0 2024-03-23T15:44:51.316 INFO:tasks.workunit.client.0.smithi076.stdout:1/135: mknod d0/d1/d6/d17/d20/c28 0 2024-03-23T15:44:51.317 INFO:tasks.workunit.client.0.smithi076.stdout:5/232: write f11 [3000307,116120] 0 2024-03-23T15:44:51.318 INFO:tasks.workunit.client.0.smithi076.stdout:9/239: getdents d1 0 2024-03-23T15:44:51.318 INFO:tasks.workunit.client.0.smithi076.stdout:2/273: symlink d1/d4/db/d15/l5e 0 2024-03-23T15:44:51.318 INFO:tasks.workunit.client.0.smithi076.stdout:2/274: chown d1/d4/d52 104764 1 2024-03-23T15:44:51.320 INFO:tasks.workunit.client.0.smithi076.stdout:2/275: link d1/d4/f54 d1/d4/f5f 0 2024-03-23T15:44:51.321 INFO:tasks.workunit.client.0.smithi076.stdout:1/136: dread d0/d1/d6/d17/d20/f25 [0,4194304] 0 2024-03-23T15:44:51.321 INFO:tasks.workunit.client.0.smithi076.stdout:6/194: dwrite d1/d4/f35 [0,4194304] 0 2024-03-23T15:44:51.324 INFO:tasks.workunit.client.0.smithi076.stdout:1/137: mknod d0/d2/c29 0 2024-03-23T15:44:51.325 INFO:tasks.workunit.client.0.smithi076.stdout:6/195: write d1/dd/d2a/d23/d36/f3e [566720,16800] 0 2024-03-23T15:44:51.325 INFO:tasks.workunit.client.0.smithi076.stdout:6/196: chown d1/d4/d10/d39 25275 1 2024-03-23T15:44:51.325 INFO:tasks.workunit.client.0.smithi076.stdout:6/197: write d1/dd/f26 [5021805,68277] 0 2024-03-23T15:44:51.325 INFO:tasks.workunit.client.0.smithi076.stdout:6/198: chown d1/d2 93282538 1 2024-03-23T15:44:51.325 INFO:tasks.workunit.client.0.smithi076.stdout:2/276: dread d1/d4/db/f5b [0,4194304] 0 2024-03-23T15:44:51.329 INFO:tasks.workunit.client.0.smithi076.stdout:7/313: dwrite d8/d11/f2c [4194304,4194304] 0 2024-03-23T15:44:51.330 INFO:tasks.workunit.client.0.smithi076.stdout:2/277: dread d1/d4/db/d15/f1f [0,4194304] 0 2024-03-23T15:44:51.331 INFO:tasks.workunit.client.0.smithi076.stdout:7/314: symlink d8/de/l63 0 2024-03-23T15:44:51.332 INFO:tasks.workunit.client.0.smithi076.stdout:3/233: dwrite d5/f2a [0,4194304] 0 2024-03-23T15:44:51.334 INFO:tasks.workunit.client.0.smithi076.stdout:4/291: dwrite d8/d9/fc [0,4194304] 0 2024-03-23T15:44:51.335 INFO:tasks.workunit.client.0.smithi076.stdout:6/199: dread d1/dd/df/f13 [0,4194304] 0 2024-03-23T15:44:51.336 INFO:tasks.workunit.client.0.smithi076.stdout:4/292: creat d8/f5d x:0 0 0 2024-03-23T15:44:51.338 INFO:tasks.workunit.client.0.smithi076.stdout:6/200: truncate d1/f11 146301 0 2024-03-23T15:44:51.339 INFO:tasks.workunit.client.0.smithi076.stdout:5/233: dwrite d1a/f3f [0,4194304] 0 2024-03-23T15:44:51.339 INFO:tasks.workunit.client.0.smithi076.stdout:9/240: dwrite d1/d20/f25 [0,4194304] 0 2024-03-23T15:44:51.339 INFO:tasks.workunit.client.0.smithi076.stdout:9/241: stat d1/f1a 0 2024-03-23T15:44:51.340 INFO:tasks.workunit.client.0.smithi076.stdout:4/293: symlink d8/d3a/l5e 0 2024-03-23T15:44:51.340 INFO:tasks.workunit.client.0.smithi076.stdout:4/294: read - d8/d3a/d3b/f4c zero size 2024-03-23T15:44:51.341 INFO:tasks.workunit.client.0.smithi076.stdout:6/201: symlink d1/dd/l47 0 2024-03-23T15:44:51.342 INFO:tasks.workunit.client.0.smithi076.stdout:2/278: mknod d1/c60 0 2024-03-23T15:44:51.344 INFO:tasks.workunit.client.0.smithi076.stdout:0/313: sync 2024-03-23T15:44:51.344 INFO:tasks.workunit.client.0.smithi076.stdout:0/314: dread - d2/d8/d32/f48 zero size 2024-03-23T15:44:51.344 INFO:tasks.workunit.client.0.smithi076.stdout:0/315: fsync d2/d8/f53 0 2024-03-23T15:44:51.344 INFO:tasks.workunit.client.0.smithi076.stdout:7/315: write d8/d11/f2c [3516979,3643] 0 2024-03-23T15:44:51.344 INFO:tasks.workunit.client.0.smithi076.stdout:9/242: mkdir d1/d2f/d3a/d4d 0 2024-03-23T15:44:51.345 INFO:tasks.workunit.client.0.smithi076.stdout:7/316: creat d8/de/d13/d40/d5d/d3c/d46/f64 x:0 0 0 2024-03-23T15:44:51.345 INFO:tasks.workunit.client.0.smithi076.stdout:7/317: chown f3 5902 1 2024-03-23T15:44:51.346 INFO:tasks.workunit.client.0.smithi076.stdout:4/295: truncate d8/d3a/d18/f20 1699221 0 2024-03-23T15:44:51.346 INFO:tasks.workunit.client.0.smithi076.stdout:4/296: chown d8/d3a/d3b/f4c 2 1 2024-03-23T15:44:51.346 INFO:tasks.workunit.client.0.smithi076.stdout:7/318: creat d8/de/d13/d40/d5d/f65 x:0 0 0 2024-03-23T15:44:51.347 INFO:tasks.workunit.client.0.smithi076.stdout:9/243: dread d1/d1f/f21 [0,4194304] 0 2024-03-23T15:44:51.348 INFO:tasks.workunit.client.0.smithi076.stdout:3/234: dwrite d5/f11 [4194304,4194304] 0 2024-03-23T15:44:51.349 INFO:tasks.workunit.client.0.smithi076.stdout:4/297: symlink d8/d3a/d18/d1b/l5f 0 2024-03-23T15:44:51.350 INFO:tasks.workunit.client.0.smithi076.stdout:6/202: getdents d1/dd 0 2024-03-23T15:44:51.352 INFO:tasks.workunit.client.0.smithi076.stdout:4/298: write d8/d3a/d18/f2b [550329,84953] 0 2024-03-23T15:44:51.352 INFO:tasks.workunit.client.0.smithi076.stdout:4/299: write d8/f42 [4272395,105400] 0 2024-03-23T15:44:51.352 INFO:tasks.workunit.client.0.smithi076.stdout:2/279: write d1/d4/d12/f20 [1957669,46310] 0 2024-03-23T15:44:51.352 INFO:tasks.workunit.client.0.smithi076.stdout:9/244: mknod d1/d1f/d29/d2d/c4e 0 2024-03-23T15:44:51.352 INFO:tasks.workunit.client.0.smithi076.stdout:9/245: chown d1/d20/f25 9502297 1 2024-03-23T15:44:51.357 INFO:tasks.workunit.client.0.smithi076.stdout:2/280: dread d1/d4/d12/d1b/d32/f38 [0,4194304] 0 2024-03-23T15:44:51.362 INFO:tasks.workunit.client.0.smithi076.stdout:3/235: mkdir d5/d35/d4c 0 2024-03-23T15:44:51.362 INFO:tasks.workunit.client.0.smithi076.stdout:9/246: symlink d1/d1f/d46/d4a/l4f 0 2024-03-23T15:44:51.362 INFO:tasks.workunit.client.0.smithi076.stdout:9/247: truncate d1/d20/f43 457731 0 2024-03-23T15:44:51.362 INFO:tasks.workunit.client.0.smithi076.stdout:9/248: stat d1/c1e 0 2024-03-23T15:44:51.366 INFO:tasks.workunit.client.0.smithi076.stdout:3/236: read d5/f12 [361674,45963] 0 2024-03-23T15:44:51.368 INFO:tasks.workunit.client.0.smithi076.stdout:9/249: creat d1/d1f/d46/d4a/f50 x:0 0 0 2024-03-23T15:44:51.368 INFO:tasks.workunit.client.0.smithi076.stdout:8/175: sync 2024-03-23T15:44:51.369 INFO:tasks.workunit.client.0.smithi076.stdout:0/316: dwrite d2/d8/d32/f39 [0,4194304] 0 2024-03-23T15:44:51.369 INFO:tasks.workunit.client.0.smithi076.stdout:5/234: dwrite d1a/f2a [0,4194304] 0 2024-03-23T15:44:51.371 INFO:tasks.workunit.client.0.smithi076.stdout:8/176: creat d8/d26/d27/f37 x:0 0 0 2024-03-23T15:44:51.371 INFO:tasks.workunit.client.0.smithi076.stdout:9/250: getdents d1/d1f/d29/d2d/d37 0 2024-03-23T15:44:51.375 INFO:tasks.workunit.client.0.smithi076.stdout:4/300: dwrite d8/f3d [0,4194304] 0 2024-03-23T15:44:51.377 INFO:tasks.workunit.client.0.smithi076.stdout:8/177: symlink d8/d2e/l38 0 2024-03-23T15:44:51.378 INFO:tasks.workunit.client.0.smithi076.stdout:9/251: mknod d1/d1f/d29/d2d/d37/d47/c51 0 2024-03-23T15:44:51.378 INFO:tasks.workunit.client.0.smithi076.stdout:9/252: chown d1/f48 462999 1 2024-03-23T15:44:51.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/203: dwrite d1/dd/d2a/d23/f3a [4194304,4194304] 0 2024-03-23T15:44:51.381 INFO:tasks.workunit.client.0.smithi076.stdout:5/235: write d1a/f29 [4098870,35913] 0 2024-03-23T15:44:51.382 INFO:tasks.workunit.client.0.smithi076.stdout:4/301: rename d8/d9/f17 to d8/d9/d1d/f60 0 2024-03-23T15:44:51.383 INFO:tasks.workunit.client.0.smithi076.stdout:7/319: dwrite d8/de/d13/f2f [0,4194304] 0 2024-03-23T15:44:51.383 INFO:tasks.workunit.client.0.smithi076.stdout:3/237: dwrite d5/d17/d1c/d32/d39/f44 [0,4194304] 0 2024-03-23T15:44:51.383 INFO:tasks.workunit.client.0.smithi076.stdout:3/238: chown d5/c10 33 1 2024-03-23T15:44:51.383 INFO:tasks.workunit.client.0.smithi076.stdout:3/239: write d5/d35/f36 [3350777,117156] 0 2024-03-23T15:44:51.383 INFO:tasks.workunit.client.0.smithi076.stdout:3/240: fsync d5/f16 0 2024-03-23T15:44:51.383 INFO:tasks.workunit.client.0.smithi076.stdout:6/204: dread d1/d2/f22 [0,4194304] 0 2024-03-23T15:44:51.384 INFO:tasks.workunit.client.0.smithi076.stdout:9/253: rmdir d1/d1f/d29/d2d 39 2024-03-23T15:44:51.386 INFO:tasks.workunit.client.0.smithi076.stdout:7/320: read d8/de/d13/d40/d5d/d3c/f4f [287811,33816] 0 2024-03-23T15:44:51.387 INFO:tasks.workunit.client.0.smithi076.stdout:5/236: truncate d1a/d20/f34 464083 0 2024-03-23T15:44:51.388 INFO:tasks.workunit.client.0.smithi076.stdout:7/321: creat d8/de/d13/d40/d5d/d3c/d4d/f66 x:0 0 0 2024-03-23T15:44:51.390 INFO:tasks.workunit.client.0.smithi076.stdout:4/302: dread d8/f1a [4194304,4194304] 0 2024-03-23T15:44:51.390 INFO:tasks.workunit.client.0.smithi076.stdout:7/322: mkdir d8/de/d13/d40/d67 0 2024-03-23T15:44:51.390 INFO:tasks.workunit.client.0.smithi076.stdout:5/237: creat d1a/d33/f54 x:0 0 0 2024-03-23T15:44:51.390 INFO:tasks.workunit.client.0.smithi076.stdout:5/238: stat d1a/d20/c27 0 2024-03-23T15:44:51.392 INFO:tasks.workunit.client.0.smithi076.stdout:4/303: truncate d8/d9/d1d/f60 3982785 0 2024-03-23T15:44:51.393 INFO:tasks.workunit.client.0.smithi076.stdout:6/205: dread d1/d4/f35 [0,4194304] 0 2024-03-23T15:44:51.393 INFO:tasks.workunit.client.0.smithi076.stdout:7/323: rename d8/de/d13/d40/d5d/l2e to d8/de/d13/d40/d5d/d3c/d46/l68 0 2024-03-23T15:44:51.394 INFO:tasks.workunit.client.0.smithi076.stdout:7/324: mknod d8/de/d13/d40/d5d/d3c/c69 0 2024-03-23T15:44:51.395 INFO:tasks.workunit.client.0.smithi076.stdout:4/304: readlink d8/d9/l46 0 2024-03-23T15:44:51.395 INFO:tasks.workunit.client.0.smithi076.stdout:4/305: fdatasync d8/f36 0 2024-03-23T15:44:51.395 INFO:tasks.workunit.client.0.smithi076.stdout:4/306: chown d8/d3a/d18/d1b/l3c 4558 1 2024-03-23T15:44:51.397 INFO:tasks.workunit.client.0.smithi076.stdout:6/206: symlink d1/l48 0 2024-03-23T15:44:51.400 INFO:tasks.workunit.client.0.smithi076.stdout:3/241: dwrite d5/d17/d1c/d32/d39/f3a [0,4194304] 0 2024-03-23T15:44:51.400 INFO:tasks.workunit.client.0.smithi076.stdout:3/242: read d5/f6 [5969475,69622] 0 2024-03-23T15:44:51.401 INFO:tasks.workunit.client.0.smithi076.stdout:4/307: creat d8/d3a/d18/d52/f61 x:0 0 0 2024-03-23T15:44:51.402 INFO:tasks.workunit.client.0.smithi076.stdout:8/178: dwrite d8/f14 [0,4194304] 0 2024-03-23T15:44:51.402 INFO:tasks.workunit.client.0.smithi076.stdout:8/179: chown d8/d26/d27/c2b 396 1 2024-03-23T15:44:51.402 INFO:tasks.workunit.client.0.smithi076.stdout:8/180: dread - d8/d2e/d30/f32 zero size 2024-03-23T15:44:51.408 INFO:tasks.workunit.client.0.smithi076.stdout:9/254: dwrite d1/f32 [0,4194304] 0 2024-03-23T15:44:51.409 INFO:tasks.workunit.client.0.smithi076.stdout:8/181: write d8/f1e [1161916,105573] 0 2024-03-23T15:44:51.413 INFO:tasks.workunit.client.0.smithi076.stdout:9/255: creat d1/d1f/d29/d2d/d37/d47/f52 x:0 0 0 2024-03-23T15:44:51.413 INFO:tasks.workunit.client.0.smithi076.stdout:9/256: write d1/d20/f45 [210145,37975] 0 2024-03-23T15:44:51.415 INFO:tasks.workunit.client.0.smithi076.stdout:4/308: write d8/d9/d1d/f60 [4144047,81803] 0 2024-03-23T15:44:51.416 INFO:tasks.workunit.client.0.smithi076.stdout:4/309: readlink d8/d3a/d18/d1b/l2c 0 2024-03-23T15:44:51.416 INFO:tasks.workunit.client.0.smithi076.stdout:4/310: truncate d8/d9/d1d/f4f 137414 0 2024-03-23T15:44:51.416 INFO:tasks.workunit.client.0.smithi076.stdout:9/257: write d1/f48 [735671,125221] 0 2024-03-23T15:44:51.417 INFO:tasks.workunit.client.0.smithi076.stdout:8/182: dread f2 [0,4194304] 0 2024-03-23T15:44:51.419 INFO:tasks.workunit.client.0.smithi076.stdout:4/311: rename d8/d3a/d3b/c40 to d8/d9/d32/c62 0 2024-03-23T15:44:51.419 INFO:tasks.workunit.client.0.smithi076.stdout:4/312: chown d8/c23 757841586 1 2024-03-23T15:44:51.419 INFO:tasks.workunit.client.0.smithi076.stdout:9/258: truncate d1/f1a 2152566 0 2024-03-23T15:44:51.420 INFO:tasks.workunit.client.0.smithi076.stdout:9/259: mkdir d1/d2f/d3a/d53 0 2024-03-23T15:44:51.420 INFO:tasks.workunit.client.0.smithi076.stdout:7/325: dwrite d8/de/d13/d40/d5d/f5b [0,4194304] 0 2024-03-23T15:44:51.425 INFO:tasks.workunit.client.0.smithi076.stdout:4/313: dread d8/d3a/d18/f21 [0,4194304] 0 2024-03-23T15:44:51.425 INFO:tasks.workunit.client.0.smithi076.stdout:4/314: chown d8/f1a 175948165 1 2024-03-23T15:44:51.425 INFO:tasks.workunit.client.0.smithi076.stdout:3/243: dwrite d5/d17/d1c/f1e [0,4194304] 0 2024-03-23T15:44:51.430 INFO:tasks.workunit.client.0.smithi076.stdout:3/244: rmdir d5/d35/d3c 39 2024-03-23T15:44:51.434 INFO:tasks.workunit.client.0.smithi076.stdout:7/326: write d8/de/d13/d40/d5d/f2a [957219,118573] 0 2024-03-23T15:44:51.434 INFO:tasks.workunit.client.0.smithi076.stdout:7/327: truncate d8/d11/f25 397615 0 2024-03-23T15:44:51.434 INFO:tasks.workunit.client.0.smithi076.stdout:7/328: dread - d8/f58 zero size 2024-03-23T15:44:51.434 INFO:tasks.workunit.client.0.smithi076.stdout:3/245: symlink d5/d17/d1c/d31/d47/l4d 0 2024-03-23T15:44:51.435 INFO:tasks.workunit.client.0.smithi076.stdout:4/315: unlink d8/d9/c27 0 2024-03-23T15:44:51.436 INFO:tasks.workunit.client.0.smithi076.stdout:3/246: symlink d5/d35/d3c/l4e 0 2024-03-23T15:44:51.436 INFO:tasks.workunit.client.0.smithi076.stdout:3/247: stat d5/f1f 0 2024-03-23T15:44:51.437 INFO:tasks.workunit.client.0.smithi076.stdout:3/248: mknod d5/d17/d1c/d32/c4f 0 2024-03-23T15:44:51.438 INFO:tasks.workunit.client.0.smithi076.stdout:3/249: symlink d5/d17/d1c/d32/l50 0 2024-03-23T15:44:51.444 INFO:tasks.workunit.client.0.smithi076.stdout:8/183: dwrite f2 [12582912,4194304] 0 2024-03-23T15:44:51.459 INFO:tasks.workunit.client.0.smithi076.stdout:4/316: dwrite d8/d3a/f48 [0,4194304] 0 2024-03-23T15:44:51.459 INFO:tasks.workunit.client.0.smithi076.stdout:4/317: truncate d8/d3a/d18/f4d 575315 0 2024-03-23T15:44:51.460 INFO:tasks.workunit.client.0.smithi076.stdout:7/329: dwrite d8/de/d13/d40/d5d/d3c/d46/f64 [0,4194304] 0 2024-03-23T15:44:51.460 INFO:tasks.workunit.client.0.smithi076.stdout:8/184: dread d8/f1d [0,4194304] 0 2024-03-23T15:44:51.461 INFO:tasks.workunit.client.0.smithi076.stdout:8/185: readlink d8/d26/l34 0 2024-03-23T15:44:51.462 INFO:tasks.workunit.client.0.smithi076.stdout:4/318: mknod d8/d3a/d18/c63 0 2024-03-23T15:44:51.463 INFO:tasks.workunit.client.0.smithi076.stdout:7/330: truncate d8/de/d13/d40/d5d/f34 253629 0 2024-03-23T15:44:51.465 INFO:tasks.workunit.client.0.smithi076.stdout:3/250: dwrite d5/d35/f36 [4194304,4194304] 0 2024-03-23T15:44:51.465 INFO:tasks.workunit.client.0.smithi076.stdout:7/331: link d8/de/d13/d18/f2b d8/de/d13/d40/d5d/f6a 0 2024-03-23T15:44:51.465 INFO:tasks.workunit.client.0.smithi076.stdout:7/332: read - d8/de/d13/d40/d5d/d3c/d4d/f66 zero size 2024-03-23T15:44:51.465 INFO:tasks.workunit.client.0.smithi076.stdout:7/333: read - d8/f62 zero size 2024-03-23T15:44:51.465 INFO:tasks.workunit.client.0.smithi076.stdout:7/334: write d8/de/d13/d40/d5d/f6a [1018467,11063] 0 2024-03-23T15:44:51.466 INFO:tasks.workunit.client.0.smithi076.stdout:7/335: dread d8/de/d13/d40/d5d/f3e [0,4194304] 0 2024-03-23T15:44:51.467 INFO:tasks.workunit.client.0.smithi076.stdout:7/336: symlink d8/de/d13/d40/l6b 0 2024-03-23T15:44:51.469 INFO:tasks.workunit.client.0.smithi076.stdout:8/186: rename l6 to d8/l39 0 2024-03-23T15:44:51.470 INFO:tasks.workunit.client.0.smithi076.stdout:8/187: stat d8/f2a 0 2024-03-23T15:44:51.470 INFO:tasks.workunit.client.0.smithi076.stdout:7/337: rmdir d8/de/d13/d3a 39 2024-03-23T15:44:51.470 INFO:tasks.workunit.client.0.smithi076.stdout:7/338: fsync d8/de/d13/f21 0 2024-03-23T15:44:51.470 INFO:tasks.workunit.client.0.smithi076.stdout:4/319: creat d8/d3a/d18/d52/f64 x:0 0 0 2024-03-23T15:44:51.470 INFO:tasks.workunit.client.0.smithi076.stdout:7/339: creat d8/de/d13/d40/d5d/d3c/d4d/f6c x:0 0 0 2024-03-23T15:44:51.471 INFO:tasks.workunit.client.0.smithi076.stdout:7/340: mknod d8/de/d13/d40/d5d/d3c/d4d/c6d 0 2024-03-23T15:44:51.473 INFO:tasks.workunit.client.0.smithi076.stdout:7/341: creat d8/de/d13/d40/d5d/f6e x:0 0 0 2024-03-23T15:44:51.473 INFO:tasks.workunit.client.0.smithi076.stdout:7/342: chown d8/de/d13/d18/c23 21 1 2024-03-23T15:44:51.474 INFO:tasks.workunit.client.0.smithi076.stdout:7/343: creat d8/de/f6f x:0 0 0 2024-03-23T15:44:51.474 INFO:tasks.workunit.client.0.smithi076.stdout:3/251: creat d5/d35/d4c/f51 x:0 0 0 2024-03-23T15:44:51.475 INFO:tasks.workunit.client.0.smithi076.stdout:7/344: rmdir d8/de/d13/d40/d5d 39 2024-03-23T15:44:51.476 INFO:tasks.workunit.client.0.smithi076.stdout:8/188: creat d8/d26/d33/f3a x:0 0 0 2024-03-23T15:44:51.477 INFO:tasks.workunit.client.0.smithi076.stdout:7/345: creat d8/de/d13/d40/d5d/f70 x:0 0 0 2024-03-23T15:44:51.477 INFO:tasks.workunit.client.0.smithi076.stdout:7/346: write d8/d11/f25 [1009451,54387] 0 2024-03-23T15:44:51.478 INFO:tasks.workunit.client.0.smithi076.stdout:7/347: mknod d8/de/d13/d40/d5d/d3c/d4d/c71 0 2024-03-23T15:44:51.480 INFO:tasks.workunit.client.0.smithi076.stdout:7/348: mknod d8/de/d13/c72 0 2024-03-23T15:44:51.480 INFO:tasks.workunit.client.0.smithi076.stdout:7/349: stat d8/de/d13 0 2024-03-23T15:44:51.482 INFO:tasks.workunit.client.0.smithi076.stdout:7/350: link d8/de/f26 d8/de/d13/d40/d5d/d3c/f73 0 2024-03-23T15:44:51.483 INFO:tasks.workunit.client.0.smithi076.stdout:7/351: stat f1 0 2024-03-23T15:44:51.483 INFO:tasks.workunit.client.0.smithi076.stdout:3/252: dread d5/d35/f36 [0,4194304] 0 2024-03-23T15:44:51.483 INFO:tasks.workunit.client.0.smithi076.stdout:7/352: readlink d8/de/d13/d40/l6b 0 2024-03-23T15:44:51.483 INFO:tasks.workunit.client.0.smithi076.stdout:8/189: dread d8/f10 [0,4194304] 0 2024-03-23T15:44:51.483 INFO:tasks.workunit.client.0.smithi076.stdout:4/320: dwrite d8/f30 [4194304,4194304] 0 2024-03-23T15:44:51.484 INFO:tasks.workunit.client.0.smithi076.stdout:8/190: mknod d8/d2e/c3b 0 2024-03-23T15:44:51.485 INFO:tasks.workunit.client.0.smithi076.stdout:4/321: creat d8/d9/d1d/f65 x:0 0 0 2024-03-23T15:44:51.487 INFO:tasks.workunit.client.0.smithi076.stdout:3/253: read d5/d35/f36 [2041799,99882] 0 2024-03-23T15:44:51.487 INFO:tasks.workunit.client.0.smithi076.stdout:8/191: mkdir d8/d2e/d30/d3c 0 2024-03-23T15:44:51.487 INFO:tasks.workunit.client.0.smithi076.stdout:8/192: chown d8/d26/d27 104344618 1 2024-03-23T15:44:51.487 INFO:tasks.workunit.client.0.smithi076.stdout:8/193: truncate d8/d2e/d30/f32 530022 0 2024-03-23T15:44:51.488 INFO:tasks.workunit.client.0.smithi076.stdout:7/353: unlink d8/de/d13/f5c 0 2024-03-23T15:44:51.489 INFO:tasks.workunit.client.0.smithi076.stdout:4/322: creat d8/f66 x:0 0 0 2024-03-23T15:44:51.489 INFO:tasks.workunit.client.0.smithi076.stdout:4/323: fsync d8/d3a/d3b/f25 0 2024-03-23T15:44:51.490 INFO:tasks.workunit.client.0.smithi076.stdout:7/354: write d8/de/f26 [266146,90093] 0 2024-03-23T15:44:51.491 INFO:tasks.workunit.client.0.smithi076.stdout:3/254: creat d5/d17/d1c/f52 x:0 0 0 2024-03-23T15:44:51.491 INFO:tasks.workunit.client.0.smithi076.stdout:7/355: creat d8/de/d13/f74 x:0 0 0 2024-03-23T15:44:51.491 INFO:tasks.workunit.client.0.smithi076.stdout:7/356: stat d8/de/d13/d40/d67 0 2024-03-23T15:44:51.492 INFO:tasks.workunit.client.0.smithi076.stdout:8/194: creat d8/d26/d33/f3d x:0 0 0 2024-03-23T15:44:51.493 INFO:tasks.workunit.client.0.smithi076.stdout:7/357: mknod d8/de/d13/d3a/c75 0 2024-03-23T15:44:51.494 INFO:tasks.workunit.client.0.smithi076.stdout:7/358: symlink d8/d11/l76 0 2024-03-23T15:44:51.494 INFO:tasks.workunit.client.0.smithi076.stdout:7/359: readlink d8/de/d13/d3a/l4b 0 2024-03-23T15:44:51.505 INFO:tasks.workunit.client.0.smithi076.stdout:4/324: truncate d8/f36 163 0 2024-03-23T15:44:51.505 INFO:tasks.workunit.client.0.smithi076.stdout:8/195: dread f2 [4194304,4194304] 0 2024-03-23T15:44:51.505 INFO:tasks.workunit.client.0.smithi076.stdout:8/196: dread - d8/d26/d27/f37 zero size 2024-03-23T15:44:51.505 INFO:tasks.workunit.client.0.smithi076.stdout:8/197: write d8/d26/d33/f3a [367494,109445] 0 2024-03-23T15:44:51.506 INFO:tasks.workunit.client.0.smithi076.stdout:4/325: creat d8/d3a/d18/d1b/f67 x:0 0 0 2024-03-23T15:44:51.507 INFO:tasks.workunit.client.0.smithi076.stdout:7/360: write d8/de/d13/f2f [3440668,123458] 0 2024-03-23T15:44:51.507 INFO:tasks.workunit.client.0.smithi076.stdout:7/361: write d8/de/d13/d18/f2b [1921853,99732] 0 2024-03-23T15:44:51.508 INFO:tasks.workunit.client.0.smithi076.stdout:8/198: mkdir d8/d2e/d3e 0 2024-03-23T15:44:51.509 INFO:tasks.workunit.client.0.smithi076.stdout:3/255: dwrite d5/d17/d1c/f52 [0,4194304] 0 2024-03-23T15:44:51.509 INFO:tasks.workunit.client.0.smithi076.stdout:3/256: write d5/d35/f36 [1941090,112196] 0 2024-03-23T15:44:51.510 INFO:tasks.workunit.client.0.smithi076.stdout:7/362: creat d8/de/d13/d40/d5d/d3c/d4d/f77 x:0 0 0 2024-03-23T15:44:51.510 INFO:tasks.workunit.client.0.smithi076.stdout:4/326: getdents d8/d9 0 2024-03-23T15:44:51.510 INFO:tasks.workunit.client.0.smithi076.stdout:4/327: write d8/d9/f12 [3921725,51068] 0 2024-03-23T15:44:51.511 INFO:tasks.workunit.client.0.smithi076.stdout:8/199: mknod d8/d2e/d30/d3c/c3f 0 2024-03-23T15:44:51.511 INFO:tasks.workunit.client.0.smithi076.stdout:8/200: getdents d8/d2e/d3e 0 2024-03-23T15:44:51.516 INFO:tasks.workunit.client.0.smithi076.stdout:7/363: unlink d8/de/d13/d18/c49 0 2024-03-23T15:44:51.517 INFO:tasks.workunit.client.0.smithi076.stdout:7/364: symlink d8/de/d13/d40/d67/l78 0 2024-03-23T15:44:51.518 INFO:tasks.workunit.client.0.smithi076.stdout:7/365: symlink d8/d11/l79 0 2024-03-23T15:44:51.520 INFO:tasks.workunit.client.0.smithi076.stdout:7/366: unlink d8/de/d13/c35 0 2024-03-23T15:44:51.527 INFO:tasks.workunit.client.0.smithi076.stdout:1/138: sync 2024-03-23T15:44:51.528 INFO:tasks.workunit.client.0.smithi076.stdout:1/139: mknod d0/d1/d6/d17/c2a 0 2024-03-23T15:44:51.528 INFO:tasks.workunit.client.0.smithi076.stdout:1/140: write d0/d1/d6/d17/f1d [1639761,11259] 0 2024-03-23T15:44:51.529 INFO:tasks.workunit.client.0.smithi076.stdout:1/141: write d0/f26 [76842,10602] 0 2024-03-23T15:44:51.529 INFO:tasks.workunit.client.0.smithi076.stdout:4/328: dwrite d8/d3a/d3b/f26 [0,4194304] 0 2024-03-23T15:44:51.531 INFO:tasks.workunit.client.0.smithi076.stdout:4/329: symlink d8/l68 0 2024-03-23T15:44:51.531 INFO:tasks.workunit.client.0.smithi076.stdout:4/330: chown d8/d3a/d18/d1b/f67 209242124 1 2024-03-23T15:44:51.532 INFO:tasks.workunit.client.0.smithi076.stdout:1/142: unlink d0/d2/lc 0 2024-03-23T15:44:51.533 INFO:tasks.workunit.client.0.smithi076.stdout:1/143: getdents d0/d2/d1f 0 2024-03-23T15:44:51.534 INFO:tasks.workunit.client.0.smithi076.stdout:3/257: dwrite d5/f6 [0,4194304] 0 2024-03-23T15:44:51.535 INFO:tasks.workunit.client.0.smithi076.stdout:1/144: link d0/d2/c29 d0/c2b 0 2024-03-23T15:44:51.535 INFO:tasks.workunit.client.0.smithi076.stdout:1/145: stat d0/d1/d6/d17/d20/c28 0 2024-03-23T15:44:51.536 INFO:tasks.workunit.client.0.smithi076.stdout:7/367: dwrite d8/d11/f33 [0,4194304] 0 2024-03-23T15:44:51.536 INFO:tasks.workunit.client.0.smithi076.stdout:8/201: dwrite d8/f1a [0,4194304] 0 2024-03-23T15:44:51.538 INFO:tasks.workunit.client.0.smithi076.stdout:7/368: mkdir d8/de/d13/d40/d67/d7a 0 2024-03-23T15:44:51.539 INFO:tasks.workunit.client.0.smithi076.stdout:7/369: mknod d8/de/d13/d40/d67/d7a/c7b 0 2024-03-23T15:44:51.541 INFO:tasks.workunit.client.0.smithi076.stdout:8/202: rename d8/d26/c2d to d8/d26/d33/c40 0 2024-03-23T15:44:51.542 INFO:tasks.workunit.client.0.smithi076.stdout:8/203: dread d8/d2e/d30/f32 [0,4194304] 0 2024-03-23T15:44:51.544 INFO:tasks.workunit.client.0.smithi076.stdout:8/204: mknod d8/d2e/c41 0 2024-03-23T15:44:51.544 INFO:tasks.workunit.client.0.smithi076.stdout:8/205: stat l1 0 2024-03-23T15:44:51.544 INFO:tasks.workunit.client.0.smithi076.stdout:7/370: rename d8/c17 to d8/de/d13/d40/c7c 0 2024-03-23T15:44:51.544 INFO:tasks.workunit.client.0.smithi076.stdout:7/371: fdatasync d8/de/d13/d40/d5d/f6a 0 2024-03-23T15:44:51.545 INFO:tasks.workunit.client.0.smithi076.stdout:3/258: read d5/d17/d1c/f52 [2613759,98185] 0 2024-03-23T15:44:51.546 INFO:tasks.workunit.client.0.smithi076.stdout:7/372: dread d8/d11/f2d [0,4194304] 0 2024-03-23T15:44:51.546 INFO:tasks.workunit.client.0.smithi076.stdout:8/206: rename d8/l21 to d8/d26/d33/l42 0 2024-03-23T15:44:51.546 INFO:tasks.workunit.client.0.smithi076.stdout:7/373: getdents d8/de/d13/d18 0 2024-03-23T15:44:51.547 INFO:tasks.workunit.client.0.smithi076.stdout:9/260: sync 2024-03-23T15:44:51.547 INFO:tasks.workunit.client.0.smithi076.stdout:2/281: sync 2024-03-23T15:44:51.547 INFO:tasks.workunit.client.0.smithi076.stdout:6/207: sync 2024-03-23T15:44:51.547 INFO:tasks.workunit.client.0.smithi076.stdout:5/239: sync 2024-03-23T15:44:51.547 INFO:tasks.workunit.client.0.smithi076.stdout:0/317: sync 2024-03-23T15:44:51.547 INFO:tasks.workunit.client.0.smithi076.stdout:2/282: write d1/d4/f9 [1720541,104127] 0 2024-03-23T15:44:51.548 INFO:tasks.workunit.client.0.smithi076.stdout:7/374: symlink d8/de/d13/d3a/l7d 0 2024-03-23T15:44:51.548 INFO:tasks.workunit.client.0.smithi076.stdout:3/259: link d5/d17/d1c/f2b d5/d17/d1c/d32/d39/f53 0 2024-03-23T15:44:51.549 INFO:tasks.workunit.client.0.smithi076.stdout:2/283: dread d1/d4/db/f3b [0,4194304] 0 2024-03-23T15:44:51.549 INFO:tasks.workunit.client.0.smithi076.stdout:2/284: truncate d1/d4/d12/f26 250739 0 2024-03-23T15:44:51.549 INFO:tasks.workunit.client.0.smithi076.stdout:8/207: mknod d8/d2e/c43 0 2024-03-23T15:44:51.549 INFO:tasks.workunit.client.0.smithi076.stdout:5/240: mknod d1a/d39/c55 0 2024-03-23T15:44:51.549 INFO:tasks.workunit.client.0.smithi076.stdout:5/241: chown d1a/d39/f44 325304137 1 2024-03-23T15:44:51.550 INFO:tasks.workunit.client.0.smithi076.stdout:6/208: mknod d1/dd/df/c49 0 2024-03-23T15:44:51.550 INFO:tasks.workunit.client.0.smithi076.stdout:6/209: readlink d1/l48 0 2024-03-23T15:44:51.551 INFO:tasks.workunit.client.0.smithi076.stdout:0/318: creat d2/d8/d52/f5f x:0 0 0 2024-03-23T15:44:51.551 INFO:tasks.workunit.client.0.smithi076.stdout:3/260: mknod d5/d17/c54 0 2024-03-23T15:44:51.552 INFO:tasks.workunit.client.0.smithi076.stdout:3/261: write d5/f2d [836080,120688] 0 2024-03-23T15:44:51.552 INFO:tasks.workunit.client.0.smithi076.stdout:6/210: link d1/f21 d1/d4/d10/f4a 0 2024-03-23T15:44:51.553 INFO:tasks.workunit.client.0.smithi076.stdout:3/262: creat d5/d17/d1c/d32/d4a/f55 x:0 0 0 2024-03-23T15:44:51.553 INFO:tasks.workunit.client.0.smithi076.stdout:3/263: write d5/d17/f29 [165198,2627] 0 2024-03-23T15:44:51.554 INFO:tasks.workunit.client.0.smithi076.stdout:3/264: write d5/d17/d1c/f1d [300868,108694] 0 2024-03-23T15:44:51.554 INFO:tasks.workunit.client.0.smithi076.stdout:5/242: write f13 [80541,69459] 0 2024-03-23T15:44:51.556 INFO:tasks.workunit.client.0.smithi076.stdout:2/285: creat d1/d4/db/d15/d29/f61 x:0 0 0 2024-03-23T15:44:51.557 INFO:tasks.workunit.client.0.smithi076.stdout:1/146: dwrite d0/f26 [4194304,4194304] 0 2024-03-23T15:44:51.557 INFO:tasks.workunit.client.0.smithi076.stdout:3/265: unlink d5/f2d 0 2024-03-23T15:44:51.557 INFO:tasks.workunit.client.0.smithi076.stdout:3/266: chown d5/d17/d37/f49 676613 1 2024-03-23T15:44:51.557 INFO:tasks.workunit.client.0.smithi076.stdout:6/211: readlink d1/d2/l2f 0 2024-03-23T15:44:51.558 INFO:tasks.workunit.client.0.smithi076.stdout:2/286: creat d1/d4/d52/f62 x:0 0 0 2024-03-23T15:44:51.559 INFO:tasks.workunit.client.0.smithi076.stdout:1/147: creat d0/d1/d6/d17/f2c x:0 0 0 2024-03-23T15:44:51.561 INFO:tasks.workunit.client.0.smithi076.stdout:3/267: creat d5/f56 x:0 0 0 2024-03-23T15:44:51.561 INFO:tasks.workunit.client.0.smithi076.stdout:9/261: dread d1/f18 [0,4194304] 0 2024-03-23T15:44:51.562 INFO:tasks.workunit.client.0.smithi076.stdout:6/212: symlink d1/dd/df/l4b 0 2024-03-23T15:44:51.562 INFO:tasks.workunit.client.0.smithi076.stdout:0/319: write d2/f9 [3535728,96968] 0 2024-03-23T15:44:51.562 INFO:tasks.workunit.client.0.smithi076.stdout:0/320: write d2/d8/f4f [928994,65312] 0 2024-03-23T15:44:51.562 INFO:tasks.workunit.client.0.smithi076.stdout:0/321: read - d2/d8/d2d/f3f zero size 2024-03-23T15:44:51.563 INFO:tasks.workunit.client.0.smithi076.stdout:1/148: getdents d0/d1/d6/d17/d20 0 2024-03-23T15:44:51.564 INFO:tasks.workunit.client.0.smithi076.stdout:2/287: dread d1/d4/d12/f18 [0,4194304] 0 2024-03-23T15:44:51.565 INFO:tasks.workunit.client.0.smithi076.stdout:3/268: symlink d5/d17/d37/d48/l57 0 2024-03-23T15:44:51.565 INFO:tasks.workunit.client.0.smithi076.stdout:3/269: readlink d5/d17/d1c/d32/d39/l3d 0 2024-03-23T15:44:51.566 INFO:tasks.workunit.client.0.smithi076.stdout:9/262: truncate d1/d1f/d29/f40 2058882 0 2024-03-23T15:44:51.566 INFO:tasks.workunit.client.0.smithi076.stdout:9/263: fsync d1/f35 0 2024-03-23T15:44:51.567 INFO:tasks.workunit.client.0.smithi076.stdout:6/213: symlink d1/dd/df/l4c 0 2024-03-23T15:44:51.567 INFO:tasks.workunit.client.0.smithi076.stdout:1/149: dread d0/d1/d6/f9 [4194304,4194304] 0 2024-03-23T15:44:51.567 INFO:tasks.workunit.client.0.smithi076.stdout:1/150: stat d0/d1/d6/d17/d20/c28 0 2024-03-23T15:44:51.568 INFO:tasks.workunit.client.0.smithi076.stdout:2/288: rename d1/d4/d12/d1b/f40 to d1/d4/d12/d1b/f63 0 2024-03-23T15:44:51.569 INFO:tasks.workunit.client.0.smithi076.stdout:7/375: dwrite d8/f3f [0,4194304] 0 2024-03-23T15:44:51.569 INFO:tasks.workunit.client.0.smithi076.stdout:9/264: creat d1/d1f/d29/f54 x:0 0 0 2024-03-23T15:44:51.570 INFO:tasks.workunit.client.0.smithi076.stdout:9/265: read - d1/d1f/d29/d2d/d37/d47/f52 zero size 2024-03-23T15:44:51.571 INFO:tasks.workunit.client.0.smithi076.stdout:6/214: truncate d1/d4/d10/f4a 1706055 0 2024-03-23T15:44:51.573 INFO:tasks.workunit.client.0.smithi076.stdout:1/151: rename d0/d1/l16 to d0/d1/d6/d17/d20/l2d 0 2024-03-23T15:44:51.575 INFO:tasks.workunit.client.0.smithi076.stdout:7/376: rename d8/d11/l79 to d8/d4e/l7e 0 2024-03-23T15:44:51.577 INFO:tasks.workunit.client.0.smithi076.stdout:8/208: dwrite f7 [0,4194304] 0 2024-03-23T15:44:51.577 INFO:tasks.workunit.client.0.smithi076.stdout:7/377: creat d8/d11/d5a/f7f x:0 0 0 2024-03-23T15:44:51.577 INFO:tasks.workunit.client.0.smithi076.stdout:7/378: write d8/de/d13/d40/d5d/f20 [1233135,19735] 0 2024-03-23T15:44:51.577 INFO:tasks.workunit.client.0.smithi076.stdout:1/152: link d0/d1/d6/f9 d0/f2e 0 2024-03-23T15:44:51.578 INFO:tasks.workunit.client.0.smithi076.stdout:1/153: chown d0/l11 0 1 2024-03-23T15:44:51.579 INFO:tasks.workunit.client.0.smithi076.stdout:1/154: rmdir d0/d2 39 2024-03-23T15:44:51.579 INFO:tasks.workunit.client.0.smithi076.stdout:1/155: write d0/d1/d6/d17/f2c [692279,127982] 0 2024-03-23T15:44:51.579 INFO:tasks.workunit.client.0.smithi076.stdout:1/156: fsync d0/d1/d6/f23 0 2024-03-23T15:44:51.579 INFO:tasks.workunit.client.0.smithi076.stdout:1/157: readlink d0/l11 0 2024-03-23T15:44:51.581 INFO:tasks.workunit.client.0.smithi076.stdout:1/158: unlink d0/d1/d6/d17/d20/c28 0 2024-03-23T15:44:51.584 INFO:tasks.workunit.client.0.smithi076.stdout:2/289: rename d1/d4/d12 to d1/d4/d64 0 2024-03-23T15:44:51.586 INFO:tasks.workunit.client.0.smithi076.stdout:2/290: link d1/d4/c5d d1/d4/db/c65 0 2024-03-23T15:44:51.588 INFO:tasks.workunit.client.0.smithi076.stdout:2/291: rmdir d1/d4/d64/d1d 39 2024-03-23T15:44:51.588 INFO:tasks.workunit.client.0.smithi076.stdout:5/243: dwrite f4 [0,4194304] 0 2024-03-23T15:44:51.588 INFO:tasks.workunit.client.0.smithi076.stdout:5/244: stat d1a/l2f 0 2024-03-23T15:44:51.588 INFO:tasks.workunit.client.0.smithi076.stdout:6/215: dread d1/dd/d2a/d23/f2d [0,4194304] 0 2024-03-23T15:44:51.589 INFO:tasks.workunit.client.0.smithi076.stdout:2/292: link d1/d4/c35 d1/d4/db/c66 0 2024-03-23T15:44:51.589 INFO:tasks.workunit.client.0.smithi076.stdout:8/209: dread d8/f1a [0,4194304] 0 2024-03-23T15:44:51.591 INFO:tasks.workunit.client.0.smithi076.stdout:7/379: dwrite d8/de/d13/d40/d5d/f6a [0,4194304] 0 2024-03-23T15:44:51.591 INFO:tasks.workunit.client.0.smithi076.stdout:5/245: link d1a/c28 d1a/d39/c56 0 2024-03-23T15:44:51.593 INFO:tasks.workunit.client.0.smithi076.stdout:1/159: dread d0/d1/d6/d17/f2c [0,4194304] 0 2024-03-23T15:44:51.593 INFO:tasks.workunit.client.0.smithi076.stdout:0/322: dwrite d2/d8/f53 [0,4194304] 0 2024-03-23T15:44:51.593 INFO:tasks.workunit.client.0.smithi076.stdout:2/293: dread d1/d4/d64/f18 [0,4194304] 0 2024-03-23T15:44:51.593 INFO:tasks.workunit.client.0.smithi076.stdout:2/294: chown d1/d4/d64/d1d/f3c 1493585 1 2024-03-23T15:44:51.594 INFO:tasks.workunit.client.0.smithi076.stdout:5/246: chown f6 20564059 1 2024-03-23T15:44:51.594 INFO:tasks.workunit.client.0.smithi076.stdout:5/247: chown d1a/d33/d3a/f41 334 1 2024-03-23T15:44:51.595 INFO:tasks.workunit.client.0.smithi076.stdout:7/380: rename d8/de/d13/d40/d5d/d3c/d46/c52 to d8/de/d13/d40/c80 0 2024-03-23T15:44:51.598 INFO:tasks.workunit.client.0.smithi076.stdout:2/295: mknod d1/d4/c67 0 2024-03-23T15:44:51.600 INFO:tasks.workunit.client.0.smithi076.stdout:0/323: unlink d2/d1b/f1e 0 2024-03-23T15:44:51.600 INFO:tasks.workunit.client.0.smithi076.stdout:7/381: rename d8/de/d13/d40/d5d/f2a to d8/de/f81 0 2024-03-23T15:44:51.601 INFO:tasks.workunit.client.0.smithi076.stdout:8/210: dread d8/f2a [0,4194304] 0 2024-03-23T15:44:51.601 INFO:tasks.workunit.client.0.smithi076.stdout:5/248: rmdir d1a/d33/d36 39 2024-03-23T15:44:51.602 INFO:tasks.workunit.client.0.smithi076.stdout:0/324: rmdir d2 39 2024-03-23T15:44:51.603 INFO:tasks.workunit.client.0.smithi076.stdout:2/296: link d1/d4/db/d15/f55 d1/d4/db/f68 0 2024-03-23T15:44:51.604 INFO:tasks.workunit.client.0.smithi076.stdout:8/211: creat d8/d2e/d3e/f44 x:0 0 0 2024-03-23T15:44:51.604 INFO:tasks.workunit.client.0.smithi076.stdout:8/212: fsync d8/f1a 0 2024-03-23T15:44:51.604 INFO:tasks.workunit.client.0.smithi076.stdout:8/213: stat d8/d2e/d3e 0 2024-03-23T15:44:51.604 INFO:tasks.workunit.client.0.smithi076.stdout:0/325: write d2/de/d10/d12/f23 [454624,114239] 0 2024-03-23T15:44:51.604 INFO:tasks.workunit.client.0.smithi076.stdout:0/326: dread - d2/de/d10/d43/f57 zero size 2024-03-23T15:44:51.604 INFO:tasks.workunit.client.0.smithi076.stdout:9/266: dwrite d1/d20/f43 [0,4194304] 0 2024-03-23T15:44:51.605 INFO:tasks.workunit.client.0.smithi076.stdout:8/214: mknod d8/d2e/d30/c45 0 2024-03-23T15:44:51.605 INFO:tasks.workunit.client.0.smithi076.stdout:8/215: readlink d8/l24 0 2024-03-23T15:44:51.607 INFO:tasks.workunit.client.0.smithi076.stdout:0/327: link d2/d8/d2d/f3f d2/de/d10/d43/d56/f60 0 2024-03-23T15:44:51.607 INFO:tasks.workunit.client.0.smithi076.stdout:0/328: fsync d2/f9 0 2024-03-23T15:44:51.607 INFO:tasks.workunit.client.0.smithi076.stdout:0/329: dread - d2/d8/d2d/f3f zero size 2024-03-23T15:44:51.607 INFO:tasks.workunit.client.0.smithi076.stdout:0/330: truncate d2/d8/f4f 1331258 0 2024-03-23T15:44:51.609 INFO:tasks.workunit.client.0.smithi076.stdout:2/297: truncate d1/d4/d64/f18 914310 0 2024-03-23T15:44:51.609 INFO:tasks.workunit.client.0.smithi076.stdout:2/298: fdatasync d1/d4/d64/d1b/d32/f38 0 2024-03-23T15:44:51.609 INFO:tasks.workunit.client.0.smithi076.stdout:8/216: creat d8/d26/f46 x:0 0 0 2024-03-23T15:44:51.610 INFO:tasks.workunit.client.0.smithi076.stdout:5/249: write f4 [1077969,21065] 0 2024-03-23T15:44:51.613 INFO:tasks.workunit.client.0.smithi076.stdout:0/331: link d2/d8/fa d2/de/d10/f61 0 2024-03-23T15:44:51.615 INFO:tasks.workunit.client.0.smithi076.stdout:8/217: dread f2 [8388608,4194304] 0 2024-03-23T15:44:51.617 INFO:tasks.workunit.client.0.smithi076.stdout:5/250: mknod d1a/d33/d3a/c57 0 2024-03-23T15:44:51.617 INFO:tasks.workunit.client.0.smithi076.stdout:2/299: unlink d1/d4/db/d15/d29/c33 0 2024-03-23T15:44:51.619 INFO:tasks.workunit.client.0.smithi076.stdout:0/332: link d2/de/d10/d3a/d47/f59 d2/d8/f62 0 2024-03-23T15:44:51.619 INFO:tasks.workunit.client.0.smithi076.stdout:8/218: dread d8/f1d [0,4194304] 0 2024-03-23T15:44:51.621 INFO:tasks.workunit.client.0.smithi076.stdout:9/267: dwrite d1/d1f/d46/d4a/f50 [0,4194304] 0 2024-03-23T15:44:51.621 INFO:tasks.workunit.client.0.smithi076.stdout:1/160: fsync d0/d1/d6/f9 0 2024-03-23T15:44:51.626 INFO:tasks.workunit.client.0.smithi076.stdout:9/268: symlink d1/d2f/d3f/l55 0 2024-03-23T15:44:51.627 INFO:tasks.workunit.client.0.smithi076.stdout:9/269: write d1/f48 [1245094,127815] 0 2024-03-23T15:44:51.627 INFO:tasks.workunit.client.0.smithi076.stdout:5/251: read f18 [3784348,3540] 0 2024-03-23T15:44:51.629 INFO:tasks.workunit.client.0.smithi076.stdout:9/270: symlink d1/d20/l56 0 2024-03-23T15:44:51.629 INFO:tasks.workunit.client.0.smithi076.stdout:9/271: read - d1/d2f/f42 zero size 2024-03-23T15:44:51.630 INFO:tasks.workunit.client.0.smithi076.stdout:8/219: write d8/f1a [1824186,12854] 0 2024-03-23T15:44:51.630 INFO:tasks.workunit.client.0.smithi076.stdout:8/220: chown d8/d26/d33/f3d 0 1 2024-03-23T15:44:51.631 INFO:tasks.workunit.client.0.smithi076.stdout:5/252: creat d1a/d33/d3a/f58 x:0 0 0 2024-03-23T15:44:51.632 INFO:tasks.workunit.client.0.smithi076.stdout:9/272: mknod d1/d1f/d29/d2d/d37/c57 0 2024-03-23T15:44:51.635 INFO:tasks.workunit.client.0.smithi076.stdout:9/273: symlink d1/d2f/d3a/d4d/l58 0 2024-03-23T15:44:51.638 INFO:tasks.workunit.client.0.smithi076.stdout:9/274: creat d1/d1f/d29/d2d/d37/f59 x:0 0 0 2024-03-23T15:44:51.638 INFO:tasks.workunit.client.0.smithi076.stdout:5/253: dread d1a/f29 [0,4194304] 0 2024-03-23T15:44:51.640 INFO:tasks.workunit.client.0.smithi076.stdout:2/300: dwrite d1/d4/db/f68 [0,4194304] 0 2024-03-23T15:44:51.640 INFO:tasks.workunit.client.0.smithi076.stdout:2/301: write d1/d4/d64/d1d/f3c [751380,25069] 0 2024-03-23T15:44:51.642 INFO:tasks.workunit.client.0.smithi076.stdout:0/333: dwrite d2/de/f1a [0,4194304] 0 2024-03-23T15:44:51.645 INFO:tasks.workunit.client.0.smithi076.stdout:9/275: write d1/f18 [1595196,69096] 0 2024-03-23T15:44:51.645 INFO:tasks.workunit.client.0.smithi076.stdout:1/161: dwrite d0/d1/d6/d17/f2c [0,4194304] 0 2024-03-23T15:44:51.645 INFO:tasks.workunit.client.0.smithi076.stdout:9/276: fdatasync d1/d1f/f21 0 2024-03-23T15:44:51.645 INFO:tasks.workunit.client.0.smithi076.stdout:9/277: chown d1/d1f/d29/d2d 80 1 2024-03-23T15:44:51.648 INFO:tasks.workunit.client.0.smithi076.stdout:0/334: creat d2/de/d10/d43/f63 x:0 0 0 2024-03-23T15:44:51.649 INFO:tasks.workunit.client.0.smithi076.stdout:2/302: dread d1/d4/f10 [0,4194304] 0 2024-03-23T15:44:51.650 INFO:tasks.workunit.client.0.smithi076.stdout:2/303: rmdir d1/d4/d64/d1b 39 2024-03-23T15:44:51.651 INFO:tasks.workunit.client.0.smithi076.stdout:1/162: mknod d0/d1/d6/c2f 0 2024-03-23T15:44:51.651 INFO:tasks.workunit.client.0.smithi076.stdout:2/304: symlink d1/d4/d64/l69 0 2024-03-23T15:44:51.651 INFO:tasks.workunit.client.0.smithi076.stdout:2/305: write d1/d4/d64/d1b/f63 [12835,54040] 0 2024-03-23T15:44:51.653 INFO:tasks.workunit.client.0.smithi076.stdout:2/306: rmdir d1/d4/d64/d43 39 2024-03-23T15:44:51.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/278: mkdir d1/d1f/d46/d4a/d5a 0 2024-03-23T15:44:51.654 INFO:tasks.workunit.client.0.smithi076.stdout:0/335: creat d2/d8/d52/f64 x:0 0 0 2024-03-23T15:44:51.656 INFO:tasks.workunit.client.0.smithi076.stdout:8/221: dwrite d8/fa [0,4194304] 0 2024-03-23T15:44:51.656 INFO:tasks.workunit.client.0.smithi076.stdout:0/336: creat d2/d8/f65 x:0 0 0 2024-03-23T15:44:51.658 INFO:tasks.workunit.client.0.smithi076.stdout:2/307: truncate d1/f14 443368 0 2024-03-23T15:44:51.658 INFO:tasks.workunit.client.0.smithi076.stdout:2/308: write d1/d4/d64/d1d/f3c [931405,124780] 0 2024-03-23T15:44:51.660 INFO:tasks.workunit.client.0.smithi076.stdout:2/309: dread d1/d4/db/d15/d29/f4b [0,4194304] 0 2024-03-23T15:44:51.663 INFO:tasks.workunit.client.0.smithi076.stdout:0/337: symlink d2/de/d10/l66 0 2024-03-23T15:44:51.663 INFO:tasks.workunit.client.0.smithi076.stdout:0/338: dread - d2/d8/d52/f5f zero size 2024-03-23T15:44:51.663 INFO:tasks.workunit.client.0.smithi076.stdout:0/339: fsync d2/d8/f4f 0 2024-03-23T15:44:51.666 INFO:tasks.workunit.client.0.smithi076.stdout:0/340: getdents d2 0 2024-03-23T15:44:51.666 INFO:tasks.workunit.client.0.smithi076.stdout:5/254: dwrite d1a/d33/d3a/f43 [0,4194304] 0 2024-03-23T15:44:51.672 INFO:tasks.workunit.client.0.smithi076.stdout:2/310: dread d1/d4/f1e [0,4194304] 0 2024-03-23T15:44:51.674 INFO:tasks.workunit.client.0.smithi076.stdout:2/311: mkdir d1/d2e/d6a 0 2024-03-23T15:44:51.674 INFO:tasks.workunit.client.0.smithi076.stdout:2/312: chown d1/d4/d64/f18 2700687 1 2024-03-23T15:44:51.674 INFO:tasks.workunit.client.0.smithi076.stdout:2/313: write d1/d4/f54 [355476,83948] 0 2024-03-23T15:44:51.675 INFO:tasks.workunit.client.0.smithi076.stdout:2/314: symlink d1/d2e/l6b 0 2024-03-23T15:44:51.677 INFO:tasks.workunit.client.0.smithi076.stdout:1/163: dwrite d0/d1/d6/d17/d20/f25 [0,4194304] 0 2024-03-23T15:44:51.677 INFO:tasks.workunit.client.0.smithi076.stdout:2/315: rename d1/d4/db/f3b to d1/d4/d64/d1b/f6c 0 2024-03-23T15:44:51.678 INFO:tasks.workunit.client.0.smithi076.stdout:9/279: dwrite d1/f15 [0,4194304] 0 2024-03-23T15:44:51.679 INFO:tasks.workunit.client.0.smithi076.stdout:2/316: mknod d1/d4/db/d3e/c6d 0 2024-03-23T15:44:51.680 INFO:tasks.workunit.client.0.smithi076.stdout:1/164: creat d0/d1/d6/d17/f30 x:0 0 0 2024-03-23T15:44:51.681 INFO:tasks.workunit.client.0.smithi076.stdout:5/255: dwrite d1a/f29 [4194304,4194304] 0 2024-03-23T15:44:51.682 INFO:tasks.workunit.client.0.smithi076.stdout:2/317: truncate d1/d4/db/f5b 1946883 0 2024-03-23T15:44:51.684 INFO:tasks.workunit.client.0.smithi076.stdout:2/318: mkdir d1/d4/d52/d6e 0 2024-03-23T15:44:51.685 INFO:tasks.workunit.client.0.smithi076.stdout:9/280: truncate d1/f1a 275182 0 2024-03-23T15:44:51.686 INFO:tasks.workunit.client.0.smithi076.stdout:2/319: mknod d1/d4/d64/c6f 0 2024-03-23T15:44:51.688 INFO:tasks.workunit.client.0.smithi076.stdout:9/281: creat d1/f5b x:0 0 0 2024-03-23T15:44:51.688 INFO:tasks.workunit.client.0.smithi076.stdout:2/320: creat d1/d4/d52/f70 x:0 0 0 2024-03-23T15:44:51.688 INFO:tasks.workunit.client.0.smithi076.stdout:2/321: write d1/d4/d64/d1b/f63 [458664,102336] 0 2024-03-23T15:44:51.691 INFO:tasks.workunit.client.0.smithi076.stdout:2/322: creat d1/d4/d64/d1b/f71 x:0 0 0 2024-03-23T15:44:51.691 INFO:tasks.workunit.client.0.smithi076.stdout:2/323: truncate d1/f2f 4515088 0 2024-03-23T15:44:51.694 INFO:tasks.workunit.client.0.smithi076.stdout:0/341: dwrite d2/de/f22 [0,4194304] 0 2024-03-23T15:44:51.696 INFO:tasks.workunit.client.0.smithi076.stdout:0/342: creat d2/de/d10/f67 x:0 0 0 2024-03-23T15:44:51.698 INFO:tasks.workunit.client.0.smithi076.stdout:0/343: read d2/de/f13 [1182193,119449] 0 2024-03-23T15:44:51.699 INFO:tasks.workunit.client.0.smithi076.stdout:0/344: creat d2/de/d10/f68 x:0 0 0 2024-03-23T15:44:51.701 INFO:tasks.workunit.client.0.smithi076.stdout:0/345: mkdir d2/de/d10/d3a/d47/d69 0 2024-03-23T15:44:51.701 INFO:tasks.workunit.client.0.smithi076.stdout:1/165: dwrite d0/f26 [0,4194304] 0 2024-03-23T15:44:51.703 INFO:tasks.workunit.client.0.smithi076.stdout:4/331: sync 2024-03-23T15:44:51.703 INFO:tasks.workunit.client.0.smithi076.stdout:4/332: fdatasync d8/f56 0 2024-03-23T15:44:51.703 INFO:tasks.workunit.client.0.smithi076.stdout:4/333: write d8/d9/d32/f5b [563369,75116] 0 2024-03-23T15:44:51.704 INFO:tasks.workunit.client.0.smithi076.stdout:4/334: getdents d8/d3a/d18/d1b 0 2024-03-23T15:44:51.704 INFO:tasks.workunit.client.0.smithi076.stdout:4/335: dread - d8/d3a/d18/d52/f61 zero size 2024-03-23T15:44:51.706 INFO:tasks.workunit.client.0.smithi076.stdout:4/336: mkdir d8/d3a/d69 0 2024-03-23T15:44:51.706 INFO:tasks.workunit.client.0.smithi076.stdout:4/337: chown d8/d3a/d18/d1b/l3c 262251519 1 2024-03-23T15:44:51.706 INFO:tasks.workunit.client.0.smithi076.stdout:4/338: truncate d8/f36 521795 0 2024-03-23T15:44:51.707 INFO:tasks.workunit.client.0.smithi076.stdout:5/256: dwrite d1a/f31 [0,4194304] 0 2024-03-23T15:44:51.709 INFO:tasks.workunit.client.0.smithi076.stdout:9/282: dwrite d1/d1f/d29/f40 [0,4194304] 0 2024-03-23T15:44:51.710 INFO:tasks.workunit.client.0.smithi076.stdout:5/257: link d1a/d33/d3a/c57 d1a/c59 0 2024-03-23T15:44:51.711 INFO:tasks.workunit.client.0.smithi076.stdout:1/166: write d0/d1/d6/d17/d20/f25 [2258232,83703] 0 2024-03-23T15:44:51.712 INFO:tasks.workunit.client.0.smithi076.stdout:9/283: mknod d1/d2f/d3a/d53/c5c 0 2024-03-23T15:44:51.715 INFO:tasks.workunit.client.0.smithi076.stdout:5/258: fsync f5 0 2024-03-23T15:44:51.715 INFO:tasks.workunit.client.0.smithi076.stdout:5/259: chown d1a/f3f 32 1 2024-03-23T15:44:51.717 INFO:tasks.workunit.client.0.smithi076.stdout:9/284: getdents d1/d2f/d3f 0 2024-03-23T15:44:51.722 INFO:tasks.workunit.client.0.smithi076.stdout:2/324: dwrite d1/d4/d64/d1b/f44 [0,4194304] 0 2024-03-23T15:44:51.722 INFO:tasks.workunit.client.0.smithi076.stdout:0/346: dwrite d2/de/d10/d12/f23 [0,4194304] 0 2024-03-23T15:44:51.722 INFO:tasks.workunit.client.0.smithi076.stdout:9/285: rmdir d1/d1f 39 2024-03-23T15:44:51.724 INFO:tasks.workunit.client.0.smithi076.stdout:2/325: creat d1/d4/f72 x:0 0 0 2024-03-23T15:44:51.726 INFO:tasks.workunit.client.0.smithi076.stdout:2/326: rmdir d1/d4/d64/d43 39 2024-03-23T15:44:51.727 INFO:tasks.workunit.client.0.smithi076.stdout:2/327: creat d1/d4/d64/d1d/f73 x:0 0 0 2024-03-23T15:44:51.728 INFO:tasks.workunit.client.0.smithi076.stdout:2/328: stat d1/d4/db/d3e 0 2024-03-23T15:44:51.728 INFO:tasks.workunit.client.0.smithi076.stdout:2/329: chown d1/d4/d64/d1b 245208008 1 2024-03-23T15:44:51.728 INFO:tasks.workunit.client.0.smithi076.stdout:0/347: link d2/de/d10/d43/f63 d2/de/d10/d12/f6a 0 2024-03-23T15:44:51.728 INFO:tasks.workunit.client.0.smithi076.stdout:4/339: dwrite d8/f1a [0,4194304] 0 2024-03-23T15:44:51.728 INFO:tasks.workunit.client.0.smithi076.stdout:2/330: getdents d1/d4/db 0 2024-03-23T15:44:51.729 INFO:tasks.workunit.client.0.smithi076.stdout:0/348: mknod d2/de/d10/d3a/d47/d69/c6b 0 2024-03-23T15:44:51.730 INFO:tasks.workunit.client.0.smithi076.stdout:5/260: dwrite d1a/f3c [0,4194304] 0 2024-03-23T15:44:51.732 INFO:tasks.workunit.client.0.smithi076.stdout:2/331: link d1/d4/l3a d1/d4/db/d15/l74 0 2024-03-23T15:44:51.733 INFO:tasks.workunit.client.0.smithi076.stdout:0/349: mknod d2/de/d10/d43/d56/c6c 0 2024-03-23T15:44:51.733 INFO:tasks.workunit.client.0.smithi076.stdout:0/350: chown d2/d8/fa 23240 1 2024-03-23T15:44:51.735 INFO:tasks.workunit.client.0.smithi076.stdout:2/332: creat d1/d2e/d6a/f75 x:0 0 0 2024-03-23T15:44:51.735 INFO:tasks.workunit.client.0.smithi076.stdout:2/333: truncate d1/d4/d52/f62 166623 0 2024-03-23T15:44:51.739 INFO:tasks.workunit.client.0.smithi076.stdout:2/334: truncate d1/d4/d64/d1b/f44 3196808 0 2024-03-23T15:44:51.739 INFO:tasks.workunit.client.0.smithi076.stdout:2/335: truncate d1/d4/d64/f4e 996059 0 2024-03-23T15:44:51.740 INFO:tasks.workunit.client.0.smithi076.stdout:2/336: rmdir d1/d4/d52 39 2024-03-23T15:44:51.740 INFO:tasks.workunit.client.0.smithi076.stdout:2/337: read d1/d4/d64/d1b/d32/f38 [291533,66977] 0 2024-03-23T15:44:51.742 INFO:tasks.workunit.client.0.smithi076.stdout:2/338: rename d1/d4/db/d15/c46 to d1/d4/c76 0 2024-03-23T15:44:51.742 INFO:tasks.workunit.client.0.smithi076.stdout:2/339: truncate d1/d4/f72 181559 0 2024-03-23T15:44:51.742 INFO:tasks.workunit.client.0.smithi076.stdout:2/340: truncate d1/d4/db/f3d 563156 0 2024-03-23T15:44:51.742 INFO:tasks.workunit.client.0.smithi076.stdout:2/341: chown d1/d4/d64 7353 1 2024-03-23T15:44:51.744 INFO:tasks.workunit.client.0.smithi076.stdout:2/342: creat d1/d4/db/d3e/f77 x:0 0 0 2024-03-23T15:44:51.747 INFO:tasks.workunit.client.0.smithi076.stdout:2/343: dread d1/f19 [4194304,4194304] 0 2024-03-23T15:44:51.747 INFO:tasks.workunit.client.0.smithi076.stdout:2/344: write d1/d4/d64/d1b/f6c [858067,93040] 0 2024-03-23T15:44:51.747 INFO:tasks.workunit.client.0.smithi076.stdout:4/340: dwrite d8/f3f [0,4194304] 0 2024-03-23T15:44:51.747 INFO:tasks.workunit.client.0.smithi076.stdout:5/261: write d1a/f31 [1446134,61417] 0 2024-03-23T15:44:51.748 INFO:tasks.workunit.client.0.smithi076.stdout:0/351: dwrite d2/d8/d52/f5f [0,4194304] 0 2024-03-23T15:44:51.752 INFO:tasks.workunit.client.0.smithi076.stdout:5/262: creat d1a/f5a x:0 0 0 2024-03-23T15:44:51.755 INFO:tasks.workunit.client.0.smithi076.stdout:0/352: mknod d2/de/d10/d12/c6d 0 2024-03-23T15:44:51.755 INFO:tasks.workunit.client.0.smithi076.stdout:0/353: dread - d2/de/d10/d3a/f5c zero size 2024-03-23T15:44:51.757 INFO:tasks.workunit.client.0.smithi076.stdout:0/354: creat d2/de/d10/d12/f6e x:0 0 0 2024-03-23T15:44:51.759 INFO:tasks.workunit.client.0.smithi076.stdout:5/263: unlink d1a/d39/l47 0 2024-03-23T15:44:51.759 INFO:tasks.workunit.client.0.smithi076.stdout:4/341: dwrite d8/d3a/d3b/f26 [0,4194304] 0 2024-03-23T15:44:51.760 INFO:tasks.workunit.client.0.smithi076.stdout:2/345: dwrite d1/f19 [4194304,4194304] 0 2024-03-23T15:44:51.760 INFO:tasks.workunit.client.0.smithi076.stdout:2/346: chown d1/d4/c76 1882 1 2024-03-23T15:44:51.760 INFO:tasks.workunit.client.0.smithi076.stdout:0/355: creat d2/de/d10/d43/f6f x:0 0 0 2024-03-23T15:44:51.760 INFO:tasks.workunit.client.0.smithi076.stdout:0/356: fdatasync d2/d8/f28 0 2024-03-23T15:44:51.761 INFO:tasks.workunit.client.0.smithi076.stdout:5/264: truncate d1a/f2a 4053825 0 2024-03-23T15:44:51.761 INFO:tasks.workunit.client.0.smithi076.stdout:2/347: creat d1/d4/d52/d6e/f78 x:0 0 0 2024-03-23T15:44:51.762 INFO:tasks.workunit.client.0.smithi076.stdout:4/342: creat d8/f6a x:0 0 0 2024-03-23T15:44:51.763 INFO:tasks.workunit.client.0.smithi076.stdout:0/357: mknod d2/d8/c70 0 2024-03-23T15:44:51.766 INFO:tasks.workunit.client.0.smithi076.stdout:0/358: mkdir d2/de/d71 0 2024-03-23T15:44:51.766 INFO:tasks.workunit.client.0.smithi076.stdout:0/359: truncate d2/de/d10/d12/f6e 854652 0 2024-03-23T15:44:51.766 INFO:tasks.workunit.client.0.smithi076.stdout:0/360: dread - d2/de/d10/d12/f6a zero size 2024-03-23T15:44:51.766 INFO:tasks.workunit.client.0.smithi076.stdout:0/361: symlink d2/de/d10/l72 0 2024-03-23T15:44:51.768 INFO:tasks.workunit.client.0.smithi076.stdout:0/362: symlink d2/de/d10/d43/d56/l73 0 2024-03-23T15:44:51.769 INFO:tasks.workunit.client.0.smithi076.stdout:0/363: truncate d2/de/f14 4814249 0 2024-03-23T15:44:51.770 INFO:tasks.workunit.client.0.smithi076.stdout:0/364: symlink d2/de/d10/d43/d56/l74 0 2024-03-23T15:44:51.771 INFO:tasks.workunit.client.0.smithi076.stdout:0/365: symlink d2/de/d71/l75 0 2024-03-23T15:44:51.773 INFO:tasks.workunit.client.0.smithi076.stdout:2/348: dwrite d1/d2e/f3f [0,4194304] 0 2024-03-23T15:44:51.773 INFO:tasks.workunit.client.0.smithi076.stdout:2/349: dread - d1/d4/d64/f23 zero size 2024-03-23T15:44:51.776 INFO:tasks.workunit.client.0.smithi076.stdout:2/350: mkdir d1/d4/d64/d1b/d32/d79 0 2024-03-23T15:44:51.776 INFO:tasks.workunit.client.0.smithi076.stdout:2/351: truncate d1/d4/d52/f70 804425 0 2024-03-23T15:44:51.776 INFO:tasks.workunit.client.0.smithi076.stdout:2/352: mknod d1/d4/d64/d43/c7a 0 2024-03-23T15:44:51.781 INFO:tasks.workunit.client.0.smithi076.stdout:5/265: truncate d1a/f2a 1451021 0 2024-03-23T15:44:51.781 INFO:tasks.workunit.client.0.smithi076.stdout:5/266: chown d1a/d33/f4d 214083 1 2024-03-23T15:44:51.786 INFO:tasks.workunit.client.0.smithi076.stdout:0/366: dwrite d2/d8/f40 [0,4194304] 0 2024-03-23T15:44:51.786 INFO:tasks.workunit.client.0.smithi076.stdout:0/367: truncate d2/de/d10/d43/f57 564684 0 2024-03-23T15:44:51.786 INFO:tasks.workunit.client.0.smithi076.stdout:0/368: truncate d2/de/d10/d43/f63 510216 0 2024-03-23T15:44:51.786 INFO:tasks.workunit.client.0.smithi076.stdout:0/369: chown d2/de/d10/d43/f6f 31254 1 2024-03-23T15:44:51.786 INFO:tasks.workunit.client.0.smithi076.stdout:0/370: write d2/de/d10/d43/f57 [1498819,59798] 0 2024-03-23T15:44:51.787 INFO:tasks.workunit.client.0.smithi076.stdout:5/267: dread f6 [0,4194304] 0 2024-03-23T15:44:51.788 INFO:tasks.workunit.client.0.smithi076.stdout:0/371: mknod d2/de/d10/d3a/d47/d69/c76 0 2024-03-23T15:44:51.789 INFO:tasks.workunit.client.0.smithi076.stdout:5/268: creat d1a/d39/f5b x:0 0 0 2024-03-23T15:44:51.790 INFO:tasks.workunit.client.0.smithi076.stdout:0/372: dread d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:51.796 INFO:tasks.workunit.client.0.smithi076.stdout:0/373: write d2/d8/f62 [1893768,35165] 0 2024-03-23T15:44:51.811 INFO:tasks.workunit.client.0.smithi076.stdout:5/269: dwrite f19 [0,4194304] 0 2024-03-23T15:44:51.813 INFO:tasks.workunit.client.0.smithi076.stdout:0/374: dwrite d2/de/d10/f61 [0,4194304] 0 2024-03-23T15:44:51.814 INFO:tasks.workunit.client.0.smithi076.stdout:0/375: chown d2/de/d10/f61 28 1 2024-03-23T15:44:51.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:51 smithi192 ceph-mon[36653]: pgmap v48: 97 pgs: 97 active+clean; 97 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 1.47k op/s 2024-03-23T15:44:51.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:51 smithi192 ceph-mon[36653]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-23T15:44:51.823 INFO:tasks.workunit.client.0.smithi076.stdout:8/222: sync 2024-03-23T15:44:51.823 INFO:tasks.workunit.client.0.smithi076.stdout:3/270: sync 2024-03-23T15:44:51.823 INFO:tasks.workunit.client.0.smithi076.stdout:7/382: sync 2024-03-23T15:44:51.823 INFO:tasks.workunit.client.0.smithi076.stdout:6/216: sync 2024-03-23T15:44:51.823 INFO:tasks.workunit.client.0.smithi076.stdout:3/271: fdatasync d5/ff 0 2024-03-23T15:44:51.824 INFO:tasks.workunit.client.0.smithi076.stdout:6/217: fsync d1/d2/d27/f46 0 2024-03-23T15:44:51.824 INFO:tasks.workunit.client.0.smithi076.stdout:8/223: mknod d8/d26/d33/c47 0 2024-03-23T15:44:51.825 INFO:tasks.workunit.client.0.smithi076.stdout:6/218: dread d1/d2/f1c [0,4194304] 0 2024-03-23T15:44:51.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:51 smithi076 ceph-mon[32176]: pgmap v48: 97 pgs: 97 active+clean; 97 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 1.47k op/s 2024-03-23T15:44:51.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:51 smithi076 ceph-mon[32176]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-23T15:44:51.826 INFO:tasks.workunit.client.0.smithi076.stdout:3/272: dread d5/f16 [0,4194304] 0 2024-03-23T15:44:51.826 INFO:tasks.workunit.client.0.smithi076.stdout:7/383: rename d8/de/d13/d40/d5d/d3c/d4d to d8/de/d13/d3a/d82 0 2024-03-23T15:44:51.827 INFO:tasks.workunit.client.0.smithi076.stdout:7/384: rmdir d8/d11/d5a 39 2024-03-23T15:44:51.827 INFO:tasks.workunit.client.0.smithi076.stdout:6/219: symlink d1/d4/d10/l4d 0 2024-03-23T15:44:51.827 INFO:tasks.workunit.client.0.smithi076.stdout:6/220: write d1/d4/d10/f4a [2214003,2321] 0 2024-03-23T15:44:51.827 INFO:tasks.workunit.client.0.smithi076.stdout:6/221: fsync d1/f2c 0 2024-03-23T15:44:51.827 INFO:tasks.workunit.client.0.smithi076.stdout:6/222: truncate d1/dd/df/f13 5045906 0 2024-03-23T15:44:51.827 INFO:tasks.workunit.client.0.smithi076.stdout:5/270: dwrite d1a/d2c/f4a [0,4194304] 0 2024-03-23T15:44:51.828 INFO:tasks.workunit.client.0.smithi076.stdout:7/385: getdents d8/de 0 2024-03-23T15:44:51.829 INFO:tasks.workunit.client.0.smithi076.stdout:6/223: mkdir d1/d2/d27/d28/d4e 0 2024-03-23T15:44:51.830 INFO:tasks.workunit.client.0.smithi076.stdout:5/271: symlink d1a/d20/l5c 0 2024-03-23T15:44:51.830 INFO:tasks.workunit.client.0.smithi076.stdout:5/272: stat d1a/c24 0 2024-03-23T15:44:51.830 INFO:tasks.workunit.client.0.smithi076.stdout:5/273: chown d1a/d39/c55 5 1 2024-03-23T15:44:51.836 INFO:tasks.workunit.client.0.smithi076.stdout:7/386: rename d8/d11/f25 to d8/f83 0 2024-03-23T15:44:51.836 INFO:tasks.workunit.client.0.smithi076.stdout:7/387: write d8/de/d13/d3a/d82/f6c [447930,87843] 0 2024-03-23T15:44:51.837 INFO:tasks.workunit.client.0.smithi076.stdout:7/388: write d8/de/d13/d40/d5d/f6a [4857350,39529] 0 2024-03-23T15:44:51.837 INFO:tasks.workunit.client.0.smithi076.stdout:0/376: dwrite d2/d8/f65 [0,4194304] 0 2024-03-23T15:44:51.838 INFO:tasks.workunit.client.0.smithi076.stdout:7/389: creat d8/de/d48/f84 x:0 0 0 2024-03-23T15:44:51.838 INFO:tasks.workunit.client.0.smithi076.stdout:7/390: chown d8/de/d13/d3a/l4a 192221 1 2024-03-23T15:44:51.838 INFO:tasks.workunit.client.0.smithi076.stdout:7/391: fdatasync d8/de/f81 0 2024-03-23T15:44:51.839 INFO:tasks.workunit.client.0.smithi076.stdout:7/392: rmdir d8/de/d13/d40/d67 39 2024-03-23T15:44:51.841 INFO:tasks.workunit.client.0.smithi076.stdout:7/393: mkdir d8/de/d13/d3a/d85 0 2024-03-23T15:44:51.844 INFO:tasks.workunit.client.0.smithi076.stdout:7/394: unlink d8/l47 0 2024-03-23T15:44:51.845 INFO:tasks.workunit.client.0.smithi076.stdout:8/224: dread f2 [4194304,4194304] 0 2024-03-23T15:44:51.846 INFO:tasks.workunit.client.0.smithi076.stdout:7/395: symlink d8/de/d13/d18/l86 0 2024-03-23T15:44:51.846 INFO:tasks.workunit.client.0.smithi076.stdout:6/224: dread d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:51.847 INFO:tasks.workunit.client.0.smithi076.stdout:8/225: creat d8/f48 x:0 0 0 2024-03-23T15:44:51.848 INFO:tasks.workunit.client.0.smithi076.stdout:7/396: creat d8/d4e/f87 x:0 0 0 2024-03-23T15:44:51.849 INFO:tasks.workunit.client.0.smithi076.stdout:6/225: symlink d1/d2/d27/l4f 0 2024-03-23T15:44:51.851 INFO:tasks.workunit.client.0.smithi076.stdout:7/397: unlink d8/de/d13/d40/d5d/d3c/d46/l56 0 2024-03-23T15:44:51.853 INFO:tasks.workunit.client.0.smithi076.stdout:7/398: getdents d8/de/d13/d40/d67/d7a 0 2024-03-23T15:44:51.858 INFO:tasks.workunit.client.0.smithi076.stdout:3/273: dwrite d5/f7 [0,4194304] 0 2024-03-23T15:44:51.859 INFO:tasks.workunit.client.0.smithi076.stdout:3/274: truncate d5/d17/d1c/d32/d39/f3a 74835 0 2024-03-23T15:44:51.860 INFO:tasks.workunit.client.0.smithi076.stdout:6/226: write d1/f2c [3688388,64732] 0 2024-03-23T15:44:51.860 INFO:tasks.workunit.client.0.smithi076.stdout:3/275: rmdir d5/d17/d1c/d31/d47 39 2024-03-23T15:44:51.861 INFO:tasks.workunit.client.0.smithi076.stdout:5/274: dwrite d1a/f5a [0,4194304] 0 2024-03-23T15:44:51.862 INFO:tasks.workunit.client.0.smithi076.stdout:3/276: rmdir d5/d35/d3c 39 2024-03-23T15:44:51.864 INFO:tasks.workunit.client.0.smithi076.stdout:0/377: dwrite d2/de/d10/d43/f4c [0,4194304] 0 2024-03-23T15:44:51.865 INFO:tasks.workunit.client.0.smithi076.stdout:0/378: dread d2/de/d10/d43/f63 [0,4194304] 0 2024-03-23T15:44:51.865 INFO:tasks.workunit.client.0.smithi076.stdout:5/275: symlink d1a/l5d 0 2024-03-23T15:44:51.866 INFO:tasks.workunit.client.0.smithi076.stdout:8/226: dwrite d8/f1a [0,4194304] 0 2024-03-23T15:44:51.868 INFO:tasks.workunit.client.0.smithi076.stdout:0/379: mkdir d2/d1b/d77 0 2024-03-23T15:44:51.869 INFO:tasks.workunit.client.0.smithi076.stdout:8/227: fsync d8/f48 0 2024-03-23T15:44:51.871 INFO:tasks.workunit.client.0.smithi076.stdout:6/227: dread d1/dd/df/f13 [0,4194304] 0 2024-03-23T15:44:51.872 INFO:tasks.workunit.client.0.smithi076.stdout:6/228: truncate d1/d2/f9 3030933 0 2024-03-23T15:44:51.873 INFO:tasks.workunit.client.0.smithi076.stdout:8/228: creat d8/d2e/d3e/f49 x:0 0 0 2024-03-23T15:44:51.878 INFO:tasks.workunit.client.0.smithi076.stdout:7/399: dwrite d8/de/d13/d40/d5d/f3e [0,4194304] 0 2024-03-23T15:44:51.878 INFO:tasks.workunit.client.0.smithi076.stdout:8/229: creat d8/d2e/f4a x:0 0 0 2024-03-23T15:44:51.878 INFO:tasks.workunit.client.0.smithi076.stdout:8/230: fdatasync d8/d26/d27/f37 0 2024-03-23T15:44:51.878 INFO:tasks.workunit.client.0.smithi076.stdout:8/231: fdatasync d8/d2e/d3e/f49 0 2024-03-23T15:44:51.878 INFO:tasks.workunit.client.0.smithi076.stdout:0/380: dread d2/d8/d52/f5f [0,4194304] 0 2024-03-23T15:44:51.878 INFO:tasks.workunit.client.0.smithi076.stdout:0/381: fdatasync d2/d8/f65 0 2024-03-23T15:44:51.878 INFO:tasks.workunit.client.0.smithi076.stdout:6/229: unlink d1/dd/l47 0 2024-03-23T15:44:51.879 INFO:tasks.workunit.client.0.smithi076.stdout:8/232: mknod d8/d2e/d3e/c4b 0 2024-03-23T15:44:51.880 INFO:tasks.workunit.client.0.smithi076.stdout:8/233: rmdir d8/d2e 39 2024-03-23T15:44:51.882 INFO:tasks.workunit.client.0.smithi076.stdout:8/234: getdents d8/d26/d33 0 2024-03-23T15:44:51.882 INFO:tasks.workunit.client.0.smithi076.stdout:8/235: write d8/f1d [842936,88316] 0 2024-03-23T15:44:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:51 smithi183 ceph-mon[36296]: pgmap v48: 97 pgs: 97 active+clean; 97 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 1.47k op/s 2024-03-23T15:44:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:51 smithi183 ceph-mon[36296]: mgrmap e25: x(active, since 90s), standbys: y, z 2024-03-23T15:44:51.887 INFO:tasks.workunit.client.0.smithi076.stdout:1/167: sync 2024-03-23T15:44:51.888 INFO:tasks.workunit.client.0.smithi076.stdout:3/277: dwrite d5/d35/f36 [4194304,4194304] 0 2024-03-23T15:44:51.888 INFO:tasks.workunit.client.0.smithi076.stdout:1/168: chown d0/f2e 916817 1 2024-03-23T15:44:51.888 INFO:tasks.workunit.client.0.smithi076.stdout:3/278: chown d5/d17/c54 52 1 2024-03-23T15:44:51.888 INFO:tasks.workunit.client.0.smithi076.stdout:0/382: write d2/d8/d32/f39 [977669,127245] 0 2024-03-23T15:44:51.890 INFO:tasks.workunit.client.0.smithi076.stdout:5/276: dwrite d1a/f3f [4194304,4194304] 0 2024-03-23T15:44:51.891 INFO:tasks.workunit.client.0.smithi076.stdout:1/169: rename d0/d1/d6/f23 to d0/d2/d1f/f31 0 2024-03-23T15:44:51.894 INFO:tasks.workunit.client.0.smithi076.stdout:5/277: truncate f7 4423758 0 2024-03-23T15:44:51.895 INFO:tasks.workunit.client.0.smithi076.stdout:8/236: dread f7 [4194304,4194304] 0 2024-03-23T15:44:51.895 INFO:tasks.workunit.client.0.smithi076.stdout:2/353: sync 2024-03-23T15:44:51.895 INFO:tasks.workunit.client.0.smithi076.stdout:9/286: sync 2024-03-23T15:44:51.895 INFO:tasks.workunit.client.0.smithi076.stdout:4/343: sync 2024-03-23T15:44:51.895 INFO:tasks.workunit.client.0.smithi076.stdout:4/344: fsync d8/d9/fc 0 2024-03-23T15:44:51.895 INFO:tasks.workunit.client.0.smithi076.stdout:4/345: fdatasync d8/d9/f2d 0 2024-03-23T15:44:51.896 INFO:tasks.workunit.client.0.smithi076.stdout:2/354: dread d1/d4/d64/d43/f50 [0,4194304] 0 2024-03-23T15:44:51.897 INFO:tasks.workunit.client.0.smithi076.stdout:2/355: mknod d1/d4/d64/d43/c7b 0 2024-03-23T15:44:51.897 INFO:tasks.workunit.client.0.smithi076.stdout:3/279: rename d5/d17/d37/d48/l57 to d5/d17/l58 0 2024-03-23T15:44:51.898 INFO:tasks.workunit.client.0.smithi076.stdout:2/356: dread d1/d4/f72 [0,4194304] 0 2024-03-23T15:44:51.898 INFO:tasks.workunit.client.0.smithi076.stdout:7/400: write d8/f3f [1275757,110383] 0 2024-03-23T15:44:51.899 INFO:tasks.workunit.client.0.smithi076.stdout:5/278: rmdir d1a/d33/d36 39 2024-03-23T15:44:51.899 INFO:tasks.workunit.client.0.smithi076.stdout:9/287: getdents d1/d1f/d29/d2d/d37/d47 0 2024-03-23T15:44:51.900 INFO:tasks.workunit.client.0.smithi076.stdout:0/383: dread d2/f9 [0,4194304] 0 2024-03-23T15:44:51.900 INFO:tasks.workunit.client.0.smithi076.stdout:0/384: fsync d2/d8/f53 0 2024-03-23T15:44:51.901 INFO:tasks.workunit.client.0.smithi076.stdout:7/401: rename d8/d11 to d8/de/d13/d40/d88 0 2024-03-23T15:44:51.902 INFO:tasks.workunit.client.0.smithi076.stdout:4/346: getdents d8/d41 0 2024-03-23T15:44:51.902 INFO:tasks.workunit.client.0.smithi076.stdout:4/347: dread - d8/d3a/d3b/f25 zero size 2024-03-23T15:44:51.903 INFO:tasks.workunit.client.0.smithi076.stdout:4/348: chown d8/f5d 0 1 2024-03-23T15:44:51.903 INFO:tasks.workunit.client.0.smithi076.stdout:4/349: readlink d8/d3a/l5e 0 2024-03-23T15:44:51.903 INFO:tasks.workunit.client.0.smithi076.stdout:0/385: dread d2/de/d10/d43/f57 [0,4194304] 0 2024-03-23T15:44:51.903 INFO:tasks.workunit.client.0.smithi076.stdout:7/402: creat d8/de/d13/d3a/f89 x:0 0 0 2024-03-23T15:44:51.903 INFO:tasks.workunit.client.0.smithi076.stdout:7/403: write d8/de/d13/d3a/f50 [3524635,124869] 0 2024-03-23T15:44:51.904 INFO:tasks.workunit.client.0.smithi076.stdout:3/280: mkdir d5/d17/d1c/d32/d39/d59 0 2024-03-23T15:44:51.904 INFO:tasks.workunit.client.0.smithi076.stdout:8/237: read d8/fe [4032699,60537] 0 2024-03-23T15:44:51.905 INFO:tasks.workunit.client.0.smithi076.stdout:7/404: creat d8/de/f8a x:0 0 0 2024-03-23T15:44:51.906 INFO:tasks.workunit.client.0.smithi076.stdout:1/170: dwrite d0/f2e [4194304,4194304] 0 2024-03-23T15:44:51.906 INFO:tasks.workunit.client.0.smithi076.stdout:5/279: dread d1a/d33/f3b [0,4194304] 0 2024-03-23T15:44:51.906 INFO:tasks.workunit.client.0.smithi076.stdout:5/280: read fc [7724015,47274] 0 2024-03-23T15:44:51.907 INFO:tasks.workunit.client.0.smithi076.stdout:7/405: creat d8/de/d13/d3a/f8b x:0 0 0 2024-03-23T15:44:51.908 INFO:tasks.workunit.client.0.smithi076.stdout:9/288: mknod d1/d1f/d29/d2d/d37/c5d 0 2024-03-23T15:44:51.909 INFO:tasks.workunit.client.0.smithi076.stdout:4/350: mknod d8/c6b 0 2024-03-23T15:44:51.910 INFO:tasks.workunit.client.0.smithi076.stdout:0/386: getdents d2/d8 0 2024-03-23T15:44:51.913 INFO:tasks.workunit.client.0.smithi076.stdout:5/281: write f19 [3112695,68837] 0 2024-03-23T15:44:51.913 INFO:tasks.workunit.client.0.smithi076.stdout:8/238: rmdir d8/d2e/d30/d3c 39 2024-03-23T15:44:51.914 INFO:tasks.workunit.client.0.smithi076.stdout:8/239: write d8/fe [2112570,129390] 0 2024-03-23T15:44:51.915 INFO:tasks.workunit.client.0.smithi076.stdout:2/357: dwrite d1/d4/f1e [0,4194304] 0 2024-03-23T15:44:51.916 INFO:tasks.workunit.client.0.smithi076.stdout:7/406: read d8/fd [3674767,8493] 0 2024-03-23T15:44:51.918 INFO:tasks.workunit.client.0.smithi076.stdout:2/358: mkdir d1/d4/d64/d7c 0 2024-03-23T15:44:51.918 INFO:tasks.workunit.client.0.smithi076.stdout:9/289: rename d1/d2f/d3f/l55 to d1/d2f/d3a/l5e 0 2024-03-23T15:44:51.918 INFO:tasks.workunit.client.0.smithi076.stdout:9/290: write d1/d2f/f42 [550933,55567] 0 2024-03-23T15:44:51.918 INFO:tasks.workunit.client.0.smithi076.stdout:9/291: write d1/f1a [900320,78992] 0 2024-03-23T15:44:51.918 INFO:tasks.workunit.client.0.smithi076.stdout:9/292: write d1/f48 [702837,30839] 0 2024-03-23T15:44:51.919 INFO:tasks.workunit.client.0.smithi076.stdout:9/293: chown d1/d20 64682 1 2024-03-23T15:44:51.919 INFO:tasks.workunit.client.0.smithi076.stdout:9/294: chown d1/d20/l38 23498314 1 2024-03-23T15:44:51.920 INFO:tasks.workunit.client.0.smithi076.stdout:7/407: creat d8/de/d13/d40/d88/d5a/f8c x:0 0 0 2024-03-23T15:44:51.920 INFO:tasks.workunit.client.0.smithi076.stdout:2/359: truncate d1/f19 7435154 0 2024-03-23T15:44:51.920 INFO:tasks.workunit.client.0.smithi076.stdout:2/360: stat d1/d4/db/c59 0 2024-03-23T15:44:51.921 INFO:tasks.workunit.client.0.smithi076.stdout:5/282: rename d1a/d20/f53 to d1a/d33/d36/f5e 0 2024-03-23T15:44:51.921 INFO:tasks.workunit.client.0.smithi076.stdout:5/283: stat d1a/l5d 0 2024-03-23T15:44:51.922 INFO:tasks.workunit.client.0.smithi076.stdout:7/408: mkdir d8/d4e/d8d 0 2024-03-23T15:44:51.922 INFO:tasks.workunit.client.0.smithi076.stdout:7/409: fsync d8/de/d13/d40/d5d/d3c/f43 0 2024-03-23T15:44:51.922 INFO:tasks.workunit.client.0.smithi076.stdout:7/410: dread - d8/de/d13/d40/f45 zero size 2024-03-23T15:44:51.923 INFO:tasks.workunit.client.0.smithi076.stdout:9/295: rmdir d1/d1f/d29/d2d/d37 39 2024-03-23T15:44:51.923 INFO:tasks.workunit.client.0.smithi076.stdout:0/387: dread d2/d8/f25 [0,4194304] 0 2024-03-23T15:44:51.923 INFO:tasks.workunit.client.0.smithi076.stdout:0/388: chown d2/de/d10/d3a/d47/f59 16329262 1 2024-03-23T15:44:51.923 INFO:tasks.workunit.client.0.smithi076.stdout:2/361: mkdir d1/d4/db/d15/d29/d7d 0 2024-03-23T15:44:51.923 INFO:tasks.workunit.client.0.smithi076.stdout:2/362: stat d1/d4/db/d15/f55 0 2024-03-23T15:44:51.924 INFO:tasks.workunit.client.0.smithi076.stdout:5/284: mkdir d1a/d39/d5f 0 2024-03-23T15:44:51.924 INFO:tasks.workunit.client.0.smithi076.stdout:2/363: creat d1/d2e/d6a/f7e x:0 0 0 2024-03-23T15:44:51.926 INFO:tasks.workunit.client.0.smithi076.stdout:9/296: symlink d1/d1f/d29/d2d/l5f 0 2024-03-23T15:44:51.926 INFO:tasks.workunit.client.0.smithi076.stdout:9/297: fsync d1/d1f/d46/d4a/f50 0 2024-03-23T15:44:51.927 INFO:tasks.workunit.client.0.smithi076.stdout:2/364: mkdir d1/d4/d64/d43/d7f 0 2024-03-23T15:44:51.927 INFO:tasks.workunit.client.0.smithi076.stdout:0/389: symlink d2/de/d10/d3a/d47/d69/l78 0 2024-03-23T15:44:51.927 INFO:tasks.workunit.client.0.smithi076.stdout:0/390: fsync d2/de/d10/d43/f50 0 2024-03-23T15:44:51.928 INFO:tasks.workunit.client.0.smithi076.stdout:9/298: symlink d1/d2f/d3a/l60 0 2024-03-23T15:44:51.930 INFO:tasks.workunit.client.0.smithi076.stdout:0/391: creat d2/de/d10/d43/f79 x:0 0 0 2024-03-23T15:44:51.930 INFO:tasks.workunit.client.0.smithi076.stdout:0/392: dread - d2/d8/d52/f64 zero size 2024-03-23T15:44:51.930 INFO:tasks.workunit.client.0.smithi076.stdout:1/171: dread d0/d1/d6/d17/f2c [0,4194304] 0 2024-03-23T15:44:51.930 INFO:tasks.workunit.client.0.smithi076.stdout:3/281: dwrite d5/d17/d37/f41 [0,4194304] 0 2024-03-23T15:44:51.931 INFO:tasks.workunit.client.0.smithi076.stdout:5/285: dread d1a/d33/f4d [0,4194304] 0 2024-03-23T15:44:51.931 INFO:tasks.workunit.client.0.smithi076.stdout:5/286: write d1a/d39/f44 [854599,1906] 0 2024-03-23T15:44:51.931 INFO:tasks.workunit.client.0.smithi076.stdout:2/365: creat d1/f80 x:0 0 0 2024-03-23T15:44:51.932 INFO:tasks.workunit.client.0.smithi076.stdout:0/393: write d2/de/d10/d3a/d47/f4e [391035,95013] 0 2024-03-23T15:44:51.932 INFO:tasks.workunit.client.0.smithi076.stdout:0/394: chown d2/de/d10/d3a/d47/f59 26 1 2024-03-23T15:44:51.932 INFO:tasks.workunit.client.0.smithi076.stdout:0/395: chown d2/de/d10/d43/f4c 90838350 1 2024-03-23T15:44:51.933 INFO:tasks.workunit.client.0.smithi076.stdout:4/351: dwrite d8/d3a/d18/d52/f64 [0,4194304] 0 2024-03-23T15:44:51.933 INFO:tasks.workunit.client.0.smithi076.stdout:9/299: rename d1/d1f/f21 to d1/d2f/d3a/d53/f61 0 2024-03-23T15:44:51.933 INFO:tasks.workunit.client.0.smithi076.stdout:8/240: dwrite d8/f18 [0,4194304] 0 2024-03-23T15:44:51.933 INFO:tasks.workunit.client.0.smithi076.stdout:8/241: write d8/d2e/f4a [256189,41315] 0 2024-03-23T15:44:51.934 INFO:tasks.workunit.client.0.smithi076.stdout:1/172: truncate d0/f26 7012935 0 2024-03-23T15:44:51.936 INFO:tasks.workunit.client.0.smithi076.stdout:2/366: unlink d1/d4/db/d15/d29/f4b 0 2024-03-23T15:44:51.936 INFO:tasks.workunit.client.0.smithi076.stdout:3/282: truncate d5/ff 417762 0 2024-03-23T15:44:51.936 INFO:tasks.workunit.client.0.smithi076.stdout:5/287: mknod d1a/d33/c60 0 2024-03-23T15:44:51.938 INFO:tasks.workunit.client.0.smithi076.stdout:7/411: dwrite d8/de/d13/d40/d88/f57 [0,4194304] 0 2024-03-23T15:44:51.938 INFO:tasks.workunit.client.0.smithi076.stdout:7/412: write d8/de/f8a [263920,79237] 0 2024-03-23T15:44:51.938 INFO:tasks.workunit.client.0.smithi076.stdout:7/413: write d8/de/d13/f21 [184772,42011] 0 2024-03-23T15:44:51.938 INFO:tasks.workunit.client.0.smithi076.stdout:7/414: stat d8/de/d13/d40/d88/d5a/f7f 0 2024-03-23T15:44:51.938 INFO:tasks.workunit.client.0.smithi076.stdout:2/367: unlink d1/d4/d52/f62 0 2024-03-23T15:44:51.939 INFO:tasks.workunit.client.0.smithi076.stdout:4/352: symlink d8/d3a/d18/d1b/l6c 0 2024-03-23T15:44:51.939 INFO:tasks.workunit.client.0.smithi076.stdout:0/396: dread d2/de/d10/d43/f50 [0,4194304] 0 2024-03-23T15:44:51.939 INFO:tasks.workunit.client.0.smithi076.stdout:0/397: stat d2/d8/f28 0 2024-03-23T15:44:51.941 INFO:tasks.workunit.client.0.smithi076.stdout:0/398: dread d2/de/d10/d12/f6e [0,4194304] 0 2024-03-23T15:44:51.941 INFO:tasks.workunit.client.0.smithi076.stdout:8/242: symlink d8/d2e/d3e/l4c 0 2024-03-23T15:44:51.942 INFO:tasks.workunit.client.0.smithi076.stdout:2/368: symlink d1/d2e/l81 0 2024-03-23T15:44:51.942 INFO:tasks.workunit.client.0.smithi076.stdout:1/173: mknod d0/d1/d6/d17/d20/c32 0 2024-03-23T15:44:51.942 INFO:tasks.workunit.client.0.smithi076.stdout:1/174: fdatasync d0/d1/d6/d17/d20/f25 0 2024-03-23T15:44:51.944 INFO:tasks.workunit.client.0.smithi076.stdout:3/283: truncate d5/d17/d1c/f1e 1628596 0 2024-03-23T15:44:51.946 INFO:tasks.workunit.client.0.smithi076.stdout:5/288: creat d1a/d39/f61 x:0 0 0 2024-03-23T15:44:51.946 INFO:tasks.workunit.client.0.smithi076.stdout:5/289: write d1a/f2a [1824014,92335] 0 2024-03-23T15:44:51.947 INFO:tasks.workunit.client.0.smithi076.stdout:7/415: rmdir d8/de 39 2024-03-23T15:44:51.947 INFO:tasks.workunit.client.0.smithi076.stdout:8/243: creat d8/d26/f4d x:0 0 0 2024-03-23T15:44:51.949 INFO:tasks.workunit.client.0.smithi076.stdout:1/175: truncate d0/d2/d1f/f31 3506400 0 2024-03-23T15:44:51.952 INFO:tasks.workunit.client.0.smithi076.stdout:7/416: mkdir d8/de/d13/d40/d5d/d3c/d8e 0 2024-03-23T15:44:51.953 INFO:tasks.workunit.client.0.smithi076.stdout:0/399: write d2/de/d10/f61 [1322721,69567] 0 2024-03-23T15:44:51.953 INFO:tasks.workunit.client.0.smithi076.stdout:9/300: dwrite d1/d2f/d3a/d53/f61 [0,4194304] 0 2024-03-23T15:44:51.953 INFO:tasks.workunit.client.0.smithi076.stdout:0/400: fdatasync d2/de/d10/d43/f79 0 2024-03-23T15:44:51.954 INFO:tasks.workunit.client.0.smithi076.stdout:0/401: dread - d2/d8/d2d/f3f zero size 2024-03-23T15:44:51.955 INFO:tasks.workunit.client.0.smithi076.stdout:7/417: mkdir d8/de/d13/d3a/d82/d8f 0 2024-03-23T15:44:51.959 INFO:tasks.workunit.client.0.smithi076.stdout:8/244: creat d8/d2e/d30/d3c/f4e x:0 0 0 2024-03-23T15:44:51.961 INFO:tasks.workunit.client.0.smithi076.stdout:1/176: creat d0/d1/d6/d17/d20/f33 x:0 0 0 2024-03-23T15:44:51.963 INFO:tasks.workunit.client.0.smithi076.stdout:7/418: write d8/de/d13/d40/d88/f2c [8242959,89066] 0 2024-03-23T15:44:51.963 INFO:tasks.workunit.client.0.smithi076.stdout:7/419: dread - d8/de/d13/d24/f55 zero size 2024-03-23T15:44:51.965 INFO:tasks.workunit.client.0.smithi076.stdout:7/420: creat d8/de/d13/d40/f90 x:0 0 0 2024-03-23T15:44:51.966 INFO:tasks.workunit.client.0.smithi076.stdout:9/301: mknod d1/d1f/c62 0 2024-03-23T15:44:51.966 INFO:tasks.workunit.client.0.smithi076.stdout:9/302: fsync d1/f32 0 2024-03-23T15:44:51.966 INFO:tasks.workunit.client.0.smithi076.stdout:7/421: mkdir d8/de/d13/d24/d91 0 2024-03-23T15:44:51.968 INFO:tasks.workunit.client.0.smithi076.stdout:0/402: symlink d2/de/d10/d43/d56/l7a 0 2024-03-23T15:44:51.969 INFO:tasks.workunit.client.0.smithi076.stdout:2/369: dwrite d1/d4/d64/d1d/f3c [0,4194304] 0 2024-03-23T15:44:51.971 INFO:tasks.workunit.client.0.smithi076.stdout:8/245: mkdir d8/d4f 0 2024-03-23T15:44:51.971 INFO:tasks.workunit.client.0.smithi076.stdout:8/246: chown d8/d26/d33/c40 10031 1 2024-03-23T15:44:51.972 INFO:tasks.workunit.client.0.smithi076.stdout:1/177: creat d0/f34 x:0 0 0 2024-03-23T15:44:51.972 INFO:tasks.workunit.client.0.smithi076.stdout:1/178: read - d0/d1/d6/d17/d20/f33 zero size 2024-03-23T15:44:51.973 INFO:tasks.workunit.client.0.smithi076.stdout:2/370: rename d1/d4/d64/d43 to d1/d4/d64/d82 0 2024-03-23T15:44:51.973 INFO:tasks.workunit.client.0.smithi076.stdout:3/284: dwrite d5/f56 [0,4194304] 0 2024-03-23T15:44:51.973 INFO:tasks.workunit.client.0.smithi076.stdout:3/285: chown d5/d17/f1a 121 1 2024-03-23T15:44:51.973 INFO:tasks.workunit.client.0.smithi076.stdout:3/286: readlink d5/l23 0 2024-03-23T15:44:51.974 INFO:tasks.workunit.client.0.smithi076.stdout:3/287: chown d5/f16 456480 1 2024-03-23T15:44:51.974 INFO:tasks.workunit.client.0.smithi076.stdout:0/403: truncate d2/d8/f25 4672425 0 2024-03-23T15:44:51.974 INFO:tasks.workunit.client.0.smithi076.stdout:0/404: dread - d2/d8/d52/f64 zero size 2024-03-23T15:44:51.974 INFO:tasks.workunit.client.0.smithi076.stdout:0/405: stat d2/d8/c70 0 2024-03-23T15:44:51.976 INFO:tasks.workunit.client.0.smithi076.stdout:2/371: link d1/d2e/l39 d1/d4/d64/d1d/l83 0 2024-03-23T15:44:51.976 INFO:tasks.workunit.client.0.smithi076.stdout:2/372: chown d1/d4/d64/d7c 0 1 2024-03-23T15:44:51.979 INFO:tasks.workunit.client.0.smithi076.stdout:8/247: link d8/f10 d8/f50 0 2024-03-23T15:44:51.979 INFO:tasks.workunit.client.0.smithi076.stdout:8/248: fdatasync d8/f1a 0 2024-03-23T15:44:51.979 INFO:tasks.workunit.client.0.smithi076.stdout:2/373: link d1/d4/d64/f23 d1/d4/db/d3e/f84 0 2024-03-23T15:44:51.980 INFO:tasks.workunit.client.0.smithi076.stdout:8/249: fsync d8/f14 0 2024-03-23T15:44:51.980 INFO:tasks.workunit.client.0.smithi076.stdout:2/374: readlink d1/l4a 0 2024-03-23T15:44:51.980 INFO:tasks.workunit.client.0.smithi076.stdout:2/375: getdents d1/d4/db 0 2024-03-23T15:44:51.982 INFO:tasks.workunit.client.0.smithi076.stdout:9/303: read d1/f32 [609393,68510] 0 2024-03-23T15:44:51.983 INFO:tasks.workunit.client.0.smithi076.stdout:5/290: dwrite f12 [0,4194304] 0 2024-03-23T15:44:51.983 INFO:tasks.workunit.client.0.smithi076.stdout:8/250: symlink d8/d26/d27/l51 0 2024-03-23T15:44:51.983 INFO:tasks.workunit.client.0.smithi076.stdout:2/376: rmdir d1/d4/d64/d82 39 2024-03-23T15:44:51.984 INFO:tasks.workunit.client.0.smithi076.stdout:0/406: write d2/d1b/f42 [110756,88395] 0 2024-03-23T15:44:51.984 INFO:tasks.workunit.client.0.smithi076.stdout:5/291: creat d1a/d39/d5f/f62 x:0 0 0 2024-03-23T15:44:51.984 INFO:tasks.workunit.client.0.smithi076.stdout:5/292: read - d1a/d4b/f4c zero size 2024-03-23T15:44:51.986 INFO:tasks.workunit.client.0.smithi076.stdout:3/288: dwrite d5/d17/f1a [0,4194304] 0 2024-03-23T15:44:51.987 INFO:tasks.workunit.client.0.smithi076.stdout:0/407: mknod d2/d1b/d77/c7b 0 2024-03-23T15:44:51.987 INFO:tasks.workunit.client.0.smithi076.stdout:0/408: chown d2/de/d10/d12/c6d 447693607 1 2024-03-23T15:44:51.989 INFO:tasks.workunit.client.0.smithi076.stdout:5/293: mknod d1a/d4b/c63 0 2024-03-23T15:44:51.991 INFO:tasks.workunit.client.0.smithi076.stdout:8/251: write f7 [3959474,124243] 0 2024-03-23T15:44:51.992 INFO:tasks.workunit.client.0.smithi076.stdout:9/304: write d1/d20/f43 [3684382,119785] 0 2024-03-23T15:44:51.994 INFO:tasks.workunit.client.0.smithi076.stdout:3/289: rename d5/d17/d37 to d5/d35/d4c/d5a 0 2024-03-23T15:44:51.994 INFO:tasks.workunit.client.0.smithi076.stdout:0/409: rmdir d2/de/d10/d43/d56 39 2024-03-23T15:44:51.995 INFO:tasks.workunit.client.0.smithi076.stdout:3/290: dread d5/f12 [0,4194304] 0 2024-03-23T15:44:51.995 INFO:tasks.workunit.client.0.smithi076.stdout:3/291: write d5/d17/f29 [236541,24408] 0 2024-03-23T15:44:51.995 INFO:tasks.workunit.client.0.smithi076.stdout:3/292: stat d5/d17/d1c/d32/d4a 0 2024-03-23T15:44:51.996 INFO:tasks.workunit.client.0.smithi076.stdout:9/305: symlink d1/d1f/d29/d2d/l63 0 2024-03-23T15:44:51.998 INFO:tasks.workunit.client.0.smithi076.stdout:3/293: dread d5/d17/d1c/f1e [0,4194304] 0 2024-03-23T15:44:51.998 INFO:tasks.workunit.client.0.smithi076.stdout:0/410: rename d2/de/d10/d43/f57 to d2/de/d10/d3a/d47/f7c 0 2024-03-23T15:44:52.000 INFO:tasks.workunit.client.0.smithi076.stdout:5/294: unlink d1a/d20/l5c 0 2024-03-23T15:44:52.000 INFO:tasks.workunit.client.0.smithi076.stdout:5/295: write d1a/d4b/f4c [51493,116038] 0 2024-03-23T15:44:52.002 INFO:tasks.workunit.client.0.smithi076.stdout:3/294: symlink d5/d17/d1c/d32/l5b 0 2024-03-23T15:44:52.002 INFO:tasks.workunit.client.0.smithi076.stdout:3/295: chown d5/d17/d1c/d32/c4f 5817567 1 2024-03-23T15:44:52.002 INFO:tasks.workunit.client.0.smithi076.stdout:3/296: write d5/d35/d4c/f51 [168211,53927] 0 2024-03-23T15:44:52.004 INFO:tasks.workunit.client.0.smithi076.stdout:0/411: symlink d2/de/d10/d43/l7d 0 2024-03-23T15:44:52.004 INFO:tasks.workunit.client.0.smithi076.stdout:2/377: dwrite d1/d4/d64/d1d/f73 [0,4194304] 0 2024-03-23T15:44:52.006 INFO:tasks.workunit.client.0.smithi076.stdout:8/252: dwrite d8/ff [16777216,4194304] 0 2024-03-23T15:44:52.006 INFO:tasks.workunit.client.0.smithi076.stdout:9/306: write d1/d1f/f30 [5403605,22400] 0 2024-03-23T15:44:52.008 INFO:tasks.workunit.client.0.smithi076.stdout:9/307: mkdir d1/d2f/d3a/d64 0 2024-03-23T15:44:52.015 INFO:tasks.workunit.client.0.smithi076.stdout:8/253: dread d8/f1e [0,4194304] 0 2024-03-23T15:44:52.015 INFO:tasks.workunit.client.0.smithi076.stdout:2/378: dread d1/d4/d64/f4e [0,4194304] 0 2024-03-23T15:44:52.017 INFO:tasks.workunit.client.0.smithi076.stdout:2/379: creat d1/d4/d64/d7c/f85 x:0 0 0 2024-03-23T15:44:52.018 INFO:tasks.workunit.client.0.smithi076.stdout:8/254: creat d8/f52 x:0 0 0 2024-03-23T15:44:52.018 INFO:tasks.workunit.client.0.smithi076.stdout:8/255: chown f2 1 1 2024-03-23T15:44:52.019 INFO:tasks.workunit.client.0.smithi076.stdout:2/380: unlink d1/d4/f10 0 2024-03-23T15:44:52.019 INFO:tasks.workunit.client.0.smithi076.stdout:2/381: truncate d1/d4/d52/f70 776092 0 2024-03-23T15:44:52.019 INFO:tasks.workunit.client.0.smithi076.stdout:2/382: fsync d1/d4/d64/d1d/f73 0 2024-03-23T15:44:52.022 INFO:tasks.workunit.client.0.smithi076.stdout:2/383: unlink d1/c41 0 2024-03-23T15:44:52.025 INFO:tasks.workunit.client.0.smithi076.stdout:5/296: dwrite d1a/f2a [0,4194304] 0 2024-03-23T15:44:52.025 INFO:tasks.workunit.client.0.smithi076.stdout:8/256: dread d8/f2a [0,4194304] 0 2024-03-23T15:44:52.026 INFO:tasks.workunit.client.0.smithi076.stdout:8/257: dread d8/f11 [0,4194304] 0 2024-03-23T15:44:52.027 INFO:tasks.workunit.client.0.smithi076.stdout:5/297: mkdir d1a/d33/d3a/d64 0 2024-03-23T15:44:52.029 INFO:tasks.workunit.client.0.smithi076.stdout:5/298: mkdir d1a/d39/d65 0 2024-03-23T15:44:52.030 INFO:tasks.workunit.client.0.smithi076.stdout:2/384: dread d1/d4/d64/d1b/f1c [0,4194304] 0 2024-03-23T15:44:52.032 INFO:tasks.workunit.client.0.smithi076.stdout:2/385: symlink d1/d4/db/d15/d29/d7d/l86 0 2024-03-23T15:44:52.032 INFO:tasks.workunit.client.0.smithi076.stdout:2/386: fdatasync d1/d4/db/d15/d29/f30 0 2024-03-23T15:44:52.032 INFO:tasks.workunit.client.0.smithi076.stdout:5/299: unlink f13 0 2024-03-23T15:44:52.034 INFO:tasks.workunit.client.0.smithi076.stdout:2/387: symlink d1/d4/db/d15/d29/l87 0 2024-03-23T15:44:52.036 INFO:tasks.workunit.client.0.smithi076.stdout:9/308: dwrite d1/d20/f25 [4194304,4194304] 0 2024-03-23T15:44:52.038 INFO:tasks.workunit.client.0.smithi076.stdout:3/297: dwrite d5/d17/d1c/f1e [0,4194304] 0 2024-03-23T15:44:52.038 INFO:tasks.workunit.client.0.smithi076.stdout:9/309: mkdir d1/d2f/d3a/d65 0 2024-03-23T15:44:52.039 INFO:tasks.workunit.client.0.smithi076.stdout:9/310: read d1/f12 [2982813,21420] 0 2024-03-23T15:44:52.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/412: dwrite d2/d8/f28 [0,4194304] 0 2024-03-23T15:44:52.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/413: fsync d2/d8/d32/f39 0 2024-03-23T15:44:52.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/414: fsync d2/de/d10/f5d 0 2024-03-23T15:44:52.045 INFO:tasks.workunit.client.0.smithi076.stdout:9/311: getdents d1/d2f/d3a/d53 0 2024-03-23T15:44:52.046 INFO:tasks.workunit.client.0.smithi076.stdout:9/312: write d1/f26 [1219134,22963] 0 2024-03-23T15:44:52.046 INFO:tasks.workunit.client.0.smithi076.stdout:9/313: readlink d1/d1f/d29/d2d/l5f 0 2024-03-23T15:44:52.048 INFO:tasks.workunit.client.0.smithi076.stdout:0/415: link d2/de/d10/d12/c5b d2/de/d10/d3a/d47/c7e 0 2024-03-23T15:44:52.048 INFO:tasks.workunit.client.0.smithi076.stdout:0/416: dread - d2/de/d10/f5d zero size 2024-03-23T15:44:52.049 INFO:tasks.workunit.client.0.smithi076.stdout:0/417: chown d2/de/d10/d12/l2b 1 1 2024-03-23T15:44:52.052 INFO:tasks.workunit.client.0.smithi076.stdout:3/298: dwrite d5/d17/f1a [4194304,4194304] 0 2024-03-23T15:44:52.053 INFO:tasks.workunit.client.0.smithi076.stdout:3/299: write d5/d17/d1c/d32/d4a/f55 [1008108,46797] 0 2024-03-23T15:44:52.053 INFO:tasks.workunit.client.0.smithi076.stdout:2/388: dwrite d1/d4/db/d3e/f77 [0,4194304] 0 2024-03-23T15:44:52.053 INFO:tasks.workunit.client.0.smithi076.stdout:2/389: chown d1/d4/d64/d82/c7b 1643426951 1 2024-03-23T15:44:52.053 INFO:tasks.workunit.client.0.smithi076.stdout:8/258: dwrite d8/f14 [0,4194304] 0 2024-03-23T15:44:52.056 INFO:tasks.workunit.client.0.smithi076.stdout:9/314: creat d1/d1f/d29/d2d/d37/f66 x:0 0 0 2024-03-23T15:44:52.057 INFO:tasks.workunit.client.0.smithi076.stdout:2/390: symlink d1/d2e/d6a/l88 0 2024-03-23T15:44:52.058 INFO:tasks.workunit.client.0.smithi076.stdout:2/391: fdatasync d1/d4/db/f3d 0 2024-03-23T15:44:52.058 INFO:tasks.workunit.client.0.smithi076.stdout:8/259: dread d8/f1d [0,4194304] 0 2024-03-23T15:44:52.058 INFO:tasks.workunit.client.0.smithi076.stdout:2/392: creat d1/d4/db/d15/f89 x:0 0 0 2024-03-23T15:44:52.060 INFO:tasks.workunit.client.0.smithi076.stdout:3/300: link d5/d17/d1c/d32/f34 d5/d35/d4c/f5c 0 2024-03-23T15:44:52.062 INFO:tasks.workunit.client.0.smithi076.stdout:0/418: dwrite d2/d8/fa [4194304,4194304] 0 2024-03-23T15:44:52.067 INFO:tasks.workunit.client.0.smithi076.stdout:8/260: creat d8/d4f/f53 x:0 0 0 2024-03-23T15:44:52.068 INFO:tasks.workunit.client.0.smithi076.stdout:3/301: mkdir d5/d17/d1c/d32/d5d 0 2024-03-23T15:44:52.069 INFO:tasks.workunit.client.0.smithi076.stdout:9/315: unlink d1/d2f/d3a/l5e 0 2024-03-23T15:44:52.071 INFO:tasks.workunit.client.0.smithi076.stdout:0/419: dread d2/d8/f53 [0,4194304] 0 2024-03-23T15:44:52.072 INFO:tasks.workunit.client.0.smithi076.stdout:0/420: truncate d2/de/d10/d43/f63 835856 0 2024-03-23T15:44:52.072 INFO:tasks.workunit.client.0.smithi076.stdout:8/261: mknod d8/c54 0 2024-03-23T15:44:52.072 INFO:tasks.workunit.client.0.smithi076.stdout:8/262: stat d8/d2e/d30/f32 0 2024-03-23T15:44:52.072 INFO:tasks.workunit.client.0.smithi076.stdout:8/263: chown d8/d2e/d30/f32 267991 1 2024-03-23T15:44:52.073 INFO:tasks.workunit.client.0.smithi076.stdout:0/421: truncate d2/d8/d32/f39 1181479 0 2024-03-23T15:44:52.073 INFO:tasks.workunit.client.0.smithi076.stdout:0/422: readlink d2/d1b/l55 0 2024-03-23T15:44:52.073 INFO:tasks.workunit.client.0.smithi076.stdout:0/423: fdatasync d2/d8/d32/f39 0 2024-03-23T15:44:52.075 INFO:tasks.workunit.client.0.smithi076.stdout:8/264: mknod d8/d26/c55 0 2024-03-23T15:44:52.075 INFO:tasks.workunit.client.0.smithi076.stdout:2/393: dwrite d1/d4/db/f3d [0,4194304] 0 2024-03-23T15:44:52.078 INFO:tasks.workunit.client.0.smithi076.stdout:2/394: mkdir d1/d2e/d8a 0 2024-03-23T15:44:52.078 INFO:tasks.workunit.client.0.smithi076.stdout:2/395: write d1/d4/db/d15/d29/f30 [614378,24848] 0 2024-03-23T15:44:52.082 INFO:tasks.workunit.client.0.smithi076.stdout:8/265: read f2 [262106,41569] 0 2024-03-23T15:44:52.082 INFO:tasks.workunit.client.0.smithi076.stdout:8/266: dread - d8/f52 zero size 2024-03-23T15:44:52.082 INFO:tasks.workunit.client.0.smithi076.stdout:3/302: dwrite d5/d35/d4c/f5c [0,4194304] 0 2024-03-23T15:44:52.085 INFO:tasks.workunit.client.0.smithi076.stdout:3/303: dread d5/d17/d1c/d32/d39/f3a [0,4194304] 0 2024-03-23T15:44:52.086 INFO:tasks.workunit.client.0.smithi076.stdout:9/316: dwrite d1/d1f/d29/d2d/d37/f3c [0,4194304] 0 2024-03-23T15:44:52.088 INFO:tasks.workunit.client.0.smithi076.stdout:3/304: rename d5/d17/d1c/l30 to d5/d35/l5e 0 2024-03-23T15:44:52.088 INFO:tasks.workunit.client.0.smithi076.stdout:3/305: stat d5/f13 0 2024-03-23T15:44:52.092 INFO:tasks.workunit.client.0.smithi076.stdout:9/317: link d1/d2f/d3a/d53/c5c d1/d1f/d46/d4a/d5a/c67 0 2024-03-23T15:44:52.093 INFO:tasks.workunit.client.0.smithi076.stdout:0/424: dwrite d2/d8/f62 [0,4194304] 0 2024-03-23T15:44:52.095 INFO:tasks.workunit.client.0.smithi076.stdout:9/318: truncate d1/f26 93595 0 2024-03-23T15:44:52.095 INFO:tasks.workunit.client.0.smithi076.stdout:9/319: fsync d1/d2f/d3f/f49 0 2024-03-23T15:44:52.095 INFO:tasks.workunit.client.0.smithi076.stdout:9/320: truncate d1/d1f/d29/d2d/d37/f66 772766 0 2024-03-23T15:44:52.099 INFO:tasks.workunit.client.0.smithi076.stdout:9/321: truncate d1/d2f/d3a/d53/f61 3691562 0 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:3/306: write d5/d35/d4c/f5c [1431898,84075] 0 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:3/307: write d5/f28 [832878,44828] 0 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:2/396: dwrite d1/d4/d64/d1d/f73 [0,4194304] 0 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:9/322: symlink d1/d2f/d3f/l68 0 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:6/230: sync 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:6/231: fsync d1/d2/d27/f46 0 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:3/308: creat d5/d17/d1c/d32/f5f x:0 0 0 2024-03-23T15:44:52.100 INFO:tasks.workunit.client.0.smithi076.stdout:2/397: symlink d1/d4/d64/d1b/d32/d79/l8b 0 2024-03-23T15:44:52.101 INFO:tasks.workunit.client.0.smithi076.stdout:8/267: dwrite d8/fc [8388608,4194304] 0 2024-03-23T15:44:52.102 INFO:tasks.workunit.client.0.smithi076.stdout:9/323: creat d1/d2f/d3a/d64/f69 x:0 0 0 2024-03-23T15:44:52.102 INFO:tasks.workunit.client.0.smithi076.stdout:6/232: truncate d1/d4/f35 84972 0 2024-03-23T15:44:52.104 INFO:tasks.workunit.client.0.smithi076.stdout:8/268: mkdir d8/d26/d27/d56 0 2024-03-23T15:44:52.110 INFO:tasks.workunit.client.0.smithi076.stdout:9/324: dread d1/f26 [0,4194304] 0 2024-03-23T15:44:52.113 INFO:tasks.workunit.client.0.smithi076.stdout:9/325: symlink d1/d2f/d3f/l6a 0 2024-03-23T15:44:52.114 INFO:tasks.workunit.client.0.smithi076.stdout:0/425: dwrite d2/d8/d2d/f3f [0,4194304] 0 2024-03-23T15:44:52.119 INFO:tasks.workunit.client.0.smithi076.stdout:0/426: getdents d2/de/d10 0 2024-03-23T15:44:52.124 INFO:tasks.workunit.client.0.smithi076.stdout:0/427: truncate d2/de/d10/f61 4457677 0 2024-03-23T15:44:52.125 INFO:tasks.workunit.client.0.smithi076.stdout:3/309: dwrite d5/f16 [0,4194304] 0 2024-03-23T15:44:52.127 INFO:tasks.workunit.client.0.smithi076.stdout:6/233: dwrite d1/dd/df/f13 [4194304,4194304] 0 2024-03-23T15:44:52.127 INFO:tasks.workunit.client.0.smithi076.stdout:6/234: fsync d1/dd/d2a/d23/d36/f3e 0 2024-03-23T15:44:52.128 INFO:tasks.workunit.client.0.smithi076.stdout:6/235: chown d1/d4 506 1 2024-03-23T15:44:52.128 INFO:tasks.workunit.client.0.smithi076.stdout:3/310: creat d5/d17/d1c/d32/d39/d59/f60 x:0 0 0 2024-03-23T15:44:52.129 INFO:tasks.workunit.client.0.smithi076.stdout:3/311: mkdir d5/d17/d1c/d31/d61 0 2024-03-23T15:44:52.129 INFO:tasks.workunit.client.0.smithi076.stdout:3/312: chown d5/d35/d4c/d5a/l3f 183204 1 2024-03-23T15:44:52.130 INFO:tasks.workunit.client.0.smithi076.stdout:3/313: creat d5/d35/d4c/d5a/d48/f62 x:0 0 0 2024-03-23T15:44:52.132 INFO:tasks.workunit.client.0.smithi076.stdout:8/269: dwrite d8/f50 [0,4194304] 0 2024-03-23T15:44:52.132 INFO:tasks.workunit.client.0.smithi076.stdout:8/270: truncate d8/d2e/f4a 359057 0 2024-03-23T15:44:52.135 INFO:tasks.workunit.client.0.smithi076.stdout:8/271: creat d8/f57 x:0 0 0 2024-03-23T15:44:52.136 INFO:tasks.workunit.client.0.smithi076.stdout:8/272: symlink d8/d2e/d30/d3c/l58 0 2024-03-23T15:44:52.137 INFO:tasks.workunit.client.0.smithi076.stdout:0/428: dread d2/d8/fa [0,4194304] 0 2024-03-23T15:44:52.139 INFO:tasks.workunit.client.0.smithi076.stdout:9/326: dwrite d1/d1f/d29/d2d/d37/f59 [0,4194304] 0 2024-03-23T15:44:52.139 INFO:tasks.workunit.client.0.smithi076.stdout:0/429: mkdir d2/de/d10/d43/d7f 0 2024-03-23T15:44:52.143 INFO:tasks.workunit.client.0.smithi076.stdout:9/327: rename d1/d1f/d29/d2d/d37/d47/c4c to d1/d2f/d3f/c6b 0 2024-03-23T15:44:52.146 INFO:tasks.workunit.client.0.smithi076.stdout:9/328: symlink d1/d2f/d3a/d53/l6c 0 2024-03-23T15:44:52.146 INFO:tasks.workunit.client.0.smithi076.stdout:9/329: fsync d1/d1f/d29/d2d/d37/f59 0 2024-03-23T15:44:52.146 INFO:tasks.workunit.client.0.smithi076.stdout:6/236: link d1/d2/d27/l4f d1/d4/d10/l50 0 2024-03-23T15:44:52.146 INFO:tasks.workunit.client.0.smithi076.stdout:3/314: dwrite d5/d17/d1c/f2b [4194304,4194304] 0 2024-03-23T15:44:52.148 INFO:tasks.workunit.client.0.smithi076.stdout:3/315: mknod d5/d17/d1c/d32/d39/d59/c63 0 2024-03-23T15:44:52.162 INFO:tasks.workunit.client.0.smithi076.stdout:3/316: dread d5/d17/d1c/f1d [4194304,4194304] 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:6/237: dwrite d1/f11 [0,4194304] 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:9/330: dwrite d1/d20/f45 [0,4194304] 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:3/317: rename d5/d17/f1a to d5/d17/d1c/d31/f64 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:6/238: mknod d1/d2/d27/d28/d4e/c51 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:9/331: truncate d1/f19 2754455 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:9/332: read - d1/d1f/d29/d2d/f33 zero size 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:6/239: mknod d1/d4/d10/c52 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:6/240: symlink d1/d4/d10/d39/l53 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:3/318: dwrite d5/d17/f29 [0,4194304] 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:3/319: creat d5/d17/d1c/d32/d39/f65 x:0 0 0 2024-03-23T15:44:52.648 INFO:tasks.workunit.client.0.smithi076.stdout:3/320: symlink d5/d17/d1c/d32/d4a/l66 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/321: mkdir d5/d35/d3c/d67 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/322: truncate d5/f28 1752460 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:9/333: dwrite d1/d1f/d29/f54 [0,4194304] 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/323: truncate d5/d17/d1c/d32/d4a/f55 278056 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:6/241: dwrite d1/d4/d10/f4a [0,4194304] 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:6/242: read - d1/d4/d10/f30 zero size 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:6/243: creat d1/dd/f54 x:0 0 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/324: write d5/d35/d4c/d5a/f49 [1211292,118273] 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:6/244: creat d1/dd/d2a/d23/d36/f55 x:0 0 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/325: rmdir d5/d17/d1c/d32/d4a 39 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/326: mknod d5/d17/d1c/d31/d61/c68 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/327: mknod d5/d17/d1c/d31/d47/c69 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/328: fsync d5/d17/d1c/d32/f34 0 2024-03-23T15:44:52.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/329: rename d5/l25 to d5/d17/d1c/l6a 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/245: dread d1/dd/d2a/d23/f3a [0,4194304] 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/246: truncate d1/dd/d2a/d23/d36/f3e 713929 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:9/334: dwrite d1/f18 [0,4194304] 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:3/330: mkdir d5/d17/d1c/d31/d6b 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/247: mknod d1/d4/d10/d39/c56 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/248: chown d1/dd/d2a/d23/d36/l40 14 1 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/249: getdents d1 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/250: dread - d1/dd/d2a/d23/d36/f55 zero size 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/251: write d1/d2/f1c [611275,96008] 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/252: readlink d1/la 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/253: fdatasync d1/f11 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/254: mknod d1/d2/d27/c57 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/255: symlink d1/dd/l58 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/256: creat d1/dd/d2a/d23/d36/f59 x:0 0 0 2024-03-23T15:44:52.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/257: fsync d1/dd/d2a/d23/f3a 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:4/353: sync 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:6/258: rename d1/dd/d2a/d23/c43 to d1/dd/d2a/c5a 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:6/259: write d1/f2c [2106361,58417] 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:6/260: write d1/d2/d27/f46 [823661,32843] 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:4/354: dwrite d8/f42 [0,4194304] 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/335: dwrite d1/d1f/d29/d2d/d37/f59 [0,4194304] 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:4/355: creat d8/d3a/d18/d52/f6d x:0 0 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/336: symlink d1/d2f/d3a/d65/l6d 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/337: write d1/f18 [4631234,73702] 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/338: mknod d1/d2f/d3a/d64/c6e 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/339: creat d1/d1f/d29/d2d/f6f x:0 0 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/340: mknod d1/c70 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/341: fdatasync d1/f12 0 2024-03-23T15:44:52.651 INFO:tasks.workunit.client.0.smithi076.stdout:9/342: symlink d1/d1f/d29/d2d/d37/l71 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:4/356: dread d8/f49 [0,4194304] 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:4/357: read - d8/f56 zero size 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:4/358: write d8/f66 [29188,55438] 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:4/359: read d8/d3a/d18/f21 [4948190,12463] 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:4/360: truncate d8/f6a 490411 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:9/343: link d1/f1a d1/d1f/f72 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:9/344: creat d1/d2f/d3a/d4d/f73 x:0 0 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:4/361: getdents d8/d3a/d18/d52 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:4/362: rename d8/d3a/d3b/c4e to d8/c6e 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:9/345: dread d1/d20/f25 [4194304,4194304] 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:9/346: write d1/f12 [3748575,10057] 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:9/347: unlink d1/l1b 0 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:9/348: chown d1/d2f/d3a/d65/l6d 217 1 2024-03-23T15:44:52.652 INFO:tasks.workunit.client.0.smithi076.stdout:9/349: chown d1/f5b 55 1 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/350: getdents d1/d20 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/351: getdents d1/d1f/d29/d2d/d37 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/352: truncate d1/d1f/d46/d4a/f50 1175415 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/353: mkdir d1/d2f/d3a/d74 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/354: fsync d1/d2f/f42 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:4/363: dwrite d8/f42 [4194304,4194304] 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:4/364: dread d8/d3a/d18/f2b [0,4194304] 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/355: dread d1/d1f/d29/d2d/d37/f3c [0,4194304] 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/356: fdatasync d1/d1f/f22 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/357: dread d1/f48 [0,4194304] 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:4/365: symlink d8/d9/l6f 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:4/366: stat d8/d3a/f34 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/358: getdents d1/d1f/d46/d4a 0 2024-03-23T15:44:52.653 INFO:tasks.workunit.client.0.smithi076.stdout:9/359: mkdir d1/d75 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:9/360: symlink d1/d1f/d29/d2d/d37/d47/l76 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:9/361: dread - d1/f5b zero size 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:9/362: readlink d1/d2f/d3a/d4d/l58 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:9/363: readlink d1/d2f/d3a/d4d/l58 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:9/364: mknod d1/d2f/d3a/d65/c77 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/367: write d8/f1a [2061460,43657] 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:9/365: rename d1/d2f/d3a/d64/c6e to d1/d1f/d29/d2d/c78 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/368: read d8/d9/d32/f5b [98638,22058] 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/369: chown d8/d9/c38 2059815 1 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/370: symlink d8/d9/l70 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/371: chown d8/d9/l70 735 1 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/372: chown d8/d9/l46 101 1 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/373: creat d8/f71 x:0 0 0 2024-03-23T15:44:52.654 INFO:tasks.workunit.client.0.smithi076.stdout:4/374: creat d8/d9/d32/f72 x:0 0 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/375: mkdir d8/d3a/d69/d73 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/376: fsync d8/d9/f12 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/377: getdents d8/d3a/d18/d1b 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:7/422: sync 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:1/179: sync 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:5/300: sync 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/378: dread d8/f1a [0,4194304] 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:5/301: dread d1a/d33/f4d [0,4194304] 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/379: creat d8/d3a/f74 x:0 0 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:5/302: creat d1a/d39/f66 x:0 0 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/380: mkdir d8/d3a/d18/d75 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/381: dread - d8/f56 zero size 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/382: write d8/d9/d32/f72 [604482,49886] 0 2024-03-23T15:44:52.655 INFO:tasks.workunit.client.0.smithi076.stdout:5/303: read d1a/d20/f34 [336891,25723] 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:5/304: stat d1a/d33/f54 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:5/305: truncate d1a/f3c 2959817 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:4/383: dread d8/d9/d32/f72 [0,4194304] 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:4/384: mkdir d8/d3a/d69/d73/d76 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:4/385: creat d8/d3a/d69/d73/d76/f77 x:0 0 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:4/386: creat d8/d3a/d69/d73/d76/f78 x:0 0 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:4/387: getdents d8/d3a/d18/d1b 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:1/180: dwrite d0/f26 [0,4194304] 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:1/181: readlink d0/d1/d6/d17/l1a 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:4/388: creat d8/d3a/d18/d52/f79 x:0 0 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:7/423: dwrite d8/de/d13/d40/d5d/f65 [0,4194304] 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:7/424: write d8/de/d13/d40/d5d/d3c/f43 [4595016,8024] 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:1/182: rename d0/d1/d19 to d0/d35 0 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:1/183: chown d0/d2/d1f/l24 2379868 1 2024-03-23T15:44:52.656 INFO:tasks.workunit.client.0.smithi076.stdout:5/306: dwrite d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:5/307: chown f4 26 1 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:5/308: chown f4 500 1 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:5/309: truncate d1a/d33/d36/f5e 740563 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:1/184: write d0/f26 [1826700,84880] 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:1/185: fsync d0/f2e 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:1/186: creat d0/d35/f36 x:0 0 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:1/187: creat d0/d2/f37 x:0 0 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:1/188: rmdir d0/d1/d6/d17/d20 39 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:5/310: dwrite d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:5/311: truncate d1a/d33/d36/f5e 838149 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:5/312: fdatasync d1a/d33/f51 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:7/425: dwrite d8/de/d13/d40/d5d/d3c/d46/f51 [0,4194304] 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:7/426: stat d8/de/f8a 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:7/427: read d8/de/d13/d40/d5d/d3c/f73 [148906,123095] 0 2024-03-23T15:44:52.657 INFO:tasks.workunit.client.0.smithi076.stdout:5/313: mkdir d1a/d33/d3a/d67 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/314: readlink d1a/d4b/l4e 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/315: creat d1a/f68 x:0 0 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:7/428: dread d8/de/f26 [0,4194304] 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:7/429: fsync d8/de/f81 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:7/430: getdents d8/d4e/d8d 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:4/389: dwrite d8/f42 [4194304,4194304] 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/316: rmdir d1a/d39/d5f 39 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:4/390: creat d8/d3a/d69/d73/d76/f7a x:0 0 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/317: creat d1a/d33/f69 x:0 0 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/318: getdents d1a/d39 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/319: mknod d1a/d33/c6a 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/320: rename f11 to d1a/d39/d65/f6b 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/321: write f4 [965871,75321] 0 2024-03-23T15:44:52.658 INFO:tasks.workunit.client.0.smithi076.stdout:5/322: getdents d1a/d39/d5f 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:1/189: dwrite d0/f26 [0,4194304] 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:1/190: stat d0/d1/d6/d17/d20/l2d 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:4/391: dwrite d8/d9/f12 [4194304,4194304] 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:7/431: dwrite d8/de/d13/d3a/f50 [0,4194304] 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:7/432: truncate d8/de/d13/f21 468962 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:4/392: rename d8/c23 to d8/d3a/d69/d73/c7b 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:4/393: dread d8/d9/d32/f5b [0,4194304] 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:1/191: dread d0/d1/d6/d17/f2c [0,4194304] 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:1/192: write d0/f2e [2404347,99360] 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:7/433: dread f3 [0,4194304] 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:4/394: getdents d8/d9/d1d 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:7/434: mkdir d8/de/d48/d92 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:7/435: creat d8/de/d13/d40/d5d/d3c/d46/f93 x:0 0 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:1/193: creat d0/d1/d6/d17/f38 x:0 0 0 2024-03-23T15:44:52.659 INFO:tasks.workunit.client.0.smithi076.stdout:5/323: dwrite d1a/d33/d36/f5e [0,4194304] 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/436: rename d8/de/d13/d40/d5d/d3c/d46/f64 to d8/de/d13/f94 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/437: truncate d8/f62 749472 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/438: readlink d8/de/d13/d18/l86 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/439: write d8/de/d13/f21 [1459116,28273] 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:4/395: symlink d8/d3a/l7c 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:4/396: write d8/f6a [912258,52724] 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/440: link d8/de/d13/d3a/d82/f66 d8/de/d13/d40/d88/d5a/f95 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/441: dread - d8/de/d13/d40/f90 zero size 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:1/194: symlink d0/l39 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/442: rename d8/de/d13/d40/d67 to d8/de/d13/d24/d91/d96 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:2/398: sync 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/443: unlink d8/de/d13/f74 0 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/444: dread - d8/de/d13/d40/f90 zero size 2024-03-23T15:44:52.660 INFO:tasks.workunit.client.0.smithi076.stdout:7/445: write d8/de/d13/d24/f55 [619284,38634] 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:7/446: fdatasync d8/f3f 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:5/324: mkdir d1a/d33/d3a/d64/d6c 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:7/447: unlink d8/de/d13/d40/d88/d5a/f8c 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:7/448: chown d8/de/d13/l1f 335 1 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:7/449: mknod d8/de/d13/d18/c97 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:7/450: fsync d8/de/d13/d40/f90 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/195: creat d0/d35/f3a x:0 0 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:7/451: mkdir d8/d4e/d98 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:7/452: mkdir d8/de/d99 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/196: dread d0/f26 [0,4194304] 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/197: symlink d0/d1/d6/d17/l3b 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/198: mkdir d0/d3c 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/199: symlink d0/d1/l3d 0 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/200: rmdir d0/d1/d6/d17 39 2024-03-23T15:44:52.661 INFO:tasks.workunit.client.0.smithi076.stdout:4/397: dwrite d8/d3a/d18/f21 [0,4194304] 0 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:4/398: stat d8/d3a/d18/d52/f6d 0 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:4/399: chown d8/d9/d1d/f4f 189288140 1 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:4/400: chown d8/d3a/d18/f4d 1 1 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:1/201: rmdir d0/d1/d6/d17 39 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:2/399: dwrite d1/d4/db/d15/f89 [0,4194304] 0 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:1/202: creat d0/d1/d6/d17/d20/f3e x:0 0 0 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:2/400: truncate d1/d4/db/f5b 1562697 0 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:7/453: dread d8/fc [0,4194304] 0 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:3/331: sync 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:8/273: sync 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:9/366: sync 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:0/430: sync 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:6/261: sync 2024-03-23T15:44:52.662 INFO:tasks.workunit.client.0.smithi076.stdout:9/367: readlink d1/l2b 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:8/274: chown d8/f1d 15085 1 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:6/262: dread - d1/dd/d2a/d23/d36/f55 zero size 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:2/401: dread d1/d4/d64/d1b/f1c [0,4194304] 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:3/332: creat d5/d17/d1c/d32/d39/f6c x:0 0 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:7/454: rename d8/de/d13/l30 to d8/de/d13/d3a/d82/d8f/l9a 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:0/431: mknod d2/de/d71/c80 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:0/432: chown d2/d1b 42597476 1 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:3/333: mkdir d5/d35/d6d 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:2/402: creat d1/d4/d64/d82/d7f/f8c x:0 0 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:2/403: creat d1/d4/f8d x:0 0 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:2/404: fsync d1/d4/d64/d1b/f44 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:6/263: dread d1/d4/d10/f4a [0,4194304] 0 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:6/264: chown d1/la 165474962 1 2024-03-23T15:44:52.663 INFO:tasks.workunit.client.0.smithi076.stdout:1/203: dread d0/d1/d6/f9 [0,4194304] 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:2/405: mknod d1/d2e/c8e 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:5/325: dwrite f18 [0,4194304] 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:2/406: creat d1/d4/db/d15/d29/f8f x:0 0 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:2/407: mknod d1/d4/d64/d82/c90 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:2/408: write d1/d4/f9 [290939,99476] 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:7/455: write d8/de/d13/d40/d5d/f65 [3044081,81115] 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:4/401: dwrite d8/d9/fc [0,4194304] 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:2/409: creat d1/d4/d52/f91 x:0 0 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:4/402: getdents d8/d3a/d18/d1b 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:5/326: unlink d1a/d39/f61 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:7/456: unlink d8/de/d13/d18/c97 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:7/457: stat d8/de/d13/f14 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:5/327: rmdir d1a/d33/d3a/d64/d6c 0 2024-03-23T15:44:52.664 INFO:tasks.workunit.client.0.smithi076.stdout:5/328: creat d1a/d33/d3a/d67/f6d x:0 0 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:8/275: dwrite d8/fe [0,4194304] 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:8/276: creat d8/d26/d33/f59 x:0 0 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:7/458: unlink d8/de/c10 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:9/368: dwrite d1/d2f/d3a/d53/f61 [0,4194304] 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:3/334: dwrite d5/f14 [0,4194304] 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:2/410: symlink d1/d4/d64/d1b/d32/l92 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:1/204: dwrite d0/f26 [0,4194304] 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:9/369: rmdir d1/d75 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:9/370: readlink d1/d2f/d3a/d65/l6d 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:4/403: dread d8/f42 [0,4194304] 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:3/335: truncate d5/f11 4329096 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:9/371: symlink d1/d1f/d29/l79 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:9/372: write d1/d1f/f30 [5031630,118704] 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:9/373: getdents d1/d20 0 2024-03-23T15:44:52.665 INFO:tasks.workunit.client.0.smithi076.stdout:6/265: rename d1/dd/d2a/d23/d36 to d1/dd/df/d5b 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:4/404: mknod d8/d3a/d69/d73/c7d 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:3/336: creat d5/d35/f6e x:0 0 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:9/374: mknod d1/d2f/d3a/d64/c7a 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:9/375: write d1/d2f/f42 [831757,34796] 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:6/266: creat d1/dd/d2a/f5c x:0 0 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:6/267: truncate d1/d2/f22 1182656 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:6/268: stat d1/dd/df/c49 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:6/269: chown d1/d2/d27/f46 1581506 1 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:6/270: chown d1/dd/d2a/f5c 511 1 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:6/271: fdatasync d1/d4/d10/f34 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:7/459: dwrite d8/fd [0,4194304] 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:3/337: mknod d5/d17/d1c/d32/d39/c6f 0 2024-03-23T15:44:52.666 INFO:tasks.workunit.client.0.smithi076.stdout:7/460: mknod d8/de/d48/d92/c9b 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:8/277: dwrite d8/f57 [0,4194304] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:1/205: dwrite d0/d2/f37 [0,4194304] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:4/405: write d8/d3a/f48 [2779877,89172] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:2/411: dwrite d1/d4/d64/d1d/f73 [0,4194304] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:2/412: mknod d1/d4/db/d3e/c93 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:6/272: mknod d1/c5d 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:6/273: chown d1/dd/d2a/d23/c3b 34910 1 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:9/376: dwrite d1/d1f/d29/d2d/d37/f66 [0,4194304] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:3/338: truncate d5/d17/d1c/d32/d39/f44 488803 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:8/278: read d8/f14 [2331627,32473] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:5/329: dwrite d1a/d33/d3a/d67/f6d [0,4194304] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:7/461: symlink d8/de/l9c 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:7/462: dread - d8/de/d13/d3a/d82/f66 zero size 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:2/413: write d1/d4/db/d15/f21 [3937102,28461] 0 2024-03-23T15:44:52.667 INFO:tasks.workunit.client.0.smithi076.stdout:2/414: truncate d1/d4/f5f 293234 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/206: truncate d0/d2/d1f/f31 1006806 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:2/415: creat d1/d2e/d8a/f94 x:0 0 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:4/406: rmdir d8/d9 39 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:4/407: write d8/d3a/f74 [171391,121321] 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:6/274: truncate d1/dd/d2a/d23/f2d 2502773 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:8/279: dwrite d8/d2e/f4a [0,4194304] 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:3/339: mkdir d5/d17/d1c/d32/d70 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:5/330: mknod d1a/d33/d3a/c6e 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:2/416: dwrite d1/d4/d64/d82/f50 [0,4194304] 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/207: mknod d0/c3f 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:7/463: mknod d8/de/c9d 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:6/275: readlink d1/dd/df/d5b/l1d 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:4/408: unlink d8/d9/d1d/f24 0 2024-03-23T15:44:52.668 INFO:tasks.workunit.client.0.smithi076.stdout:4/409: write d8/d41/f54 [993104,99963] 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:4/410: dread - d8/d3a/d18/d1b/f55 zero size 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:4/411: truncate d8/d9/d32/f5b 1458053 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:7/464: dread d8/de/d13/d40/d88/f57 [0,4194304] 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:6/276: dread d1/d2/d27/f46 [0,4194304] 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:8/280: creat d8/f5a x:0 0 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:8/281: truncate f3 5031822 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:3/340: mknod d5/c71 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/417: dread d1/d4/f1e [0,4194304] 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/418: truncate d1/d4/d64/d82/f51 124429 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:5/331: creat d1a/d2c/f6f x:0 0 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/419: creat d1/d4/d64/f95 x:0 0 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:1/208: mkdir d0/d35/d40 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:1/209: write d0/d35/f3a [259790,85195] 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:1/210: stat d0 0 2024-03-23T15:44:52.669 INFO:tasks.workunit.client.0.smithi076.stdout:1/211: fsync d0/d35/f36 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:4/412: mknod d8/d3a/d18/d75/c7e 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:4/413: write d8/d3a/d3b/f4c [927622,57140] 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:6/277: unlink d1/d4/d10/d39/c56 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:8/282: creat d8/d26/d27/d56/f5b x:0 0 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:3/341: creat d5/d17/d1c/d32/d4a/f72 x:0 0 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:3/342: fdatasync d5/d35/d4c/d5a/f41 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:5/332: truncate f18 21432 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:1/212: symlink d0/d1/d6/d17/d20/l41 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:1/213: dread - d0/d1/d6/d17/d20/f33 zero size 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:1/214: chown d0/l39 1262800037 1 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:8/283: truncate d8/f50 1238649 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:3/343: creat d5/f73 x:0 0 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:6/278: dread d1/f21 [0,4194304] 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:5/333: creat d1a/d39/d65/f70 x:0 0 0 2024-03-23T15:44:52.670 INFO:tasks.workunit.client.0.smithi076.stdout:1/215: symlink d0/d3c/l42 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:4/414: dwrite d8/d41/f54 [0,4194304] 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:4/415: fsync d8/d3a/f48 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:7/465: dwrite d8/de/d13/d3a/f89 [0,4194304] 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:7/466: truncate d8/f62 1017667 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:4/416: dread d8/d9/d1d/f4f [0,4194304] 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:8/284: creat d8/d2e/f5c x:0 0 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:8/285: truncate d8/f13 607258 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:3/344: symlink d5/d17/l74 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:3/345: stat d5/d17/d1c/d32/l50 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:7/467: symlink d8/de/d48/d92/l9e 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:6/279: rename d1/dd/d2a/d23/c3b to d1/c5e 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:7/468: creat d8/d4e/d98/f9f x:0 0 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:5/334: mknod d1a/d20/c71 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:7/469: symlink d8/de/d13/d40/d5d/la0 0 2024-03-23T15:44:52.671 INFO:tasks.workunit.client.0.smithi076.stdout:7/470: chown d8/de/c9d 498491 1 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:4/417: dread d8/d3a/d18/f21 [0,4194304] 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:4/418: read d8/d3a/d3b/f39 [6933970,95804] 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:4/419: write d8/f1a [417867,44753] 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:7/471: truncate d8/de/d13/f94 1455305 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:7/472: chown d8/de/d13 826308 1 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:8/286: rmdir d8/d26 39 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:3/346: creat d5/d17/d1c/d32/d39/f75 x:0 0 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:5/335: rmdir d1a/d39/d65 39 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:5/336: chown d1a/d39/f5b 15136 1 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:6/280: write d1/dd/d2a/d23/f3a [5371406,63312] 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:4/420: creat d8/d3a/d18/d1b/f7f x:0 0 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:3/347: dread d5/d35/f36 [0,4194304] 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:4/421: dread d8/f3d [0,4194304] 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:8/287: mknod d8/d2e/d3e/c5d 0 2024-03-23T15:44:52.672 INFO:tasks.workunit.client.0.smithi076.stdout:8/288: stat d8/l1f 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:8/289: dread d8/d2e/d30/f32 [0,4194304] 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:6/281: stat d1/dd/d2a/c5a 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:6/282: creat d1/dd/df/f5f x:0 0 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:7/473: dwrite d8/fd [4194304,4194304] 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:7/474: fsync d8/de/d13/d40/d5d/f20 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:7/475: truncate d8/fc 917024 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:3/348: dread d5/f56 [0,4194304] 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:3/349: symlink d5/d35/d4c/d5a/d48/l76 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:3/350: truncate d5/d17/d1c/d32/d4a/f55 944060 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:5/337: write d1a/d33/d3a/f41 [3711519,98160] 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:5/338: fsync d1a/d39/d65/f70 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:3/351: link d5/d17/l22 d5/d17/d1c/d31/d61/l77 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:3/352: mknod d5/d35/c78 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:6/283: dread d1/d2/f38 [0,4194304] 0 2024-03-23T15:44:52.673 INFO:tasks.workunit.client.0.smithi076.stdout:7/476: dread f1 [0,4194304] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:5/339: unlink d1a/d39/f66 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/290: dwrite d8/fa [0,4194304] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/291: write d8/f48 [485026,22381] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/292: stat d8/d26/c55 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:4/422: dwrite d8/d9/d1d/f4f [0,4194304] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/293: rename l1 to d8/d2e/d30/l5e 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/294: write d8/d4f/f53 [482633,81316] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/295: write d8/f23 [256829,33783] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/296: dread - d8/d2e/f5c zero size 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:8/297: write d8/f48 [1030003,73188] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:5/340: dwrite d1a/d33/f69 [0,4194304] 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:5/341: fdatasync d1a/f29 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:4/423: mkdir d8/d80 0 2024-03-23T15:44:52.674 INFO:tasks.workunit.client.0.smithi076.stdout:6/284: rename d1/d2/d27/d28 to d1/d2/d27/d60 0 2024-03-23T15:44:52.675 INFO:tasks.workunit.client.0.smithi076.stdout:8/298: creat d8/d2e/d30/d3c/f5f x:0 0 0 2024-03-23T15:44:52.675 INFO:tasks.workunit.client.0.smithi076.stdout:6/285: creat d1/d2/d27/d60/f61 x:0 0 0 2024-03-23T15:44:52.675 INFO:tasks.workunit.client.0.smithi076.stdout:5/342: mknod d1a/d39/d65/c72 0 2024-03-23T15:44:52.675 INFO:tasks.workunit.client.0.smithi076.stdout:8/299: mknod d8/d4f/c60 0 2024-03-23T15:44:52.675 INFO:tasks.workunit.client.0.smithi076.stdout:0/433: sync 2024-03-23T15:44:52.676 INFO:tasks.workunit.client.0.smithi076.stdout:4/424: truncate d8/d9/d32/f5b 856654 0 2024-03-23T15:44:52.676 INFO:tasks.workunit.client.0.smithi076.stdout:0/434: dwrite d2/de/d10/d43/f4c [0,4194304] 0 2024-03-23T15:44:52.678 INFO:tasks.workunit.client.0.smithi076.stdout:4/425: creat d8/d3a/d69/d73/d76/f81 x:0 0 0 2024-03-23T15:44:52.678 INFO:tasks.workunit.client.0.smithi076.stdout:4/426: dread - d8/d9/f5a zero size 2024-03-23T15:44:52.678 INFO:tasks.workunit.client.0.smithi076.stdout:4/427: dread - d8/f5d zero size 2024-03-23T15:44:52.678 INFO:tasks.workunit.client.0.smithi076.stdout:4/428: read - d8/d3a/d18/d1b/f7f zero size 2024-03-23T15:44:52.678 INFO:tasks.workunit.client.0.smithi076.stdout:4/429: truncate d8/d3a/f48 4564693 0 2024-03-23T15:44:52.678 INFO:tasks.workunit.client.0.smithi076.stdout:4/430: write d8/f66 [1118471,89611] 0 2024-03-23T15:44:52.684 INFO:tasks.workunit.client.0.smithi076.stdout:0/435: creat d2/de/d10/d43/f81 x:0 0 0 2024-03-23T15:44:52.691 INFO:tasks.workunit.client.0.smithi076.stdout:4/431: rmdir d8/d3a/d18/d52 39 2024-03-23T15:44:52.695 INFO:tasks.workunit.client.0.smithi076.stdout:4/432: getdents d8/d3a/d18/d52 0 2024-03-23T15:44:52.728 INFO:tasks.workunit.client.0.smithi076.stdout:0/436: mknod d2/de/d10/d43/d56/c82 0 2024-03-23T15:44:52.729 INFO:tasks.workunit.client.0.smithi076.stdout:0/437: mknod d2/d8/d2d/c83 0 2024-03-23T15:44:52.731 INFO:tasks.workunit.client.0.smithi076.stdout:0/438: mkdir d2/de/d10/d3a/d47/d69/d84 0 2024-03-23T15:44:52.731 INFO:tasks.workunit.client.0.smithi076.stdout:0/439: write d2/de/d10/d12/f6a [493818,65661] 0 2024-03-23T15:44:52.737 INFO:tasks.workunit.client.0.smithi076.stdout:0/440: creat d2/de/d10/d12/f85 x:0 0 0 2024-03-23T15:44:52.737 INFO:tasks.workunit.client.0.smithi076.stdout:0/441: chown d2/d1b 201122851 1 2024-03-23T15:44:52.737 INFO:tasks.workunit.client.0.smithi076.stdout:0/442: fdatasync d2/de/d10/d43/f4c 0 2024-03-23T15:44:52.737 INFO:tasks.workunit.client.0.smithi076.stdout:0/443: stat d2/de/d71/c80 0 2024-03-23T15:44:52.747 INFO:tasks.workunit.client.0.smithi076.stdout:6/286: rename d1/d2/d27/d60/d4e to d1/dd/df/d5b/d62 0 2024-03-23T15:44:52.749 INFO:tasks.workunit.client.0.smithi076.stdout:9/377: sync 2024-03-23T15:44:52.749 INFO:tasks.workunit.client.0.smithi076.stdout:8/300: sync 2024-03-23T15:44:52.749 INFO:tasks.workunit.client.0.smithi076.stdout:3/353: sync 2024-03-23T15:44:52.749 INFO:tasks.workunit.client.0.smithi076.stdout:7/477: sync 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:0/444: truncate d2/f9 391231 0 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:1/216: sync 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:2/420: sync 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:0/445: chown d2/de/d71/c80 365 1 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:7/478: truncate d8/de/d13/d3a/f8b 265713 0 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:7/479: stat d8/fc 0 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:5/343: write d1a/d4b/f4c [198191,87741] 0 2024-03-23T15:44:52.750 INFO:tasks.workunit.client.0.smithi076.stdout:5/344: write f19 [1438940,43204] 0 2024-03-23T15:44:52.751 INFO:tasks.workunit.client.0.smithi076.stdout:6/287: truncate d1/dd/f54 533164 0 2024-03-23T15:44:52.751 INFO:tasks.workunit.client.0.smithi076.stdout:1/217: chown d0/d1/d6/d17/c2a 60616621 1 2024-03-23T15:44:52.751 INFO:tasks.workunit.client.0.smithi076.stdout:8/301: dread - d8/d26/d33/f3d zero size 2024-03-23T15:44:52.751 INFO:tasks.workunit.client.0.smithi076.stdout:4/433: rename d8/d9/d1d/l29 to d8/d9/d32/l82 0 2024-03-23T15:44:52.751 INFO:tasks.workunit.client.0.smithi076.stdout:4/434: stat d8/d3a/d18/d1b/f7f 0 2024-03-23T15:44:52.751 INFO:tasks.workunit.client.0.smithi076.stdout:4/435: chown d8/f28 3 1 2024-03-23T15:44:52.751 INFO:tasks.workunit.client.0.smithi076.stdout:4/436: write d8/d9/f5a [972365,130096] 0 2024-03-23T15:44:52.753 INFO:tasks.workunit.client.0.smithi076.stdout:2/421: unlink d1/d4/d64/d82/f51 0 2024-03-23T15:44:52.753 INFO:tasks.workunit.client.0.smithi076.stdout:2/422: dread - d1/d4/d52/d6e/f78 zero size 2024-03-23T15:44:52.753 INFO:tasks.workunit.client.0.smithi076.stdout:2/423: write d1/d4/d64/f95 [177831,58402] 0 2024-03-23T15:44:52.753 INFO:tasks.workunit.client.0.smithi076.stdout:2/424: write d1/d4/d64/f20 [2001775,21835] 0 2024-03-23T15:44:52.753 INFO:tasks.workunit.client.0.smithi076.stdout:1/218: creat d0/d2/d1f/f43 x:0 0 0 2024-03-23T15:44:52.754 INFO:tasks.workunit.client.0.smithi076.stdout:7/480: creat d8/de/d13/d3a/d82/fa1 x:0 0 0 2024-03-23T15:44:52.756 INFO:tasks.workunit.client.0.smithi076.stdout:3/354: mkdir d5/d17/d1c/d32/d39/d59/d79 0 2024-03-23T15:44:52.756 INFO:tasks.workunit.client.0.smithi076.stdout:0/446: rename d2/de/d10/d43/f4c to d2/d1b/f86 0 2024-03-23T15:44:52.757 INFO:tasks.workunit.client.0.smithi076.stdout:2/425: read d1/d4/d64/d1b/f1c [1210350,90265] 0 2024-03-23T15:44:52.759 INFO:tasks.workunit.client.0.smithi076.stdout:7/481: symlink d8/de/d13/d3a/d82/d8f/la2 0 2024-03-23T15:44:52.759 INFO:tasks.workunit.client.0.smithi076.stdout:5/345: dread d1a/d33/d3a/f43 [0,4194304] 0 2024-03-23T15:44:52.759 INFO:tasks.workunit.client.0.smithi076.stdout:7/482: write d8/de/d13/d40/d5d/f6e [917274,82262] 0 2024-03-23T15:44:52.759 INFO:tasks.workunit.client.0.smithi076.stdout:1/219: mknod d0/d2/c44 0 2024-03-23T15:44:52.759 INFO:tasks.workunit.client.0.smithi076.stdout:4/437: write d8/d9/fc [3021306,109053] 0 2024-03-23T15:44:52.759 INFO:tasks.workunit.client.0.smithi076.stdout:5/346: chown d1a/f68 78440 1 2024-03-23T15:44:52.761 INFO:tasks.workunit.client.0.smithi076.stdout:3/355: mknod d5/d17/c7a 0 2024-03-23T15:44:52.761 INFO:tasks.workunit.client.0.smithi076.stdout:3/356: chown d5/d35/d4c/d5a/d48/f62 2994 1 2024-03-23T15:44:52.761 INFO:tasks.workunit.client.0.smithi076.stdout:3/357: chown d5/d17/d1c/d32 215 1 2024-03-23T15:44:52.761 INFO:tasks.workunit.client.0.smithi076.stdout:2/426: creat d1/d4/f96 x:0 0 0 2024-03-23T15:44:52.762 INFO:tasks.workunit.client.0.smithi076.stdout:5/347: creat d1a/d33/f73 x:0 0 0 2024-03-23T15:44:52.762 INFO:tasks.workunit.client.0.smithi076.stdout:4/438: creat d8/d3a/d69/d73/d76/f83 x:0 0 0 2024-03-23T15:44:52.762 INFO:tasks.workunit.client.0.smithi076.stdout:9/378: dread d1/d1f/f30 [4194304,4194304] 0 2024-03-23T15:44:52.762 INFO:tasks.workunit.client.0.smithi076.stdout:7/483: dread d8/de/d13/d40/d5d/f34 [0,4194304] 0 2024-03-23T15:44:52.762 INFO:tasks.workunit.client.0.smithi076.stdout:7/484: chown d8/de/d13/d3a 6979750 1 2024-03-23T15:44:52.763 INFO:tasks.workunit.client.0.smithi076.stdout:5/348: read - d1a/d39/f5b zero size 2024-03-23T15:44:52.763 INFO:tasks.workunit.client.0.smithi076.stdout:0/447: write d2/de/d10/d43/f50 [463526,34900] 0 2024-03-23T15:44:52.764 INFO:tasks.workunit.client.0.smithi076.stdout:2/427: link d1/d4/db/c59 d1/d4/c97 0 2024-03-23T15:44:52.764 INFO:tasks.workunit.client.0.smithi076.stdout:2/428: chown d1/d4/d64/d1b/f44 32650 1 2024-03-23T15:44:52.766 INFO:tasks.workunit.client.0.smithi076.stdout:2/429: read d1/d4/d64/f4e [570090,42195] 0 2024-03-23T15:44:52.767 INFO:tasks.workunit.client.0.smithi076.stdout:9/379: mknod d1/d1f/d29/d2d/c7b 0 2024-03-23T15:44:52.769 INFO:tasks.workunit.client.0.smithi076.stdout:9/380: stat d1/d1f/d46/d4a/l4f 0 2024-03-23T15:44:52.769 INFO:tasks.workunit.client.0.smithi076.stdout:9/381: truncate d1/f48 779404 0 2024-03-23T15:44:52.770 INFO:tasks.workunit.client.0.smithi076.stdout:4/439: unlink d8/f36 0 2024-03-23T15:44:52.770 INFO:tasks.workunit.client.0.smithi076.stdout:2/430: rmdir d1/d4 39 2024-03-23T15:44:52.771 INFO:tasks.workunit.client.0.smithi076.stdout:3/358: dread d5/d17/d1c/f52 [0,4194304] 0 2024-03-23T15:44:52.772 INFO:tasks.workunit.client.0.smithi076.stdout:3/359: write d5/f43 [623773,34135] 0 2024-03-23T15:44:52.772 INFO:tasks.workunit.client.0.smithi076.stdout:2/431: dread - d1/d4/f8d zero size 2024-03-23T15:44:52.773 INFO:tasks.workunit.client.0.smithi076.stdout:5/349: dread d1a/d39/d65/f6b [0,4194304] 0 2024-03-23T15:44:52.773 INFO:tasks.workunit.client.0.smithi076.stdout:8/302: dwrite d8/d26/d27/d56/f5b [0,4194304] 0 2024-03-23T15:44:52.773 INFO:tasks.workunit.client.0.smithi076.stdout:8/303: dread - d8/d26/d33/f3d zero size 2024-03-23T15:44:52.775 INFO:tasks.workunit.client.0.smithi076.stdout:9/382: mkdir d1/d2f/d3a/d53/d7c 0 2024-03-23T15:44:52.776 INFO:tasks.workunit.client.0.smithi076.stdout:5/350: creat d1a/d33/d3a/f74 x:0 0 0 2024-03-23T15:44:52.777 INFO:tasks.workunit.client.0.smithi076.stdout:3/360: write d5/f56 [3259473,78586] 0 2024-03-23T15:44:52.777 INFO:tasks.workunit.client.0.smithi076.stdout:3/361: fsync d5/d35/d4c/d5a/f49 0 2024-03-23T15:44:52.779 INFO:tasks.workunit.client.0.smithi076.stdout:5/351: link d1a/c24 d1a/d33/c75 0 2024-03-23T15:44:52.780 INFO:tasks.workunit.client.0.smithi076.stdout:0/448: dwrite d2/d8/f40 [0,4194304] 0 2024-03-23T15:44:52.782 INFO:tasks.workunit.client.0.smithi076.stdout:4/440: dread d8/f1a [4194304,4194304] 0 2024-03-23T15:44:52.782 INFO:tasks.workunit.client.0.smithi076.stdout:4/441: truncate d8/d9/d1d/f60 4751121 0 2024-03-23T15:44:52.782 INFO:tasks.workunit.client.0.smithi076.stdout:2/432: write d1/f5 [1083478,78036] 0 2024-03-23T15:44:52.783 INFO:tasks.workunit.client.0.smithi076.stdout:3/362: creat d5/d17/d1c/f7b x:0 0 0 2024-03-23T15:44:52.783 INFO:tasks.workunit.client.0.smithi076.stdout:3/363: fdatasync d5/f73 0 2024-03-23T15:44:52.784 INFO:tasks.workunit.client.0.smithi076.stdout:3/364: chown d5/d17/d1c/d31/d61/l77 0 1 2024-03-23T15:44:52.785 INFO:tasks.workunit.client.0.smithi076.stdout:2/433: link d1/d4/db/f68 d1/d4/f98 0 2024-03-23T15:44:52.785 INFO:tasks.workunit.client.0.smithi076.stdout:5/352: creat d1a/f76 x:0 0 0 2024-03-23T15:44:52.786 INFO:tasks.workunit.client.0.smithi076.stdout:5/353: dread - d1a/d33/d3a/f58 zero size 2024-03-23T15:44:52.786 INFO:tasks.workunit.client.0.smithi076.stdout:5/354: stat d1a/d33/d3a/c57 0 2024-03-23T15:44:52.787 INFO:tasks.workunit.client.0.smithi076.stdout:0/449: dread d2/d8/f28 [0,4194304] 0 2024-03-23T15:44:52.787 INFO:tasks.workunit.client.0.smithi076.stdout:0/450: chown d2/de/d10/d43 2677301 1 2024-03-23T15:44:52.788 INFO:tasks.workunit.client.0.smithi076.stdout:0/451: write d2/de/d10/d43/f81 [1004357,87167] 0 2024-03-23T15:44:52.789 INFO:tasks.workunit.client.0.smithi076.stdout:2/434: symlink d1/d4/d52/l99 0 2024-03-23T15:44:52.790 INFO:tasks.workunit.client.0.smithi076.stdout:2/435: creat d1/d2e/d8a/f9a x:0 0 0 2024-03-23T15:44:52.791 INFO:tasks.workunit.client.0.smithi076.stdout:5/355: link d1a/d33/d36/f3d d1a/d39/d65/f77 0 2024-03-23T15:44:52.791 INFO:tasks.workunit.client.0.smithi076.stdout:5/356: dread - d1a/d39/d65/f70 zero size 2024-03-23T15:44:52.792 INFO:tasks.workunit.client.0.smithi076.stdout:3/365: creat d5/d17/d1c/d32/d39/f7c x:0 0 0 2024-03-23T15:44:52.793 INFO:tasks.workunit.client.0.smithi076.stdout:5/357: creat d1a/d4b/f78 x:0 0 0 2024-03-23T15:44:52.794 INFO:tasks.workunit.client.0.smithi076.stdout:3/366: creat d5/d17/d1c/d31/d47/f7d x:0 0 0 2024-03-23T15:44:52.795 INFO:tasks.workunit.client.0.smithi076.stdout:0/452: rmdir d2/de 39 2024-03-23T15:44:52.795 INFO:tasks.workunit.client.0.smithi076.stdout:0/453: readlink d2/de/d71/l75 0 2024-03-23T15:44:52.795 INFO:tasks.workunit.client.0.smithi076.stdout:0/454: fsync d2/f7 0 2024-03-23T15:44:52.796 INFO:tasks.workunit.client.0.smithi076.stdout:4/442: dwrite d8/d3a/d3b/f26 [4194304,4194304] 0 2024-03-23T15:44:52.798 INFO:tasks.workunit.client.0.smithi076.stdout:5/358: mknod d1a/d4b/c79 0 2024-03-23T15:44:52.801 INFO:tasks.workunit.client.0.smithi076.stdout:4/443: mkdir d8/d9/d32/d84 0 2024-03-23T15:44:52.801 INFO:tasks.workunit.client.0.smithi076.stdout:4/444: fsync d8/d3a/d18/f4d 0 2024-03-23T15:44:52.804 INFO:tasks.workunit.client.0.smithi076.stdout:4/445: mkdir d8/d3a/d69/d73/d85 0 2024-03-23T15:44:52.804 INFO:tasks.workunit.client.0.smithi076.stdout:4/446: chown d8/d3a/d18/d52/f79 28 1 2024-03-23T15:44:52.804 INFO:tasks.workunit.client.0.smithi076.stdout:4/447: read - d8/f71 zero size 2024-03-23T15:44:52.806 INFO:tasks.workunit.client.0.smithi076.stdout:4/448: creat d8/d3a/d69/f86 x:0 0 0 2024-03-23T15:44:52.806 INFO:tasks.workunit.client.0.smithi076.stdout:4/449: read d8/d3a/d18/f20 [642420,91304] 0 2024-03-23T15:44:52.806 INFO:tasks.workunit.client.0.smithi076.stdout:5/359: getdents d1a/d4b 0 2024-03-23T15:44:52.807 INFO:tasks.workunit.client.0.smithi076.stdout:0/455: dwrite d2/de/f14 [0,4194304] 0 2024-03-23T15:44:52.807 INFO:tasks.workunit.client.0.smithi076.stdout:0/456: readlink d2/d1b/l55 0 2024-03-23T15:44:52.808 INFO:tasks.workunit.client.0.smithi076.stdout:6/288: rename d1/d2/d27/c57 to d1/dd/c63 0 2024-03-23T15:44:52.809 INFO:tasks.workunit.client.0.smithi076.stdout:5/360: write d1a/d20/f34 [1090897,28423] 0 2024-03-23T15:44:52.810 INFO:tasks.workunit.client.0.smithi076.stdout:7/485: rename d8/de/d13/d40/d88/c16 to d8/de/d13/d40/d88/d5a/ca3 0 2024-03-23T15:44:52.810 INFO:tasks.workunit.client.0.smithi076.stdout:7/486: write d8/de/d13/d40/d5d/f6e [1865627,41606] 0 2024-03-23T15:44:52.810 INFO:tasks.workunit.client.0.smithi076.stdout:6/289: unlink d1/l48 0 2024-03-23T15:44:52.810 INFO:tasks.workunit.client.0.smithi076.stdout:3/367: write d5/f13 [4065813,106620] 0 2024-03-23T15:44:52.811 INFO:tasks.workunit.client.0.smithi076.stdout:7/487: mknod d8/de/d13/d18/ca4 0 2024-03-23T15:44:52.811 INFO:tasks.workunit.client.0.smithi076.stdout:5/361: rmdir d1a/d39/d5f 39 2024-03-23T15:44:52.812 INFO:tasks.workunit.client.0.smithi076.stdout:7/488: truncate d8/de/d13/d40/d5d/d3c/d46/f51 1908442 0 2024-03-23T15:44:52.812 INFO:tasks.workunit.client.0.smithi076.stdout:2/436: dwrite d1/d2e/d8a/f9a [0,4194304] 0 2024-03-23T15:44:52.813 INFO:tasks.workunit.client.0.smithi076.stdout:5/362: creat d1a/d33/d3a/f7a x:0 0 0 2024-03-23T15:44:52.813 INFO:tasks.workunit.client.0.smithi076.stdout:9/383: rename d1/d2f/d3a/d53/l6c to d1/l7d 0 2024-03-23T15:44:52.816 INFO:tasks.workunit.client.0.smithi076.stdout:0/457: write d2/f1d [917247,51445] 0 2024-03-23T15:44:52.816 INFO:tasks.workunit.client.0.smithi076.stdout:0/458: chown d2/de/d10/d12 8063143 1 2024-03-23T15:44:52.817 INFO:tasks.workunit.client.0.smithi076.stdout:5/363: getdents d1a/d33/d3a 0 2024-03-23T15:44:52.817 INFO:tasks.workunit.client.0.smithi076.stdout:7/489: write d8/de/d13/d40/d88/f57 [1415655,52939] 0 2024-03-23T15:44:52.821 INFO:tasks.workunit.client.0.smithi076.stdout:0/459: symlink d2/de/d10/d12/l87 0 2024-03-23T15:44:52.823 INFO:tasks.workunit.client.0.smithi076.stdout:0/460: dread - d2/d8/d32/f48 zero size 2024-03-23T15:44:52.824 INFO:tasks.workunit.client.0.smithi076.stdout:9/384: getdents d1/d1f/d29 0 2024-03-23T15:44:52.826 INFO:tasks.workunit.client.0.smithi076.stdout:9/385: mknod d1/d2f/d3a/d4d/c7e 0 2024-03-23T15:44:52.826 INFO:tasks.workunit.client.0.smithi076.stdout:8/304: link d8/d26/d27/d56/f5b d8/d2e/d30/f61 0 2024-03-23T15:44:52.827 INFO:tasks.workunit.client.0.smithi076.stdout:4/450: dwrite d8/f66 [0,4194304] 0 2024-03-23T15:44:52.827 INFO:tasks.workunit.client.0.smithi076.stdout:4/451: write d8/d3a/d3b/f39 [4108132,96174] 0 2024-03-23T15:44:52.829 INFO:tasks.workunit.client.0.smithi076.stdout:8/305: dread d8/f50 [0,4194304] 0 2024-03-23T15:44:52.829 INFO:tasks.workunit.client.0.smithi076.stdout:8/306: chown f3 43894932 1 2024-03-23T15:44:52.831 INFO:tasks.workunit.client.0.smithi076.stdout:2/437: dwrite d1/d2e/f3f [0,4194304] 0 2024-03-23T15:44:52.831 INFO:tasks.workunit.client.0.smithi076.stdout:2/438: fdatasync d1/d4/db/d15/f21 0 2024-03-23T15:44:52.831 INFO:tasks.workunit.client.0.smithi076.stdout:5/364: rename d1a/d20/l26 to d1a/l7b 0 2024-03-23T15:44:52.831 INFO:tasks.workunit.client.0.smithi076.stdout:6/290: dwrite d1/dd/f54 [0,4194304] 0 2024-03-23T15:44:52.831 INFO:tasks.workunit.client.0.smithi076.stdout:3/368: rmdir d5/d17/d1c/d32/d4a 39 2024-03-23T15:44:52.833 INFO:tasks.workunit.client.0.smithi076.stdout:7/490: rename d8/de/d13/d18/ca4 to d8/de/d13/d24/d91/d96/ca5 0 2024-03-23T15:44:52.835 INFO:tasks.workunit.client.0.smithi076.stdout:4/452: truncate d8/f3f 2023128 0 2024-03-23T15:44:52.836 INFO:tasks.workunit.client.0.smithi076.stdout:7/491: readlink d8/l9 0 2024-03-23T15:44:52.836 INFO:tasks.workunit.client.0.smithi076.stdout:5/365: mkdir d1a/d39/d7c 0 2024-03-23T15:44:52.836 INFO:tasks.workunit.client.0.smithi076.stdout:5/366: chown d1a/d33/f4d 3 1 2024-03-23T15:44:52.838 INFO:tasks.workunit.client.0.smithi076.stdout:9/386: mkdir d1/d1f/d29/d7f 0 2024-03-23T15:44:52.838 INFO:tasks.workunit.client.0.smithi076.stdout:8/307: mknod d8/c62 0 2024-03-23T15:44:52.841 INFO:tasks.workunit.client.0.smithi076.stdout:6/291: write d1/dd/df/d5b/f3e [699114,2580] 0 2024-03-23T15:44:52.841 INFO:tasks.workunit.client.0.smithi076.stdout:0/461: dwrite d2/d8/d52/f64 [0,4194304] 0 2024-03-23T15:44:52.842 INFO:tasks.workunit.client.0.smithi076.stdout:9/387: truncate d1/d1f/d29/f40 3439304 0 2024-03-23T15:44:52.844 INFO:tasks.workunit.client.0.smithi076.stdout:0/462: dread d2/de/f13 [0,4194304] 0 2024-03-23T15:44:52.844 INFO:tasks.workunit.client.0.smithi076.stdout:0/463: fdatasync d2/d8/f65 0 2024-03-23T15:44:52.844 INFO:tasks.workunit.client.0.smithi076.stdout:0/464: write d2/de/d10/d12/f85 [260570,91686] 0 2024-03-23T15:44:52.844 INFO:tasks.workunit.client.0.smithi076.stdout:0/465: fsync d2/de/d10/d3a/d47/f4e 0 2024-03-23T15:44:52.845 INFO:tasks.workunit.client.0.smithi076.stdout:5/367: getdents d1a/d39 0 2024-03-23T15:44:52.845 INFO:tasks.workunit.client.0.smithi076.stdout:5/368: fdatasync d1a/f76 0 2024-03-23T15:44:52.846 INFO:tasks.workunit.client.0.smithi076.stdout:3/369: mknod d5/d35/d4c/d5a/d48/c7e 0 2024-03-23T15:44:52.846 INFO:tasks.workunit.client.0.smithi076.stdout:5/369: chown d1a/c59 22505 1 2024-03-23T15:44:52.846 INFO:tasks.workunit.client.0.smithi076.stdout:5/370: chown d1a/d33/d3a/d67/f6d 6226 1 2024-03-23T15:44:52.847 INFO:tasks.workunit.client.0.smithi076.stdout:4/453: mknod d8/d80/c87 0 2024-03-23T15:44:52.849 INFO:tasks.workunit.client.0.smithi076.stdout:2/439: dwrite d1/d4/d64/d82/f50 [0,4194304] 0 2024-03-23T15:44:52.849 INFO:tasks.workunit.client.0.smithi076.stdout:2/440: write d1/d2e/d8a/f94 [865026,97435] 0 2024-03-23T15:44:52.849 INFO:tasks.workunit.client.0.smithi076.stdout:5/371: read f18 [18045,78072] 0 2024-03-23T15:44:52.850 INFO:tasks.workunit.client.0.smithi076.stdout:6/292: mknod d1/d4/d10/c64 0 2024-03-23T15:44:52.851 INFO:tasks.workunit.client.0.smithi076.stdout:5/372: dread - d1a/f25 zero size 2024-03-23T15:44:52.852 INFO:tasks.workunit.client.0.smithi076.stdout:5/373: write d1a/d33/f54 [863822,78331] 0 2024-03-23T15:44:52.855 INFO:tasks.workunit.client.0.smithi076.stdout:5/374: rename d1a/d33/d36/c49 to d1a/d20/c7d 0 2024-03-23T15:44:52.855 INFO:tasks.workunit.client.0.smithi076.stdout:5/375: fsync d1a/d39/d65/f70 0 2024-03-23T15:44:52.855 INFO:tasks.workunit.client.0.smithi076.stdout:5/376: fdatasync d1a/d4b/f50 0 2024-03-23T15:44:52.857 INFO:tasks.workunit.client.0.smithi076.stdout:0/466: creat d2/de/d10/d43/f88 x:0 0 0 2024-03-23T15:44:52.857 INFO:tasks.workunit.client.0.smithi076.stdout:1/220: sync 2024-03-23T15:44:52.857 INFO:tasks.workunit.client.0.smithi076.stdout:0/467: chown d2/de/d10/d43/d7f 61764365 1 2024-03-23T15:44:52.857 INFO:tasks.workunit.client.0.smithi076.stdout:3/370: mkdir d5/d35/d6d/d7f 0 2024-03-23T15:44:52.858 INFO:tasks.workunit.client.0.smithi076.stdout:0/468: write d2/de/d10/d3a/d47/f4e [1817789,68904] 0 2024-03-23T15:44:52.858 INFO:tasks.workunit.client.0.smithi076.stdout:0/469: dread - d2/de/d10/f68 zero size 2024-03-23T15:44:52.858 INFO:tasks.workunit.client.0.smithi076.stdout:0/470: read - d2/de/d10/d3a/f5c zero size 2024-03-23T15:44:52.860 INFO:tasks.workunit.client.0.smithi076.stdout:3/371: truncate d5/f56 472685 0 2024-03-23T15:44:52.860 INFO:tasks.workunit.client.0.smithi076.stdout:3/372: dread - d5/d17/d1c/d32/d39/d59/f60 zero size 2024-03-23T15:44:52.863 INFO:tasks.workunit.client.0.smithi076.stdout:0/471: mknod d2/de/d10/d43/c89 0 2024-03-23T15:44:52.863 INFO:tasks.workunit.client.0.smithi076.stdout:0/472: readlink d2/d1b/l55 0 2024-03-23T15:44:52.865 INFO:tasks.workunit.client.0.smithi076.stdout:1/221: read d0/f26 [4668542,82093] 0 2024-03-23T15:44:52.865 INFO:tasks.workunit.client.0.smithi076.stdout:1/222: write d0/d2/f37 [4776171,63249] 0 2024-03-23T15:44:52.865 INFO:tasks.workunit.client.0.smithi076.stdout:1/223: dread - d0/d1/d6/d17/d20/f3e zero size 2024-03-23T15:44:52.865 INFO:tasks.workunit.client.0.smithi076.stdout:1/224: dread - d0/d1/d6/d17/f38 zero size 2024-03-23T15:44:52.865 INFO:tasks.workunit.client.0.smithi076.stdout:3/373: truncate d5/f6 764431 0 2024-03-23T15:44:52.866 INFO:tasks.workunit.client.0.smithi076.stdout:7/492: symlink d8/de/la6 0 2024-03-23T15:44:52.867 INFO:tasks.workunit.client.0.smithi076.stdout:3/374: write d5/d17/d1c/d32/d39/f75 [699005,109708] 0 2024-03-23T15:44:52.867 INFO:tasks.workunit.client.0.smithi076.stdout:1/225: mknod d0/d3c/c45 0 2024-03-23T15:44:52.868 INFO:tasks.workunit.client.0.smithi076.stdout:7/493: symlink d8/de/d48/la7 0 2024-03-23T15:44:52.870 INFO:tasks.workunit.client.0.smithi076.stdout:1/226: creat d0/d1/d6/d17/d20/f46 x:0 0 0 2024-03-23T15:44:52.870 INFO:tasks.workunit.client.0.smithi076.stdout:1/227: write d0/d1/d6/d17/d20/f3e [593597,63657] 0 2024-03-23T15:44:52.870 INFO:tasks.workunit.client.0.smithi076.stdout:1/228: stat d0 0 2024-03-23T15:44:52.871 INFO:tasks.workunit.client.0.smithi076.stdout:7/494: symlink d8/de/d13/d40/d5d/la8 0 2024-03-23T15:44:52.871 INFO:tasks.workunit.client.0.smithi076.stdout:7/495: truncate d8/de/d13/d3a/f8b 607507 0 2024-03-23T15:44:52.871 INFO:tasks.workunit.client.0.smithi076.stdout:7/496: fsync d8/de/d13/d40/d5d/d3c/f73 0 2024-03-23T15:44:52.871 INFO:tasks.workunit.client.0.smithi076.stdout:7/497: write f1 [4464082,65729] 0 2024-03-23T15:44:52.871 INFO:tasks.workunit.client.0.smithi076.stdout:0/473: dread d2/de/d10/d3a/d47/f59 [4194304,4194304] 0 2024-03-23T15:44:52.872 INFO:tasks.workunit.client.0.smithi076.stdout:9/388: dwrite d1/d20/f45 [4194304,4194304] 0 2024-03-23T15:44:52.872 INFO:tasks.workunit.client.0.smithi076.stdout:7/498: symlink d8/de/d13/d3a/d82/la9 0 2024-03-23T15:44:52.873 INFO:tasks.workunit.client.0.smithi076.stdout:3/375: link d5/d17/l58 d5/d17/d1c/d32/d5d/l80 0 2024-03-23T15:44:52.873 INFO:tasks.workunit.client.0.smithi076.stdout:3/376: readlink d5/d35/d4c/d5a/l40 0 2024-03-23T15:44:52.877 INFO:tasks.workunit.client.0.smithi076.stdout:2/441: dwrite d1/d4/d64/f4e [0,4194304] 0 2024-03-23T15:44:52.878 INFO:tasks.workunit.client.0.smithi076.stdout:3/377: unlink d5/d35/d4c/d5a/d48/f62 0 2024-03-23T15:44:52.878 INFO:tasks.workunit.client.0.smithi076.stdout:0/474: rmdir d2/de/d10 39 2024-03-23T15:44:52.878 INFO:tasks.workunit.client.0.smithi076.stdout:4/454: dwrite d8/f49 [4194304,4194304] 0 2024-03-23T15:44:52.879 INFO:tasks.workunit.client.0.smithi076.stdout:4/455: truncate d8/d3a/d3b/f25 187926 0 2024-03-23T15:44:52.879 INFO:tasks.workunit.client.0.smithi076.stdout:6/293: dwrite d1/d2/d27/f46 [0,4194304] 0 2024-03-23T15:44:52.884 INFO:tasks.workunit.client.0.smithi076.stdout:4/456: dread d8/f30 [0,4194304] 0 2024-03-23T15:44:52.884 INFO:tasks.workunit.client.0.smithi076.stdout:4/457: chown d8/d3a/d18/d52 12643 1 2024-03-23T15:44:52.886 INFO:tasks.workunit.client.0.smithi076.stdout:6/294: read d1/f11 [2956284,3122] 0 2024-03-23T15:44:52.889 INFO:tasks.workunit.client.0.smithi076.stdout:5/377: dwrite d1a/d33/d3a/d67/f6d [0,4194304] 0 2024-03-23T15:44:52.903 INFO:tasks.workunit.client.0.smithi076.stdout:3/378: write d5/d17/f29 [2939038,14447] 0 2024-03-23T15:44:52.903 INFO:tasks.workunit.client.0.smithi076.stdout:4/458: dwrite d8/f28 [0,4194304] 0 2024-03-23T15:44:52.903 INFO:tasks.workunit.client.0.smithi076.stdout:0/475: write d2/de/f1a [337075,82107] 0 2024-03-23T15:44:52.903 INFO:tasks.workunit.client.0.smithi076.stdout:3/379: creat d5/d35/d4c/d5a/f81 x:0 0 0 2024-03-23T15:44:52.903 INFO:tasks.workunit.client.0.smithi076.stdout:4/459: mkdir d8/d3a/d69/d73/d85/d88 0 2024-03-23T15:44:52.903 INFO:tasks.workunit.client.0.smithi076.stdout:9/389: dwrite d1/f18 [0,4194304] 0 2024-03-23T15:44:52.903 INFO:tasks.workunit.client.0.smithi076.stdout:5/378: dread d1a/d33/f3b [0,4194304] 0 2024-03-23T15:44:52.908 INFO:tasks.workunit.client.0.smithi076.stdout:1/229: dwrite d0/d35/f36 [0,4194304] 0 2024-03-23T15:44:52.908 INFO:tasks.workunit.client.0.smithi076.stdout:3/380: dread d5/d17/d1c/d32/d39/f44 [0,4194304] 0 2024-03-23T15:44:52.912 INFO:tasks.workunit.client.0.smithi076.stdout:5/379: dread d1a/f31 [0,4194304] 0 2024-03-23T15:44:52.915 INFO:tasks.workunit.client.0.smithi076.stdout:6/295: rename d1/dd/df/l4c to d1/d4/l65 0 2024-03-23T15:44:52.918 INFO:tasks.workunit.client.0.smithi076.stdout:6/296: creat d1/dd/df/f66 x:0 0 0 2024-03-23T15:44:52.920 INFO:tasks.workunit.client.0.smithi076.stdout:5/380: rename d1a/d39/d7c to d1a/d7e 0 2024-03-23T15:44:52.921 INFO:tasks.workunit.client.0.smithi076.stdout:1/230: dread d0/f2e [4194304,4194304] 0 2024-03-23T15:44:52.921 INFO:tasks.workunit.client.0.smithi076.stdout:1/231: write d0/d1/d6/d17/f2c [4963344,68703] 0 2024-03-23T15:44:52.926 INFO:tasks.workunit.client.0.smithi076.stdout:6/297: dread d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:52.926 INFO:tasks.workunit.client.0.smithi076.stdout:6/298: chown d1/d4/d10/c3d 288159281 1 2024-03-23T15:44:52.926 INFO:tasks.workunit.client.0.smithi076.stdout:0/476: dwrite d2/d8/d52/f64 [4194304,4194304] 0 2024-03-23T15:44:52.926 INFO:tasks.workunit.client.0.smithi076.stdout:9/390: dwrite d1/d1f/d29/d2d/d37/f66 [0,4194304] 0 2024-03-23T15:44:52.930 INFO:tasks.workunit.client.0.smithi076.stdout:0/477: rmdir d2/de/d10/d43/d7f 0 2024-03-23T15:44:52.931 INFO:tasks.workunit.client.0.smithi076.stdout:9/391: creat d1/d1f/d46/f80 x:0 0 0 2024-03-23T15:44:52.932 INFO:tasks.workunit.client.0.smithi076.stdout:4/460: dwrite d8/d41/f54 [0,4194304] 0 2024-03-23T15:44:52.932 INFO:tasks.workunit.client.0.smithi076.stdout:2/442: dwrite d1/d4/db/f68 [4194304,4194304] 0 2024-03-23T15:44:52.932 INFO:tasks.workunit.client.0.smithi076.stdout:4/461: write d8/f5d [443008,82704] 0 2024-03-23T15:44:52.932 INFO:tasks.workunit.client.0.smithi076.stdout:2/443: write d1/d4/d64/f95 [688333,123653] 0 2024-03-23T15:44:52.932 INFO:tasks.workunit.client.0.smithi076.stdout:2/444: write d1/d2e/d6a/f75 [933736,42936] 0 2024-03-23T15:44:52.932 INFO:tasks.workunit.client.0.smithi076.stdout:2/445: read d1/f14 [339464,127056] 0 2024-03-23T15:44:52.932 INFO:tasks.workunit.client.0.smithi076.stdout:0/478: truncate d2/d8/d32/f4a 495617 0 2024-03-23T15:44:52.934 INFO:tasks.workunit.client.0.smithi076.stdout:0/479: getdents d2/d8/d52 0 2024-03-23T15:44:52.935 INFO:tasks.workunit.client.0.smithi076.stdout:9/392: mknod d1/d2f/d3f/c81 0 2024-03-23T15:44:52.938 INFO:tasks.workunit.client.0.smithi076.stdout:0/480: creat d2/de/d10/d3a/f8a x:0 0 0 2024-03-23T15:44:52.938 INFO:tasks.workunit.client.0.smithi076.stdout:0/481: stat d2/de/d10/d12 0 2024-03-23T15:44:52.938 INFO:tasks.workunit.client.0.smithi076.stdout:0/482: dread - d2/de/d10/f5d zero size 2024-03-23T15:44:52.938 INFO:tasks.workunit.client.0.smithi076.stdout:0/483: chown d2/de 18 1 2024-03-23T15:44:52.939 INFO:tasks.workunit.client.0.smithi076.stdout:1/232: rename d0/d1/d6 to d0/d35/d40/d47 0 2024-03-23T15:44:52.941 INFO:tasks.workunit.client.0.smithi076.stdout:9/393: mkdir d1/d1f/d82 0 2024-03-23T15:44:52.943 INFO:tasks.workunit.client.0.smithi076.stdout:0/484: symlink d2/d1b/l8b 0 2024-03-23T15:44:52.944 INFO:tasks.workunit.client.0.smithi076.stdout:1/233: truncate d0/d35/f3a 148936 0 2024-03-23T15:44:52.944 INFO:tasks.workunit.client.0.smithi076.stdout:1/234: fsync d0/d35/f36 0 2024-03-23T15:44:52.944 INFO:tasks.workunit.client.0.smithi076.stdout:1/235: dread - d0/f34 zero size 2024-03-23T15:44:52.944 INFO:tasks.workunit.client.0.smithi076.stdout:1/236: readlink d0/d35/d40/d47/d17/l3b 0 2024-03-23T15:44:52.945 INFO:tasks.workunit.client.0.smithi076.stdout:6/299: dwrite d1/dd/f26 [4194304,4194304] 0 2024-03-23T15:44:52.946 INFO:tasks.workunit.client.0.smithi076.stdout:6/300: mknod d1/d4/d10/d39/c67 0 2024-03-23T15:44:52.946 INFO:tasks.workunit.client.0.smithi076.stdout:4/462: dwrite d8/d3a/d3b/f26 [4194304,4194304] 0 2024-03-23T15:44:52.949 INFO:tasks.workunit.client.0.smithi076.stdout:4/463: rmdir d8/d3a/d18/d1b 39 2024-03-23T15:44:52.949 INFO:tasks.workunit.client.0.smithi076.stdout:4/464: chown d8/d3a/c35 7 1 2024-03-23T15:44:52.949 INFO:tasks.workunit.client.0.smithi076.stdout:4/465: stat d8/d3a/d18/d1b/f67 0 2024-03-23T15:44:52.950 INFO:tasks.workunit.client.0.smithi076.stdout:4/466: creat d8/d3a/d18/f89 x:0 0 0 2024-03-23T15:44:52.952 INFO:tasks.workunit.client.0.smithi076.stdout:6/301: dread d1/f2c [0,4194304] 0 2024-03-23T15:44:52.953 INFO:tasks.workunit.client.0.smithi076.stdout:4/467: creat d8/d80/f8a x:0 0 0 2024-03-23T15:44:52.956 INFO:tasks.workunit.client.0.smithi076.stdout:1/237: dwrite d0/d35/d40/d47/d17/f2c [0,4194304] 0 2024-03-23T15:44:52.959 INFO:tasks.workunit.client.0.smithi076.stdout:1/238: creat d0/d2/f48 x:0 0 0 2024-03-23T15:44:52.959 INFO:tasks.workunit.client.0.smithi076.stdout:4/468: getdents d8/d3a/d18 0 2024-03-23T15:44:52.960 INFO:tasks.workunit.client.0.smithi076.stdout:1/239: creat d0/d35/d40/d47/d17/f49 x:0 0 0 2024-03-23T15:44:52.960 INFO:tasks.workunit.client.0.smithi076.stdout:1/240: chown d0/d2/d1f/f43 311532 1 2024-03-23T15:44:52.961 INFO:tasks.workunit.client.0.smithi076.stdout:6/302: creat d1/dd/d2a/d23/f68 x:0 0 0 2024-03-23T15:44:52.961 INFO:tasks.workunit.client.0.smithi076.stdout:4/469: mknod d8/d3a/d18/d52/c8b 0 2024-03-23T15:44:52.962 INFO:tasks.workunit.client.0.smithi076.stdout:4/470: write d8/f49 [8845905,73939] 0 2024-03-23T15:44:52.965 INFO:tasks.workunit.client.0.smithi076.stdout:4/471: dread d8/d9/d1d/f60 [0,4194304] 0 2024-03-23T15:44:52.965 INFO:tasks.workunit.client.0.smithi076.stdout:4/472: chown d8/d3a/d18/d1b/l5f 2984 1 2024-03-23T15:44:52.965 INFO:tasks.workunit.client.0.smithi076.stdout:4/473: chown d8/d3a/d69/d73/d76/f83 208025825 1 2024-03-23T15:44:52.978 INFO:tasks.workunit.client.0.smithi076.stdout:6/303: dwrite d1/d4/d10/f34 [0,4194304] 0 2024-03-23T15:44:52.978 INFO:tasks.workunit.client.0.smithi076.stdout:6/304: stat d1/dd/df/c49 0 2024-03-23T15:44:52.982 INFO:tasks.workunit.client.0.smithi076.stdout:9/394: rename d1/d2f/d3a/d4d to d1/d1f/d46/d83 0 2024-03-23T15:44:52.982 INFO:tasks.workunit.client.0.smithi076.stdout:9/395: dread - d1/f5b zero size 2024-03-23T15:44:52.983 INFO:tasks.workunit.client.0.smithi076.stdout:1/241: link d0/d1/c5 d0/d35/d40/d47/d17/d20/c4a 0 2024-03-23T15:44:52.984 INFO:tasks.workunit.client.0.smithi076.stdout:4/474: dwrite d8/d9/d1d/f4f [0,4194304] 0 2024-03-23T15:44:52.984 INFO:tasks.workunit.client.0.smithi076.stdout:4/475: readlink d8/l68 0 2024-03-23T15:44:52.984 INFO:tasks.workunit.client.0.smithi076.stdout:0/485: rename d2/d8/f40 to d2/de/d10/f8c 0 2024-03-23T15:44:52.986 INFO:tasks.workunit.client.0.smithi076.stdout:1/242: creat d0/f4b x:0 0 0 2024-03-23T15:44:52.989 INFO:tasks.workunit.client.0.smithi076.stdout:4/476: rename d8/d3a/d69/d73/c7b to d8/d41/c8c 0 2024-03-23T15:44:52.991 INFO:tasks.workunit.client.0.smithi076.stdout:0/486: getdents d2/d1b 0 2024-03-23T15:44:52.991 INFO:tasks.workunit.client.0.smithi076.stdout:0/487: truncate d2/de/d10/f5d 312400 0 2024-03-23T15:44:52.992 INFO:tasks.workunit.client.0.smithi076.stdout:0/488: chown d2/de/d10/d43/d56/l74 31036 1 2024-03-23T15:44:52.999 INFO:tasks.workunit.client.0.smithi076.stdout:6/305: dwrite d1/d2/f22 [0,4194304] 0 2024-03-23T15:44:52.999 INFO:tasks.workunit.client.0.smithi076.stdout:9/396: write d1/d1f/f72 [3383197,80041] 0 2024-03-23T15:44:53.001 INFO:tasks.workunit.client.0.smithi076.stdout:6/306: truncate d1/d2/f1c 70674 0 2024-03-23T15:44:53.001 INFO:tasks.workunit.client.0.smithi076.stdout:9/397: link d1/d20/f45 d1/d2f/d3a/f84 0 2024-03-23T15:44:53.001 INFO:tasks.workunit.client.0.smithi076.stdout:4/477: dwrite d8/d3a/d3b/f26 [4194304,4194304] 0 2024-03-23T15:44:53.004 INFO:tasks.workunit.client.0.smithi076.stdout:4/478: mkdir d8/d3a/d18/d75/d8d 0 2024-03-23T15:44:53.004 INFO:tasks.workunit.client.0.smithi076.stdout:4/479: chown d8/d3a/d18/f20 79 1 2024-03-23T15:44:53.017 INFO:tasks.workunit.client.0.smithi076.stdout:4/480: dread d8/f28 [0,4194304] 0 2024-03-23T15:44:53.018 INFO:tasks.workunit.client.0.smithi076.stdout:6/307: dwrite d1/d2/f1c [0,4194304] 0 2024-03-23T15:44:53.018 INFO:tasks.workunit.client.0.smithi076.stdout:4/481: mknod d8/d9/d32/c8e 0 2024-03-23T15:44:53.019 INFO:tasks.workunit.client.0.smithi076.stdout:8/308: sync 2024-03-23T15:44:53.019 INFO:tasks.workunit.client.0.smithi076.stdout:0/489: rename d2/de/d10/d43/d56/l73 to d2/d8/d2d/l8d 0 2024-03-23T15:44:53.019 INFO:tasks.workunit.client.0.smithi076.stdout:8/309: chown d8/c62 322860919 1 2024-03-23T15:44:53.020 INFO:tasks.workunit.client.0.smithi076.stdout:8/310: fsync d8/fe 0 2024-03-23T15:44:53.020 INFO:tasks.workunit.client.0.smithi076.stdout:8/311: write d8/d4f/f53 [1225796,118712] 0 2024-03-23T15:44:53.022 INFO:tasks.workunit.client.0.smithi076.stdout:8/312: rmdir d8/d2e/d30/d3c 39 2024-03-23T15:44:53.024 INFO:tasks.workunit.client.0.smithi076.stdout:8/313: creat d8/d26/d27/d56/f63 x:0 0 0 2024-03-23T15:44:53.028 INFO:tasks.workunit.client.0.smithi076.stdout:8/314: dread f7 [0,4194304] 0 2024-03-23T15:44:53.036 INFO:tasks.workunit.client.0.smithi076.stdout:4/482: dwrite d8/f42 [4194304,4194304] 0 2024-03-23T15:44:53.036 INFO:tasks.workunit.client.0.smithi076.stdout:4/483: chown d8/d80/c87 84307410 1 2024-03-23T15:44:53.036 INFO:tasks.workunit.client.0.smithi076.stdout:6/308: link d1/d4/d10/d39/l53 d1/dd/df/d5b/d62/l69 0 2024-03-23T15:44:53.037 INFO:tasks.workunit.client.0.smithi076.stdout:6/309: getdents d1/d2/d27/d60 0 2024-03-23T15:44:53.037 INFO:tasks.workunit.client.0.smithi076.stdout:9/398: rename d1/d1f/d29/f40 to d1/d2f/d3a/d64/f85 0 2024-03-23T15:44:53.038 INFO:tasks.workunit.client.0.smithi076.stdout:6/310: creat d1/d2/d27/f6a x:0 0 0 2024-03-23T15:44:53.040 INFO:tasks.workunit.client.0.smithi076.stdout:6/311: getdents d1 0 2024-03-23T15:44:53.041 INFO:tasks.workunit.client.0.smithi076.stdout:6/312: mkdir d1/d4/d10/d6b 0 2024-03-23T15:44:53.044 INFO:tasks.workunit.client.0.smithi076.stdout:9/399: dread d1/d20/f25 [4194304,4194304] 0 2024-03-23T15:44:53.046 INFO:tasks.workunit.client.0.smithi076.stdout:6/313: getdents d1/dd/df/d5b/d62 0 2024-03-23T15:44:53.047 INFO:tasks.workunit.client.0.smithi076.stdout:8/315: dwrite d8/d2e/d30/d3c/f5f [0,4194304] 0 2024-03-23T15:44:53.048 INFO:tasks.workunit.client.0.smithi076.stdout:6/314: mknod d1/dd/df/c6c 0 2024-03-23T15:44:53.048 INFO:tasks.workunit.client.0.smithi076.stdout:6/315: truncate d1/dd/d2a/f5c 505083 0 2024-03-23T15:44:53.062 INFO:tasks.workunit.client.0.smithi076.stdout:6/316: dwrite d1/dd/d2a/d23/f2d [0,4194304] 0 2024-03-23T15:44:53.064 INFO:tasks.workunit.client.0.smithi076.stdout:6/317: link d1/d2/d27/f6a d1/d2/f6d 0 2024-03-23T15:44:53.065 INFO:tasks.workunit.client.0.smithi076.stdout:9/400: rename d1/f5b to d1/d1f/d29/d2d/f86 0 2024-03-23T15:44:53.065 INFO:tasks.workunit.client.0.smithi076.stdout:9/401: chown d1/d1f/d46/d83/f73 9375 1 2024-03-23T15:44:53.067 INFO:tasks.workunit.client.0.smithi076.stdout:8/316: dwrite d8/fa [0,4194304] 0 2024-03-23T15:44:53.069 INFO:tasks.workunit.client.0.smithi076.stdout:8/317: truncate d8/ff 20703610 0 2024-03-23T15:44:53.070 INFO:tasks.workunit.client.0.smithi076.stdout:8/318: write f7 [3359870,20352] 0 2024-03-23T15:44:53.078 INFO:tasks.workunit.client.0.smithi076.stdout:8/319: dread d8/d2e/f4a [0,4194304] 0 2024-03-23T15:44:53.078 INFO:tasks.workunit.client.0.smithi076.stdout:8/320: truncate f3 3857881 0 2024-03-23T15:44:53.080 INFO:tasks.workunit.client.0.smithi076.stdout:6/318: rename d1/dd/df/d5b/f59 to d1/dd/df/d5b/d62/f6e 0 2024-03-23T15:44:53.081 INFO:tasks.workunit.client.0.smithi076.stdout:8/321: mkdir d8/d2e/d30/d64 0 2024-03-23T15:44:53.081 INFO:tasks.workunit.client.0.smithi076.stdout:8/322: chown d8/d2e/c2f 169 1 2024-03-23T15:44:53.083 INFO:tasks.workunit.client.0.smithi076.stdout:9/402: rename d1/d2f/d3a/d53/c5c to d1/d2f/c87 0 2024-03-23T15:44:53.085 INFO:tasks.workunit.client.0.smithi076.stdout:8/323: mknod d8/d4f/c65 0 2024-03-23T15:44:53.085 INFO:tasks.workunit.client.0.smithi076.stdout:8/324: truncate d8/f50 1839860 0 2024-03-23T15:44:53.087 INFO:tasks.workunit.client.0.smithi076.stdout:8/325: creat d8/d2e/d30/f66 x:0 0 0 2024-03-23T15:44:53.087 INFO:tasks.workunit.client.0.smithi076.stdout:6/319: write d1/d4/d10/f34 [3786775,72414] 0 2024-03-23T15:44:53.089 INFO:tasks.workunit.client.0.smithi076.stdout:8/326: rename d8/ff to d8/f67 0 2024-03-23T15:44:53.103 INFO:tasks.workunit.client.0.smithi076.stdout:9/403: dwrite d1/d2f/d3a/d64/f69 [0,4194304] 0 2024-03-23T15:44:53.106 INFO:tasks.workunit.client.0.smithi076.stdout:8/327: dwrite d8/f18 [4194304,4194304] 0 2024-03-23T15:44:53.107 INFO:tasks.workunit.client.0.smithi076.stdout:6/320: dwrite d1/d2/d27/f6a [0,4194304] 0 2024-03-23T15:44:53.107 INFO:tasks.workunit.client.0.smithi076.stdout:6/321: write d1/dd/d2a/d23/f68 [176870,78215] 0 2024-03-23T15:44:53.108 INFO:tasks.workunit.client.0.smithi076.stdout:9/404: mkdir d1/d1f/d82/d88 0 2024-03-23T15:44:53.111 INFO:tasks.workunit.client.0.smithi076.stdout:9/405: mknod d1/d2f/d3a/d74/c89 0 2024-03-23T15:44:53.111 INFO:tasks.workunit.client.0.smithi076.stdout:8/328: creat d8/d2e/d30/f68 x:0 0 0 2024-03-23T15:44:53.112 INFO:tasks.workunit.client.0.smithi076.stdout:9/406: dread d1/f26 [0,4194304] 0 2024-03-23T15:44:53.112 INFO:tasks.workunit.client.0.smithi076.stdout:9/407: write d1/d1f/f22 [697690,7184] 0 2024-03-23T15:44:53.113 INFO:tasks.workunit.client.0.smithi076.stdout:9/408: symlink d1/d2f/d3a/d74/l8a 0 2024-03-23T15:44:53.113 INFO:tasks.workunit.client.0.smithi076.stdout:9/409: truncate d1/d1f/d46/f80 707283 0 2024-03-23T15:44:53.114 INFO:tasks.workunit.client.0.smithi076.stdout:9/410: truncate d1/d1f/d29/d2d/d37/f66 4682606 0 2024-03-23T15:44:53.114 INFO:tasks.workunit.client.0.smithi076.stdout:9/411: chown d1/c1c 1 1 2024-03-23T15:44:53.115 INFO:tasks.workunit.client.0.smithi076.stdout:8/329: mkdir d8/d2e/d30/d69 0 2024-03-23T15:44:53.115 INFO:tasks.workunit.client.0.smithi076.stdout:8/330: chown d8/d26/l34 649357338 1 2024-03-23T15:44:53.115 INFO:tasks.workunit.client.0.smithi076.stdout:8/331: dread - d8/d26/f46 zero size 2024-03-23T15:44:53.115 INFO:tasks.workunit.client.0.smithi076.stdout:8/332: readlink d8/d26/d33/l42 0 2024-03-23T15:44:53.115 INFO:tasks.workunit.client.0.smithi076.stdout:8/333: write d8/d2e/f5c [893261,111221] 0 2024-03-23T15:44:53.116 INFO:tasks.workunit.client.0.smithi076.stdout:6/322: write d1/d4/f5 [2075812,123510] 0 2024-03-23T15:44:53.116 INFO:tasks.workunit.client.0.smithi076.stdout:9/412: creat d1/d2f/d3f/f8b x:0 0 0 2024-03-23T15:44:53.118 INFO:tasks.workunit.client.0.smithi076.stdout:9/413: truncate d1/d20/f43 539987 0 2024-03-23T15:44:53.142 INFO:tasks.workunit.client.0.smithi076.stdout:8/334: dwrite d8/d26/f4d [0,4194304] 0 2024-03-23T15:44:53.143 INFO:tasks.workunit.client.0.smithi076.stdout:8/335: read - d8/d2e/d30/d3c/f4e zero size 2024-03-23T15:44:53.143 INFO:tasks.workunit.client.0.smithi076.stdout:9/414: dwrite d1/d2f/f42 [0,4194304] 0 2024-03-23T15:44:53.147 INFO:tasks.workunit.client.0.smithi076.stdout:9/415: dread d1/f12 [0,4194304] 0 2024-03-23T15:44:53.147 INFO:tasks.workunit.client.0.smithi076.stdout:8/336: dread d8/d2e/f4a [0,4194304] 0 2024-03-23T15:44:53.147 INFO:tasks.workunit.client.0.smithi076.stdout:9/416: dread d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:53.149 INFO:tasks.workunit.client.0.smithi076.stdout:8/337: creat d8/d2e/d30/f6a x:0 0 0 2024-03-23T15:44:53.150 INFO:tasks.workunit.client.0.smithi076.stdout:9/417: unlink d1/d1f/d29/c34 0 2024-03-23T15:44:53.168 INFO:tasks.workunit.client.0.smithi076.stdout:8/338: dwrite d8/fa [0,4194304] 0 2024-03-23T15:44:53.169 INFO:tasks.workunit.client.0.smithi076.stdout:9/418: dwrite d1/d1f/d29/d2d/d37/f3c [0,4194304] 0 2024-03-23T15:44:53.169 INFO:tasks.workunit.client.0.smithi076.stdout:8/339: mkdir d8/d6b 0 2024-03-23T15:44:53.171 INFO:tasks.workunit.client.0.smithi076.stdout:9/419: rmdir d1/d2f/d3a/d53/d7c 0 2024-03-23T15:44:53.172 INFO:tasks.workunit.client.0.smithi076.stdout:9/420: write d1/d1f/f72 [4304090,96515] 0 2024-03-23T15:44:53.182 INFO:tasks.workunit.client.0.smithi076.stdout:9/421: dread d1/d2f/d3a/d64/f69 [0,4194304] 0 2024-03-23T15:44:53.186 INFO:tasks.workunit.client.0.smithi076.stdout:8/340: dwrite d8/f52 [0,4194304] 0 2024-03-23T15:44:53.186 INFO:tasks.workunit.client.0.smithi076.stdout:8/341: chown d8/d2e/d3e/c5d 690 1 2024-03-23T15:44:53.191 INFO:tasks.workunit.client.0.smithi076.stdout:8/342: truncate d8/f14 715594 0 2024-03-23T15:44:53.191 INFO:tasks.workunit.client.0.smithi076.stdout:8/343: getdents d8/d2e/d30/d64 0 2024-03-23T15:44:53.194 INFO:tasks.workunit.client.0.smithi076.stdout:5/381: sync 2024-03-23T15:44:53.194 INFO:tasks.workunit.client.0.smithi076.stdout:7/499: sync 2024-03-23T15:44:53.195 INFO:tasks.workunit.client.0.smithi076.stdout:3/381: sync 2024-03-23T15:44:53.197 INFO:tasks.workunit.client.0.smithi076.stdout:7/500: truncate d8/de/d13/d40/d88/f33 2751645 0 2024-03-23T15:44:53.197 INFO:tasks.workunit.client.0.smithi076.stdout:3/382: creat d5/d35/d4c/d5a/f82 x:0 0 0 2024-03-23T15:44:53.198 INFO:tasks.workunit.client.0.smithi076.stdout:9/422: dwrite d1/f35 [0,4194304] 0 2024-03-23T15:44:53.199 INFO:tasks.workunit.client.0.smithi076.stdout:3/383: truncate d5/d35/f36 4742519 0 2024-03-23T15:44:53.199 INFO:tasks.workunit.client.0.smithi076.stdout:3/384: readlink d5/d17/d1c/d32/l5b 0 2024-03-23T15:44:53.203 INFO:tasks.workunit.client.0.smithi076.stdout:9/423: link d1/d2f/d3f/c81 d1/d20/c8c 0 2024-03-23T15:44:53.204 INFO:tasks.workunit.client.0.smithi076.stdout:8/344: dread f3 [0,4194304] 0 2024-03-23T15:44:53.205 INFO:tasks.workunit.client.0.smithi076.stdout:9/424: symlink d1/d1f/d29/d2d/d37/d47/l8d 0 2024-03-23T15:44:53.205 INFO:tasks.workunit.client.0.smithi076.stdout:9/425: chown d1/c1e 2166 1 2024-03-23T15:44:53.206 INFO:tasks.workunit.client.0.smithi076.stdout:3/385: symlink d5/d17/d1c/d31/d6b/l83 0 2024-03-23T15:44:53.206 INFO:tasks.workunit.client.0.smithi076.stdout:1/243: sync 2024-03-23T15:44:53.206 INFO:tasks.workunit.client.0.smithi076.stdout:2/446: sync 2024-03-23T15:44:53.207 INFO:tasks.workunit.client.0.smithi076.stdout:6/323: sync 2024-03-23T15:44:53.207 INFO:tasks.workunit.client.0.smithi076.stdout:4/484: sync 2024-03-23T15:44:53.208 INFO:tasks.workunit.client.0.smithi076.stdout:0/490: sync 2024-03-23T15:44:53.208 INFO:tasks.workunit.client.0.smithi076.stdout:0/491: dread - d2/d8/d32/f48 zero size 2024-03-23T15:44:53.208 INFO:tasks.workunit.client.0.smithi076.stdout:0/492: dread d2/d8/d32/f4a [0,4194304] 0 2024-03-23T15:44:53.210 INFO:tasks.workunit.client.0.smithi076.stdout:3/386: mknod d5/d17/d1c/d32/d39/c84 0 2024-03-23T15:44:53.211 INFO:tasks.workunit.client.0.smithi076.stdout:2/447: dread d1/d4/db/d15/f21 [0,4194304] 0 2024-03-23T15:44:53.211 INFO:tasks.workunit.client.0.smithi076.stdout:1/244: truncate d0/d2/f37 2380519 0 2024-03-23T15:44:53.212 INFO:tasks.workunit.client.0.smithi076.stdout:4/485: symlink d8/d3a/d69/d73/l8f 0 2024-03-23T15:44:53.212 INFO:tasks.workunit.client.0.smithi076.stdout:2/448: creat d1/d2e/d6a/f9b x:0 0 0 2024-03-23T15:44:53.213 INFO:tasks.workunit.client.0.smithi076.stdout:3/387: mkdir d5/d35/d3c/d67/d85 0 2024-03-23T15:44:53.214 INFO:tasks.workunit.client.0.smithi076.stdout:2/449: dread d1/d4/d64/d1b/f1c [0,4194304] 0 2024-03-23T15:44:53.215 INFO:tasks.workunit.client.0.smithi076.stdout:2/450: write d1/d4/d64/f18 [566449,97636] 0 2024-03-23T15:44:53.216 INFO:tasks.workunit.client.0.smithi076.stdout:4/486: link d8/d3a/d69/d73/d76/f83 d8/d9/d32/f90 0 2024-03-23T15:44:53.216 INFO:tasks.workunit.client.0.smithi076.stdout:4/487: truncate d8/d3a/d18/f89 974081 0 2024-03-23T15:44:53.218 INFO:tasks.workunit.client.0.smithi076.stdout:4/488: creat d8/d3a/d3b/f91 x:0 0 0 2024-03-23T15:44:53.223 INFO:tasks.workunit.client.0.smithi076.stdout:4/489: creat d8/d3a/d18/d52/f92 x:0 0 0 2024-03-23T15:44:53.223 INFO:tasks.workunit.client.0.smithi076.stdout:4/490: readlink d8/d3a/l7c 0 2024-03-23T15:44:53.225 INFO:tasks.workunit.client.0.smithi076.stdout:4/491: write d8/f6a [140040,45307] 0 2024-03-23T15:44:53.225 INFO:tasks.workunit.client.0.smithi076.stdout:4/492: dread - d8/d3a/d69/d73/d76/f83 zero size 2024-03-23T15:44:53.226 INFO:tasks.workunit.client.0.smithi076.stdout:4/493: write d8/d3a/d18/d1b/f67 [148531,32400] 0 2024-03-23T15:44:53.226 INFO:tasks.workunit.client.0.smithi076.stdout:4/494: chown d8/c6e 176300085 1 2024-03-23T15:44:53.226 INFO:tasks.workunit.client.0.smithi076.stdout:3/388: dread d5/d35/d4c/d5a/f41 [0,4194304] 0 2024-03-23T15:44:53.227 INFO:tasks.workunit.client.0.smithi076.stdout:2/451: write d1/d2e/d8a/f9a [2526055,67443] 0 2024-03-23T15:44:53.227 INFO:tasks.workunit.client.0.smithi076.stdout:5/382: dwrite d1a/f42 [0,4194304] 0 2024-03-23T15:44:53.228 INFO:tasks.workunit.client.0.smithi076.stdout:2/452: mknod d1/d4/d64/d82/d7f/c9c 0 2024-03-23T15:44:53.228 INFO:tasks.workunit.client.0.smithi076.stdout:5/383: readlink d1a/l2f 0 2024-03-23T15:44:53.229 INFO:tasks.workunit.client.0.smithi076.stdout:4/495: dread d8/d3a/d3b/f4c [0,4194304] 0 2024-03-23T15:44:53.230 INFO:tasks.workunit.client.0.smithi076.stdout:3/389: stat d5/d17/d1c/d32/d5d/l80 0 2024-03-23T15:44:53.231 INFO:tasks.workunit.client.0.smithi076.stdout:6/324: link d1/d2/d27/l4f d1/d4/d10/d6b/l6f 0 2024-03-23T15:44:53.242 INFO:tasks.workunit.client.0.smithi076.stdout:8/345: rename d8/d26/d27/c2b to d8/d2e/d30/d3c/c6c 0 2024-03-23T15:44:53.243 INFO:tasks.workunit.client.0.smithi076.stdout:8/346: mknod d8/d26/c6d 0 2024-03-23T15:44:53.243 INFO:tasks.workunit.client.0.smithi076.stdout:8/347: readlink d8/l24 0 2024-03-23T15:44:53.243 INFO:tasks.workunit.client.0.smithi076.stdout:8/348: write d8/f19 [5348016,114258] 0 2024-03-23T15:44:53.246 INFO:tasks.workunit.client.0.smithi076.stdout:0/493: dread d2/de/d10/d43/d56/f60 [0,4194304] 0 2024-03-23T15:44:53.246 INFO:tasks.workunit.client.0.smithi076.stdout:0/494: getdents d2/d8/d52 0 2024-03-23T15:44:53.246 INFO:tasks.workunit.client.0.smithi076.stdout:0/495: fdatasync d2/f7 0 2024-03-23T15:44:53.246 INFO:tasks.workunit.client.0.smithi076.stdout:0/496: truncate d2/de/d10/f67 615668 0 2024-03-23T15:44:53.246 INFO:tasks.workunit.client.0.smithi076.stdout:0/497: stat d2/d8/d2d/c83 0 2024-03-23T15:44:53.250 INFO:tasks.workunit.client.0.smithi076.stdout:0/498: unlink d2/d8/c29 0 2024-03-23T15:44:53.250 INFO:tasks.workunit.client.0.smithi076.stdout:2/453: write d1/d4/db/d3e/f77 [2462659,44278] 0 2024-03-23T15:44:53.250 INFO:tasks.workunit.client.0.smithi076.stdout:0/499: dread d2/f9 [0,4194304] 0 2024-03-23T15:44:53.254 INFO:tasks.workunit.client.0.smithi076.stdout:3/390: dwrite d5/d17/d1c/d32/d39/f3a [0,4194304] 0 2024-03-23T15:44:53.256 INFO:tasks.workunit.client.0.smithi076.stdout:3/391: dread d5/d17/d1c/d32/d39/f44 [0,4194304] 0 2024-03-23T15:44:53.261 INFO:tasks.workunit.client.0.smithi076.stdout:8/349: read d8/f1a [2926732,8471] 0 2024-03-23T15:44:53.261 INFO:tasks.workunit.client.0.smithi076.stdout:8/350: dread - d8/d26/d33/f3d zero size 2024-03-23T15:44:53.262 INFO:tasks.workunit.client.0.smithi076.stdout:8/351: truncate d8/d26/d33/f3a 1133653 0 2024-03-23T15:44:53.265 INFO:tasks.workunit.client.0.smithi076.stdout:8/352: creat d8/d2e/f6e x:0 0 0 2024-03-23T15:44:53.266 INFO:tasks.workunit.client.0.smithi076.stdout:4/496: dwrite d8/d3a/d69/d73/d76/f78 [0,4194304] 0 2024-03-23T15:44:53.268 INFO:tasks.workunit.client.0.smithi076.stdout:8/353: rmdir d8/d2e 39 2024-03-23T15:44:53.268 INFO:tasks.workunit.client.0.smithi076.stdout:8/354: chown d8/f19 0 1 2024-03-23T15:44:53.269 INFO:tasks.workunit.client.0.smithi076.stdout:2/454: dwrite d1/d4/db/d15/d29/f61 [0,4194304] 0 2024-03-23T15:44:53.269 INFO:tasks.workunit.client.0.smithi076.stdout:0/500: dwrite d2/d8/fa [0,4194304] 0 2024-03-23T15:44:53.271 INFO:tasks.workunit.client.0.smithi076.stdout:4/497: symlink d8/d9/d32/l93 0 2024-03-23T15:44:53.273 INFO:tasks.workunit.client.0.smithi076.stdout:8/355: mknod d8/c6f 0 2024-03-23T15:44:53.275 INFO:tasks.workunit.client.0.smithi076.stdout:2/455: write d1/d4/db/d15/f55 [3728451,109937] 0 2024-03-23T15:44:53.275 INFO:tasks.workunit.client.0.smithi076.stdout:2/456: fdatasync d1/d4/d64/d1b/f71 0 2024-03-23T15:44:53.276 INFO:tasks.workunit.client.0.smithi076.stdout:2/457: getdents d1/d4/db/d3e 0 2024-03-23T15:44:53.277 INFO:tasks.workunit.client.0.smithi076.stdout:2/458: read - d1/d4/db/d15/d29/f8f zero size 2024-03-23T15:44:53.277 INFO:tasks.workunit.client.0.smithi076.stdout:4/498: creat d8/d3a/d3b/f94 x:0 0 0 2024-03-23T15:44:53.278 INFO:tasks.workunit.client.0.smithi076.stdout:2/459: unlink d1/d4/d52/d6e/f78 0 2024-03-23T15:44:53.280 INFO:tasks.workunit.client.0.smithi076.stdout:4/499: unlink d8/d3a/d3b/f26 0 2024-03-23T15:44:53.281 INFO:tasks.workunit.client.0.smithi076.stdout:0/501: write d2/d8/d2d/f3f [4422948,94021] 0 2024-03-23T15:44:53.281 INFO:tasks.workunit.client.0.smithi076.stdout:1/245: rename d0/d35/d40/d47/l1b to d0/d1/l4c 0 2024-03-23T15:44:53.284 INFO:tasks.workunit.client.0.smithi076.stdout:0/502: mknod d2/de/d10/d3a/d47/c8e 0 2024-03-23T15:44:53.284 INFO:tasks.workunit.client.0.smithi076.stdout:0/503: write d2/de/f13 [2256880,77207] 0 2024-03-23T15:44:53.284 INFO:tasks.workunit.client.0.smithi076.stdout:0/504: dread - d2/de/d10/d43/f88 zero size 2024-03-23T15:44:53.284 INFO:tasks.workunit.client.0.smithi076.stdout:0/505: write d2/de/d10/d12/f6a [1030417,12303] 0 2024-03-23T15:44:53.284 INFO:tasks.workunit.client.0.smithi076.stdout:5/384: rename d1a/d4b/c79 to d1a/d20/c7f 0 2024-03-23T15:44:53.284 INFO:tasks.workunit.client.0.smithi076.stdout:5/385: dread - d1a/d33/d3a/f58 zero size 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:6/325: rename d1/d4/cb to d1/d4/d10/d6b/c70 0 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:1/246: read d0/d35/d40/d47/f9 [8309197,98437] 0 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:5/386: rename d1a/d33/d3a/d64 to d1a/d33/d3a/d80 0 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:1/247: dread - d0/d35/d40/d47/d17/f49 zero size 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:1/248: readlink d0/d1/l4c 0 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:1/249: read - d0/d2/d1f/f43 zero size 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:1/250: chown d0/d1/cb 367142 1 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:1/251: read d0/d35/d40/d47/f9 [2178321,104122] 0 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:7/501: sync 2024-03-23T15:44:53.286 INFO:tasks.workunit.client.0.smithi076.stdout:1/252: read d0/d35/d40/d47/d17/d20/f25 [1495797,78811] 0 2024-03-23T15:44:53.287 INFO:tasks.workunit.client.0.smithi076.stdout:1/253: symlink d0/d35/d40/l4d 0 2024-03-23T15:44:53.287 INFO:tasks.workunit.client.0.smithi076.stdout:1/254: fsync d0/f34 0 2024-03-23T15:44:53.287 INFO:tasks.workunit.client.0.smithi076.stdout:1/255: stat d0/d35/d40/d47/d17/f1d 0 2024-03-23T15:44:53.287 INFO:tasks.workunit.client.0.smithi076.stdout:0/506: dread d2/d1b/f42 [0,4194304] 0 2024-03-23T15:44:53.290 INFO:tasks.workunit.client.0.smithi076.stdout:0/507: mknod d2/d8/c8f 0 2024-03-23T15:44:53.298 INFO:tasks.workunit.client.0.smithi076.stdout:0/508: dread d2/de/d10/f8c [0,4194304] 0 2024-03-23T15:44:53.298 INFO:tasks.workunit.client.0.smithi076.stdout:4/500: dwrite d8/d3a/d18/d52/f64 [0,4194304] 0 2024-03-23T15:44:53.302 INFO:tasks.workunit.client.0.smithi076.stdout:2/460: dwrite d1/d4/d64/d82/d7f/f8c [0,4194304] 0 2024-03-23T15:44:53.302 INFO:tasks.workunit.client.0.smithi076.stdout:2/461: fsync d1/f2f 0 2024-03-23T15:44:53.302 INFO:tasks.workunit.client.0.smithi076.stdout:0/509: symlink d2/de/d10/d3a/d47/l90 0 2024-03-23T15:44:53.302 INFO:tasks.workunit.client.0.smithi076.stdout:0/510: fdatasync d2/d8/d32/f48 0 2024-03-23T15:44:53.304 INFO:tasks.workunit.client.0.smithi076.stdout:4/501: getdents d8/d80 0 2024-03-23T15:44:53.305 INFO:tasks.workunit.client.0.smithi076.stdout:2/462: getdents d1/d4/db/d15 0 2024-03-23T15:44:53.305 INFO:tasks.workunit.client.0.smithi076.stdout:2/463: fdatasync d1/d2e/d8a/f9a 0 2024-03-23T15:44:53.306 INFO:tasks.workunit.client.0.smithi076.stdout:0/511: chown d2/de/d10/d3a/d47/d69/c6b 127626 1 2024-03-23T15:44:53.308 INFO:tasks.workunit.client.0.smithi076.stdout:4/502: mknod d8/d3a/d18/c95 0 2024-03-23T15:44:53.308 INFO:tasks.workunit.client.0.smithi076.stdout:0/512: dread d2/f9 [0,4194304] 0 2024-03-23T15:44:53.313 INFO:tasks.workunit.client.0.smithi076.stdout:5/387: dwrite d1a/d33/f51 [0,4194304] 0 2024-03-23T15:44:53.315 INFO:tasks.workunit.client.0.smithi076.stdout:6/326: dwrite d1/dd/df/d5b/f3e [0,4194304] 0 2024-03-23T15:44:53.315 INFO:tasks.workunit.client.0.smithi076.stdout:7/502: dwrite d8/de/d13/f14 [0,4194304] 0 2024-03-23T15:44:53.317 INFO:tasks.workunit.client.0.smithi076.stdout:5/388: truncate d1a/d33/f54 681940 0 2024-03-23T15:44:53.317 INFO:tasks.workunit.client.0.smithi076.stdout:7/503: symlink d8/de/d13/d18/laa 0 2024-03-23T15:44:53.318 INFO:tasks.workunit.client.0.smithi076.stdout:7/504: truncate d8/f58 986121 0 2024-03-23T15:44:53.318 INFO:tasks.workunit.client.0.smithi076.stdout:1/256: dwrite d0/d35/d40/d47/d17/f49 [0,4194304] 0 2024-03-23T15:44:53.319 INFO:tasks.workunit.client.0.smithi076.stdout:7/505: getdents d8 0 2024-03-23T15:44:53.319 INFO:tasks.workunit.client.0.smithi076.stdout:7/506: chown d8/l44 9 1 2024-03-23T15:44:53.320 INFO:tasks.workunit.client.0.smithi076.stdout:6/327: unlink d1/d4/f5 0 2024-03-23T15:44:53.320 INFO:tasks.workunit.client.0.smithi076.stdout:5/389: truncate d1a/d39/d65/f6b 566277 0 2024-03-23T15:44:53.322 INFO:tasks.workunit.client.0.smithi076.stdout:7/507: link d8/de/d13/d40/d5d/c31 d8/de/d13/d40/d5d/d3c/d8e/cab 0 2024-03-23T15:44:53.325 INFO:tasks.workunit.client.0.smithi076.stdout:1/257: creat d0/d35/d40/d47/f4e x:0 0 0 2024-03-23T15:44:53.325 INFO:tasks.workunit.client.0.smithi076.stdout:1/258: write d0/f34 [102584,25837] 0 2024-03-23T15:44:53.325 INFO:tasks.workunit.client.0.smithi076.stdout:2/464: dwrite d1/d4/f54 [0,4194304] 0 2024-03-23T15:44:53.326 INFO:tasks.workunit.client.0.smithi076.stdout:8/356: rename d8/d2e/f6e to d8/d2e/d30/f70 0 2024-03-23T15:44:53.327 INFO:tasks.workunit.client.0.smithi076.stdout:1/259: getdents d0/d1 0 2024-03-23T15:44:53.328 INFO:tasks.workunit.client.0.smithi076.stdout:5/390: write d1a/d33/f51 [1534936,83662] 0 2024-03-23T15:44:53.328 INFO:tasks.workunit.client.0.smithi076.stdout:2/465: creat d1/d4/db/d15/d29/d7d/f9d x:0 0 0 2024-03-23T15:44:53.329 INFO:tasks.workunit.client.0.smithi076.stdout:2/466: write d1/d4/db/f5b [2535169,79697] 0 2024-03-23T15:44:53.329 INFO:tasks.workunit.client.0.smithi076.stdout:4/503: dwrite d8/d3a/d18/f20 [0,4194304] 0 2024-03-23T15:44:53.329 INFO:tasks.workunit.client.0.smithi076.stdout:4/504: write d8/d9/f5c [232402,8412] 0 2024-03-23T15:44:53.331 INFO:tasks.workunit.client.0.smithi076.stdout:7/508: dread d8/de/d13/d3a/f50 [0,4194304] 0 2024-03-23T15:44:53.331 INFO:tasks.workunit.client.0.smithi076.stdout:5/391: rename d1a/d39/d65/f77 to d1a/d39/d5f/f81 0 2024-03-23T15:44:53.332 INFO:tasks.workunit.client.0.smithi076.stdout:1/260: symlink d0/d1/l4f 0 2024-03-23T15:44:53.333 INFO:tasks.workunit.client.0.smithi076.stdout:1/261: dread - d0/d35/d40/d47/d17/d20/f46 zero size 2024-03-23T15:44:53.333 INFO:tasks.workunit.client.0.smithi076.stdout:1/262: dread - d0/d35/d40/d47/d17/f38 zero size 2024-03-23T15:44:53.334 INFO:tasks.workunit.client.0.smithi076.stdout:8/357: write d8/d2e/d30/d3c/f5f [2536350,17787] 0 2024-03-23T15:44:53.334 INFO:tasks.workunit.client.0.smithi076.stdout:8/358: chown d8/l22 442 1 2024-03-23T15:44:53.334 INFO:tasks.workunit.client.0.smithi076.stdout:2/467: dread d1/d4/d64/d1b/f1c [0,4194304] 0 2024-03-23T15:44:53.335 INFO:tasks.workunit.client.0.smithi076.stdout:7/509: mknod d8/de/d13/d3a/d82/cac 0 2024-03-23T15:44:53.336 INFO:tasks.workunit.client.0.smithi076.stdout:8/359: creat d8/d2e/d30/d69/f71 x:0 0 0 2024-03-23T15:44:53.336 INFO:tasks.workunit.client.0.smithi076.stdout:8/360: readlink d8/l25 0 2024-03-23T15:44:53.336 INFO:tasks.workunit.client.0.smithi076.stdout:8/361: write d8/f11 [1108676,90485] 0 2024-03-23T15:44:53.337 INFO:tasks.workunit.client.0.smithi076.stdout:2/468: creat d1/d4/db/d15/d29/f9e x:0 0 0 2024-03-23T15:44:53.338 INFO:tasks.workunit.client.0.smithi076.stdout:6/328: dwrite d1/dd/d2a/d23/f2d [0,4194304] 0 2024-03-23T15:44:53.340 INFO:tasks.workunit.client.0.smithi076.stdout:4/505: creat d8/d3a/d18/d1b/f96 x:0 0 0 2024-03-23T15:44:53.341 INFO:tasks.workunit.client.0.smithi076.stdout:7/510: creat d8/de/fad x:0 0 0 2024-03-23T15:44:53.341 INFO:tasks.workunit.client.0.smithi076.stdout:7/511: fsync d8/d4e/f87 0 2024-03-23T15:44:53.341 INFO:tasks.workunit.client.0.smithi076.stdout:7/512: chown d8/d4e/c53 6154 1 2024-03-23T15:44:53.342 INFO:tasks.workunit.client.0.smithi076.stdout:2/469: rmdir d1/d2e/d6a 39 2024-03-23T15:44:53.344 INFO:tasks.workunit.client.0.smithi076.stdout:1/263: unlink d0/l13 0 2024-03-23T15:44:53.346 INFO:tasks.workunit.client.0.smithi076.stdout:8/362: link d8/d2e/c2f d8/c72 0 2024-03-23T15:44:53.349 INFO:tasks.workunit.client.0.smithi076.stdout:8/363: mknod d8/d26/d33/c73 0 2024-03-23T15:44:53.350 INFO:tasks.workunit.client.0.smithi076.stdout:7/513: rename d8/de/d13/d40/f45 to d8/de/d13/d3a/fae 0 2024-03-23T15:44:53.350 INFO:tasks.workunit.client.0.smithi076.stdout:6/329: creat d1/d2/f71 x:0 0 0 2024-03-23T15:44:53.351 INFO:tasks.workunit.client.0.smithi076.stdout:6/330: readlink d1/dd/l58 0 2024-03-23T15:44:53.353 INFO:tasks.workunit.client.0.smithi076.stdout:7/514: mkdir d8/de/d13/d3a/d85/daf 0 2024-03-23T15:44:53.353 INFO:tasks.workunit.client.0.smithi076.stdout:6/331: mkdir d1/d4/d10/d6b/d72 0 2024-03-23T15:44:53.353 INFO:tasks.workunit.client.0.smithi076.stdout:4/506: write d8/f49 [8074704,26211] 0 2024-03-23T15:44:53.353 INFO:tasks.workunit.client.0.smithi076.stdout:4/507: dread - d8/d3a/d69/d73/d76/f7a zero size 2024-03-23T15:44:53.353 INFO:tasks.workunit.client.0.smithi076.stdout:4/508: write d8/d3a/d18/d52/f61 [340542,84664] 0 2024-03-23T15:44:53.354 INFO:tasks.workunit.client.0.smithi076.stdout:0/513: dwrite d2/d8/d2d/f3f [0,4194304] 0 2024-03-23T15:44:53.354 INFO:tasks.workunit.client.0.smithi076.stdout:7/515: dread d8/f58 [0,4194304] 0 2024-03-23T15:44:53.354 INFO:tasks.workunit.client.0.smithi076.stdout:2/470: dread d1/d4/db/d3e/f77 [0,4194304] 0 2024-03-23T15:44:53.354 INFO:tasks.workunit.client.0.smithi076.stdout:2/471: fdatasync d1/d2e/d6a/f75 0 2024-03-23T15:44:53.355 INFO:tasks.workunit.client.0.smithi076.stdout:4/509: dread d8/d3a/d18/d1b/f67 [0,4194304] 0 2024-03-23T15:44:53.355 INFO:tasks.workunit.client.0.smithi076.stdout:4/510: read - d8/d3a/d69/d73/d76/f81 zero size 2024-03-23T15:44:53.355 INFO:tasks.workunit.client.0.smithi076.stdout:4/511: read - d8/d3a/d3b/f91 zero size 2024-03-23T15:44:53.355 INFO:tasks.workunit.client.0.smithi076.stdout:5/392: dwrite d1a/d33/f69 [0,4194304] 0 2024-03-23T15:44:53.356 INFO:tasks.workunit.client.0.smithi076.stdout:9/426: sync 2024-03-23T15:44:53.356 INFO:tasks.workunit.client.0.smithi076.stdout:9/427: chown d1/d1f/d29/d2d/c7b 534679482 1 2024-03-23T15:44:53.357 INFO:tasks.workunit.client.0.smithi076.stdout:7/516: unlink d8/de/d13/f14 0 2024-03-23T15:44:53.358 INFO:tasks.workunit.client.0.smithi076.stdout:7/517: truncate d8/f83 64887 0 2024-03-23T15:44:53.358 INFO:tasks.workunit.client.0.smithi076.stdout:5/393: dread - d1a/d2c/f6f zero size 2024-03-23T15:44:53.359 INFO:tasks.workunit.client.0.smithi076.stdout:8/364: dread d8/f18 [0,4194304] 0 2024-03-23T15:44:53.360 INFO:tasks.workunit.client.0.smithi076.stdout:1/264: dwrite d0/d35/d40/d47/f9 [4194304,4194304] 0 2024-03-23T15:44:53.360 INFO:tasks.workunit.client.0.smithi076.stdout:9/428: symlink d1/d1f/d29/d2d/l8e 0 2024-03-23T15:44:53.362 INFO:tasks.workunit.client.0.smithi076.stdout:4/512: mkdir d8/d41/d97 0 2024-03-23T15:44:53.363 INFO:tasks.workunit.client.0.smithi076.stdout:0/514: unlink d2/d8/d32/f39 0 2024-03-23T15:44:53.364 INFO:tasks.workunit.client.0.smithi076.stdout:3/392: sync 2024-03-23T15:44:53.364 INFO:tasks.workunit.client.0.smithi076.stdout:3/393: dread - d5/d17/d1c/d32/d39/f7c zero size 2024-03-23T15:44:53.365 INFO:tasks.workunit.client.0.smithi076.stdout:1/265: mknod d0/d1/c50 0 2024-03-23T15:44:53.365 INFO:tasks.workunit.client.0.smithi076.stdout:1/266: dread - d0/d35/d40/d47/d17/f38 zero size 2024-03-23T15:44:53.366 INFO:tasks.workunit.client.0.smithi076.stdout:3/394: truncate d5/d17/d1c/f1d 919861 0 2024-03-23T15:44:53.368 INFO:tasks.workunit.client.0.smithi076.stdout:1/267: mknod d0/c51 0 2024-03-23T15:44:53.370 INFO:tasks.workunit.client.0.smithi076.stdout:3/395: rmdir d5/d35/d3c/d67/d85 0 2024-03-23T15:44:53.370 INFO:tasks.workunit.client.0.smithi076.stdout:3/396: truncate d5/d17/d1c/d32/d39/f7c 816698 0 2024-03-23T15:44:53.372 INFO:tasks.workunit.client.0.smithi076.stdout:9/429: dread d1/d1f/d29/f54 [0,4194304] 0 2024-03-23T15:44:53.373 INFO:tasks.workunit.client.0.smithi076.stdout:3/397: symlink d5/d17/d1c/d31/d6b/l86 0 2024-03-23T15:44:53.375 INFO:tasks.workunit.client.0.smithi076.stdout:9/430: read d1/d1f/d46/d4a/f50 [67482,88319] 0 2024-03-23T15:44:53.376 INFO:tasks.workunit.client.0.smithi076.stdout:3/398: mknod d5/d17/d1c/d31/d47/c87 0 2024-03-23T15:44:53.381 INFO:tasks.workunit.client.0.smithi076.stdout:3/399: unlink d5/f12 0 2024-03-23T15:44:53.381 INFO:tasks.workunit.client.0.smithi076.stdout:3/400: dread - d5/d17/d1c/d32/f5f zero size 2024-03-23T15:44:53.385 INFO:tasks.workunit.client.0.smithi076.stdout:7/518: dwrite d8/de/d13/d18/f2b [0,4194304] 0 2024-03-23T15:44:53.385 INFO:tasks.workunit.client.0.smithi076.stdout:7/519: readlink d8/de/la6 0 2024-03-23T15:44:53.386 INFO:tasks.workunit.client.0.smithi076.stdout:7/520: symlink d8/lb0 0 2024-03-23T15:44:53.388 INFO:tasks.workunit.client.0.smithi076.stdout:9/431: symlink d1/d1f/d46/d83/l8f 0 2024-03-23T15:44:53.388 INFO:tasks.workunit.client.0.smithi076.stdout:0/515: dwrite d2/de/d10/f67 [0,4194304] 0 2024-03-23T15:44:53.390 INFO:tasks.workunit.client.0.smithi076.stdout:8/365: dwrite d8/f1d [0,4194304] 0 2024-03-23T15:44:53.390 INFO:tasks.workunit.client.0.smithi076.stdout:5/394: dwrite d1a/d39/f44 [0,4194304] 0 2024-03-23T15:44:53.391 INFO:tasks.workunit.client.0.smithi076.stdout:4/513: dwrite d8/d3a/d69/d73/d76/f77 [0,4194304] 0 2024-03-23T15:44:53.392 INFO:tasks.workunit.client.0.smithi076.stdout:0/516: creat d2/d8/d2d/f91 x:0 0 0 2024-03-23T15:44:53.393 INFO:tasks.workunit.client.0.smithi076.stdout:7/521: dread d8/f3f [0,4194304] 0 2024-03-23T15:44:53.393 INFO:tasks.workunit.client.0.smithi076.stdout:7/522: chown d8/de/d13/d40/d5d/f6a 28281356 1 2024-03-23T15:44:53.393 INFO:tasks.workunit.client.0.smithi076.stdout:4/514: dread d8/d3a/d18/f21 [0,4194304] 0 2024-03-23T15:44:53.395 INFO:tasks.workunit.client.0.smithi076.stdout:7/523: symlink d8/de/d99/lb1 0 2024-03-23T15:44:53.395 INFO:tasks.workunit.client.0.smithi076.stdout:9/432: dread d1/f19 [0,4194304] 0 2024-03-23T15:44:53.397 INFO:tasks.workunit.client.0.smithi076.stdout:7/524: dread d8/de/d13/d40/d5d/f20 [0,4194304] 0 2024-03-23T15:44:53.398 INFO:tasks.workunit.client.0.smithi076.stdout:0/517: link d2/de/d10/d12/l26 d2/d8/l92 0 2024-03-23T15:44:53.401 INFO:tasks.workunit.client.0.smithi076.stdout:4/515: unlink d8/d3a/d18/d1b/f55 0 2024-03-23T15:44:53.401 INFO:tasks.workunit.client.0.smithi076.stdout:4/516: fsync d8/f3f 0 2024-03-23T15:44:53.402 INFO:tasks.workunit.client.0.smithi076.stdout:8/366: symlink d8/d26/d33/l74 0 2024-03-23T15:44:53.403 INFO:tasks.workunit.client.0.smithi076.stdout:9/433: creat d1/d20/f90 x:0 0 0 2024-03-23T15:44:53.404 INFO:tasks.workunit.client.0.smithi076.stdout:8/367: symlink d8/d2e/d30/d3c/l75 0 2024-03-23T15:44:53.406 INFO:tasks.workunit.client.0.smithi076.stdout:6/332: rename d1/c5d to d1/d2/c73 0 2024-03-23T15:44:53.406 INFO:tasks.workunit.client.0.smithi076.stdout:0/518: dread d2/d8/d52/f64 [4194304,4194304] 0 2024-03-23T15:44:53.407 INFO:tasks.workunit.client.0.smithi076.stdout:4/517: link d8/d41/f54 d8/d3a/d18/f98 0 2024-03-23T15:44:53.407 INFO:tasks.workunit.client.0.smithi076.stdout:8/368: mknod d8/d2e/d30/d69/c76 0 2024-03-23T15:44:53.407 INFO:tasks.workunit.client.0.smithi076.stdout:0/519: readlink d2/de/d10/d43/d56/l7a 0 2024-03-23T15:44:53.408 INFO:tasks.workunit.client.0.smithi076.stdout:0/520: creat d2/de/d10/d3a/d47/d69/d84/f93 x:0 0 0 2024-03-23T15:44:53.409 INFO:tasks.workunit.client.0.smithi076.stdout:2/472: rename d1/d2e/l39 to d1/d4/db/d15/d29/d7d/l9f 0 2024-03-23T15:44:53.409 INFO:tasks.workunit.client.0.smithi076.stdout:0/521: truncate d2/de/f22 649969 0 2024-03-23T15:44:53.412 INFO:tasks.workunit.client.0.smithi076.stdout:2/473: symlink d1/d2e/la0 0 2024-03-23T15:44:53.414 INFO:tasks.workunit.client.0.smithi076.stdout:2/474: rmdir d1/d4/db 39 2024-03-23T15:44:53.414 INFO:tasks.workunit.client.0.smithi076.stdout:4/518: dread d8/d41/f54 [0,4194304] 0 2024-03-23T15:44:53.417 INFO:tasks.workunit.client.0.smithi076.stdout:4/519: link d8/d9/d32/c8e d8/d3a/c99 0 2024-03-23T15:44:53.417 INFO:tasks.workunit.client.0.smithi076.stdout:4/520: chown d8/f49 58712914 1 2024-03-23T15:44:53.417 INFO:tasks.workunit.client.0.smithi076.stdout:4/521: fsync d8/d3a/f34 0 2024-03-23T15:44:53.420 INFO:tasks.workunit.client.0.smithi076.stdout:4/522: write d8/d3a/d3b/f25 [775283,128028] 0 2024-03-23T15:44:53.420 INFO:tasks.workunit.client.0.smithi076.stdout:4/523: chown d8/d9 469062 1 2024-03-23T15:44:53.420 INFO:tasks.workunit.client.0.smithi076.stdout:4/524: dread - d8/d9/f2d zero size 2024-03-23T15:44:53.424 INFO:tasks.workunit.client.0.smithi076.stdout:1/268: rename d0/d1 to d0/d35/d52 0 2024-03-23T15:44:53.424 INFO:tasks.workunit.client.0.smithi076.stdout:7/525: dwrite d8/de/d13/d40/d88/f2d [0,4194304] 0 2024-03-23T15:44:53.424 INFO:tasks.workunit.client.0.smithi076.stdout:7/526: read - d8/de/d13/d3a/d82/f66 zero size 2024-03-23T15:44:53.426 INFO:tasks.workunit.client.0.smithi076.stdout:7/527: write d8/de/d13/d40/d5d/f5b [736737,74936] 0 2024-03-23T15:44:53.428 INFO:tasks.workunit.client.0.smithi076.stdout:4/525: write d8/d3a/d18/d52/f61 [248467,18238] 0 2024-03-23T15:44:53.428 INFO:tasks.workunit.client.0.smithi076.stdout:4/526: readlink d8/l68 0 2024-03-23T15:44:53.429 INFO:tasks.workunit.client.0.smithi076.stdout:7/528: creat d8/de/d13/d40/d88/fb2 x:0 0 0 2024-03-23T15:44:53.429 INFO:tasks.workunit.client.0.smithi076.stdout:9/434: dwrite d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:53.430 INFO:tasks.workunit.client.0.smithi076.stdout:9/435: stat d1/d1f/d46/d4a 0 2024-03-23T15:44:53.430 INFO:tasks.workunit.client.0.smithi076.stdout:9/436: chown d1/d1f/d29/d2d/c4e 14 1 2024-03-23T15:44:53.430 INFO:tasks.workunit.client.0.smithi076.stdout:9/437: getdents d1/d1f/d46/d4a 0 2024-03-23T15:44:53.431 INFO:tasks.workunit.client.0.smithi076.stdout:4/527: mkdir d8/d3a/d69/d73/d9a 0 2024-03-23T15:44:53.431 INFO:tasks.workunit.client.0.smithi076.stdout:4/528: fsync d8/d3a/d18/d52/f79 0 2024-03-23T15:44:53.432 INFO:tasks.workunit.client.0.smithi076.stdout:9/438: creat d1/d2f/d3a/d64/f91 x:0 0 0 2024-03-23T15:44:53.432 INFO:tasks.workunit.client.0.smithi076.stdout:9/439: dread - d1/d2f/d3a/d64/f91 zero size 2024-03-23T15:44:53.434 INFO:tasks.workunit.client.0.smithi076.stdout:8/369: dwrite d8/f67 [0,4194304] 0 2024-03-23T15:44:53.434 INFO:tasks.workunit.client.0.smithi076.stdout:8/370: chown d8/d4f/f53 63232447 1 2024-03-23T15:44:53.434 INFO:tasks.workunit.client.0.smithi076.stdout:8/371: chown d8/d26/d27/d56/f63 949 1 2024-03-23T15:44:53.435 INFO:tasks.workunit.client.0.smithi076.stdout:8/372: readlink d8/d26/l34 0 2024-03-23T15:44:53.435 INFO:tasks.workunit.client.0.smithi076.stdout:8/373: truncate d8/d2e/d30/f66 34752 0 2024-03-23T15:44:53.435 INFO:tasks.workunit.client.0.smithi076.stdout:4/529: dread d8/d3a/d18/f21 [0,4194304] 0 2024-03-23T15:44:53.435 INFO:tasks.workunit.client.0.smithi076.stdout:4/530: chown d8/d3a/d18/d52/f64 908209 1 2024-03-23T15:44:53.437 INFO:tasks.workunit.client.0.smithi076.stdout:0/522: dwrite d2/de/d10/d3a/d47/d69/d84/f93 [0,4194304] 0 2024-03-23T15:44:53.438 INFO:tasks.workunit.client.0.smithi076.stdout:0/523: fsync d2/d8/fa 0 2024-03-23T15:44:53.438 INFO:tasks.workunit.client.0.smithi076.stdout:7/529: dread d8/de/d13/d18/f2b [0,4194304] 0 2024-03-23T15:44:53.438 INFO:tasks.workunit.client.0.smithi076.stdout:2/475: dwrite d1/d2e/f3f [0,4194304] 0 2024-03-23T15:44:53.438 INFO:tasks.workunit.client.0.smithi076.stdout:7/530: truncate d8/de/d13/d40/d5d/f20 1674583 0 2024-03-23T15:44:53.438 INFO:tasks.workunit.client.0.smithi076.stdout:7/531: write d8/de/d13/f2f [2041044,82278] 0 2024-03-23T15:44:53.438 INFO:tasks.workunit.client.0.smithi076.stdout:4/531: rmdir d8/d9/d32/d84 0 2024-03-23T15:44:53.440 INFO:tasks.workunit.client.0.smithi076.stdout:2/476: symlink d1/d4/d52/la1 0 2024-03-23T15:44:53.440 INFO:tasks.workunit.client.0.smithi076.stdout:2/477: dread - d1/d4/d64/d7c/f85 zero size 2024-03-23T15:44:53.440 INFO:tasks.workunit.client.0.smithi076.stdout:6/333: dwrite d1/d2/d27/d60/f61 [0,4194304] 0 2024-03-23T15:44:53.441 INFO:tasks.workunit.client.0.smithi076.stdout:7/532: rmdir d8/de/d13/d3a/d85 39 2024-03-23T15:44:53.442 INFO:tasks.workunit.client.0.smithi076.stdout:0/524: symlink d2/de/d10/l94 0 2024-03-23T15:44:53.442 INFO:tasks.workunit.client.0.smithi076.stdout:0/525: read d2/d1b/f42 [2665137,33145] 0 2024-03-23T15:44:53.444 INFO:tasks.workunit.client.0.smithi076.stdout:2/478: symlink d1/d4/db/d15/la2 0 2024-03-23T15:44:53.444 INFO:tasks.workunit.client.0.smithi076.stdout:9/440: write d1/d1f/d29/d2d/d37/f59 [1972002,115783] 0 2024-03-23T15:44:53.445 INFO:tasks.workunit.client.0.smithi076.stdout:2/479: dread d1/f14 [0,4194304] 0 2024-03-23T15:44:53.445 INFO:tasks.workunit.client.0.smithi076.stdout:2/480: write d1/d4/f9 [405354,72977] 0 2024-03-23T15:44:53.446 INFO:tasks.workunit.client.0.smithi076.stdout:7/533: mkdir d8/de/d13/d3a/d85/db3 0 2024-03-23T15:44:53.446 INFO:tasks.workunit.client.0.smithi076.stdout:0/526: rmdir d2/de/d71 39 2024-03-23T15:44:53.448 INFO:tasks.workunit.client.0.smithi076.stdout:1/269: dwrite d0/d2/d1f/f43 [0,4194304] 0 2024-03-23T15:44:53.449 INFO:tasks.workunit.client.0.smithi076.stdout:9/441: rmdir d1/d1f/d82/d88 0 2024-03-23T15:44:53.450 INFO:tasks.workunit.client.0.smithi076.stdout:2/481: unlink d1/d2e/l81 0 2024-03-23T15:44:53.450 INFO:tasks.workunit.client.0.smithi076.stdout:2/482: read - d1/d2e/d6a/f7e zero size 2024-03-23T15:44:53.453 INFO:tasks.workunit.client.0.smithi076.stdout:2/483: truncate d1/d4/d52/f70 459226 0 2024-03-23T15:44:53.453 INFO:tasks.workunit.client.0.smithi076.stdout:2/484: truncate d1/f5 5224930 0 2024-03-23T15:44:53.453 INFO:tasks.workunit.client.0.smithi076.stdout:2/485: write d1/f80 [831737,26834] 0 2024-03-23T15:44:53.456 INFO:tasks.workunit.client.0.smithi076.stdout:1/270: creat d0/d35/d52/f53 x:0 0 0 2024-03-23T15:44:53.456 INFO:tasks.workunit.client.0.smithi076.stdout:1/271: write d0/d35/d40/d47/d17/d20/f25 [856237,74524] 0 2024-03-23T15:44:53.458 INFO:tasks.workunit.client.0.smithi076.stdout:4/532: dread d8/d3a/d3b/f4b [4194304,4194304] 0 2024-03-23T15:44:53.460 INFO:tasks.workunit.client.0.smithi076.stdout:1/272: creat d0/d35/d40/d47/d17/f54 x:0 0 0 2024-03-23T15:44:53.460 INFO:tasks.workunit.client.0.smithi076.stdout:1/273: dread - d0/d35/d40/d47/d17/f38 zero size 2024-03-23T15:44:53.465 INFO:tasks.workunit.client.0.smithi076.stdout:8/374: dwrite d8/fe [0,4194304] 0 2024-03-23T15:44:53.465 INFO:tasks.workunit.client.0.smithi076.stdout:8/375: chown d8/f23 51770993 1 2024-03-23T15:44:53.465 INFO:tasks.workunit.client.0.smithi076.stdout:8/376: chown d8/f14 28 1 2024-03-23T15:44:53.468 INFO:tasks.workunit.client.0.smithi076.stdout:9/442: dwrite d1/d20/f25 [4194304,4194304] 0 2024-03-23T15:44:53.474 INFO:tasks.workunit.client.0.smithi076.stdout:1/274: write d0/f2e [6991689,278] 0 2024-03-23T15:44:53.475 INFO:tasks.workunit.client.0.smithi076.stdout:0/527: dwrite d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:53.478 INFO:tasks.workunit.client.0.smithi076.stdout:2/486: write d1/f2f [1636414,50611] 0 2024-03-23T15:44:53.479 INFO:tasks.workunit.client.0.smithi076.stdout:1/275: unlink d0/d35/d52/f53 0 2024-03-23T15:44:53.480 INFO:tasks.workunit.client.0.smithi076.stdout:2/487: symlink d1/d4/d64/la3 0 2024-03-23T15:44:53.480 INFO:tasks.workunit.client.0.smithi076.stdout:7/534: dwrite d8/de/d13/f2f [0,4194304] 0 2024-03-23T15:44:53.480 INFO:tasks.workunit.client.0.smithi076.stdout:7/535: stat d8/fc 0 2024-03-23T15:44:53.481 INFO:tasks.workunit.client.0.smithi076.stdout:7/536: mknod d8/de/d13/d3a/d82/d8f/cb4 0 2024-03-23T15:44:53.483 INFO:tasks.workunit.client.0.smithi076.stdout:7/537: getdents d8 0 2024-03-23T15:44:53.484 INFO:tasks.workunit.client.0.smithi076.stdout:8/377: dwrite d8/d2e/d3e/f49 [0,4194304] 0 2024-03-23T15:44:53.486 INFO:tasks.workunit.client.0.smithi076.stdout:7/538: creat d8/de/d13/d40/d88/fb5 x:0 0 0 2024-03-23T15:44:53.486 INFO:tasks.workunit.client.0.smithi076.stdout:7/539: read d8/de/d13/d40/d5d/f65 [398311,23022] 0 2024-03-23T15:44:53.488 INFO:tasks.workunit.client.0.smithi076.stdout:1/276: dread d0/d2/f37 [0,4194304] 0 2024-03-23T15:44:53.489 INFO:tasks.workunit.client.0.smithi076.stdout:9/443: dwrite d1/d2f/d3a/f84 [0,4194304] 0 2024-03-23T15:44:53.490 INFO:tasks.workunit.client.0.smithi076.stdout:9/444: chown d1/d1f/d46/d4a 193871 1 2024-03-23T15:44:53.490 INFO:tasks.workunit.client.0.smithi076.stdout:1/277: mkdir d0/d35/d40/d47/d55 0 2024-03-23T15:44:53.491 INFO:tasks.workunit.client.0.smithi076.stdout:9/445: creat d1/d2f/d3f/f92 x:0 0 0 2024-03-23T15:44:53.493 INFO:tasks.workunit.client.0.smithi076.stdout:1/278: creat d0/d35/d40/d47/d55/f56 x:0 0 0 2024-03-23T15:44:53.493 INFO:tasks.workunit.client.0.smithi076.stdout:1/279: write d0/f34 [1080282,12497] 0 2024-03-23T15:44:53.493 INFO:tasks.workunit.client.0.smithi076.stdout:9/446: truncate d1/d1f/d29/f54 4016907 0 2024-03-23T15:44:53.493 INFO:tasks.workunit.client.0.smithi076.stdout:9/447: dread - d1/d1f/d29/d2d/f33 zero size 2024-03-23T15:44:53.493 INFO:tasks.workunit.client.0.smithi076.stdout:1/280: creat d0/d3c/f57 x:0 0 0 2024-03-23T15:44:53.495 INFO:tasks.workunit.client.0.smithi076.stdout:1/281: creat d0/d35/d40/d47/d17/d20/f58 x:0 0 0 2024-03-23T15:44:53.497 INFO:tasks.workunit.client.0.smithi076.stdout:1/282: symlink d0/d35/d52/l59 0 2024-03-23T15:44:53.499 INFO:tasks.workunit.client.0.smithi076.stdout:3/401: rename d5/d35/d4c/f5c to d5/d17/d1c/d32/d39/d59/f88 0 2024-03-23T15:44:53.499 INFO:tasks.workunit.client.0.smithi076.stdout:9/448: dread d1/d20/f45 [4194304,4194304] 0 2024-03-23T15:44:53.501 INFO:tasks.workunit.client.0.smithi076.stdout:3/402: unlink d5/d17/d1c/d32/c4f 0 2024-03-23T15:44:53.501 INFO:tasks.workunit.client.0.smithi076.stdout:3/403: write d5/d17/d1c/f7b [490582,103059] 0 2024-03-23T15:44:53.501 INFO:tasks.workunit.client.0.smithi076.stdout:5/395: rename d1a/d33/d3a/l3e to d1a/d7e/l82 0 2024-03-23T15:44:53.502 INFO:tasks.workunit.client.0.smithi076.stdout:0/528: dwrite d2/f1d [0,4194304] 0 2024-03-23T15:44:53.505 INFO:tasks.workunit.client.0.smithi076.stdout:4/533: rename d8/d3a/c99 to d8/d41/c9b 0 2024-03-23T15:44:53.505 INFO:tasks.workunit.client.0.smithi076.stdout:2/488: dwrite d1/d2e/f42 [0,4194304] 0 2024-03-23T15:44:53.506 INFO:tasks.workunit.client.0.smithi076.stdout:2/489: readlink d1/d4/l3a 0 2024-03-23T15:44:53.507 INFO:tasks.workunit.client.0.smithi076.stdout:9/449: rmdir d1/d1f/d29/d2d 39 2024-03-23T15:44:53.509 INFO:tasks.workunit.client.0.smithi076.stdout:2/490: mknod d1/d4/d52/ca4 0 2024-03-23T15:44:53.509 INFO:tasks.workunit.client.0.smithi076.stdout:2/491: chown d1/l4c 1149770846 1 2024-03-23T15:44:53.509 INFO:tasks.workunit.client.0.smithi076.stdout:2/492: dread - d1/d2e/d6a/f9b zero size 2024-03-23T15:44:53.510 INFO:tasks.workunit.client.0.smithi076.stdout:2/493: symlink d1/d4/d64/d7c/la5 0 2024-03-23T15:44:53.510 INFO:tasks.workunit.client.0.smithi076.stdout:2/494: fdatasync d1/d4/d64/d1b/f71 0 2024-03-23T15:44:53.511 INFO:tasks.workunit.client.0.smithi076.stdout:5/396: dread f4 [0,4194304] 0 2024-03-23T15:44:53.511 INFO:tasks.workunit.client.0.smithi076.stdout:5/397: write d1a/f25 [413170,83672] 0 2024-03-23T15:44:53.511 INFO:tasks.workunit.client.0.smithi076.stdout:5/398: write d1a/d39/d5f/f62 [908431,64112] 0 2024-03-23T15:44:53.512 INFO:tasks.workunit.client.0.smithi076.stdout:0/529: creat d2/de/f95 x:0 0 0 2024-03-23T15:44:53.513 INFO:tasks.workunit.client.0.smithi076.stdout:5/399: symlink d1a/d33/d3a/d67/l83 0 2024-03-23T15:44:53.513 INFO:tasks.workunit.client.0.smithi076.stdout:5/400: stat f5 0 2024-03-23T15:44:53.513 INFO:tasks.workunit.client.0.smithi076.stdout:3/404: mkdir d5/d17/d1c/d32/d5d/d89 0 2024-03-23T15:44:53.513 INFO:tasks.workunit.client.0.smithi076.stdout:3/405: chown d5/d17/d1c/d32/d39/d59 87 1 2024-03-23T15:44:53.515 INFO:tasks.workunit.client.0.smithi076.stdout:5/401: mkdir d1a/d33/d3a/d67/d84 0 2024-03-23T15:44:53.515 INFO:tasks.workunit.client.0.smithi076.stdout:4/534: creat d8/d9/f9c x:0 0 0 2024-03-23T15:44:53.516 INFO:tasks.workunit.client.0.smithi076.stdout:5/402: symlink d1a/d20/l85 0 2024-03-23T15:44:53.516 INFO:tasks.workunit.client.0.smithi076.stdout:0/530: symlink d2/de/d10/d3a/d47/d69/d84/l96 0 2024-03-23T15:44:53.518 INFO:tasks.workunit.client.0.smithi076.stdout:0/531: symlink d2/de/d10/l97 0 2024-03-23T15:44:53.519 INFO:tasks.workunit.client.0.smithi076.stdout:0/532: mkdir d2/d1b/d98 0 2024-03-23T15:44:53.522 INFO:tasks.workunit.client.0.smithi076.stdout:3/406: rmdir d5/d17/d1c/d32/d4a 39 2024-03-23T15:44:53.523 INFO:tasks.workunit.client.0.smithi076.stdout:3/407: readlink d5/d17/d1c/d32/d5d/l80 0 2024-03-23T15:44:53.523 INFO:tasks.workunit.client.0.smithi076.stdout:1/283: dwrite d0/d3c/f57 [0,4194304] 0 2024-03-23T15:44:53.527 INFO:tasks.workunit.client.0.smithi076.stdout:7/540: write d8/de/d13/d3a/f89 [2156530,83708] 0 2024-03-23T15:44:53.527 INFO:tasks.workunit.client.0.smithi076.stdout:8/378: dwrite d8/f67 [4194304,4194304] 0 2024-03-23T15:44:53.528 INFO:tasks.workunit.client.0.smithi076.stdout:1/284: symlink d0/d35/d40/d47/l5a 0 2024-03-23T15:44:53.529 INFO:tasks.workunit.client.0.smithi076.stdout:2/495: dwrite d1/d4/d64/d1b/f6c [0,4194304] 0 2024-03-23T15:44:53.530 INFO:tasks.workunit.client.0.smithi076.stdout:2/496: truncate d1/d2e/d6a/f7e 187044 0 2024-03-23T15:44:53.530 INFO:tasks.workunit.client.0.smithi076.stdout:7/541: symlink d8/de/d13/d24/d91/d96/d7a/lb6 0 2024-03-23T15:44:53.530 INFO:tasks.workunit.client.0.smithi076.stdout:9/450: write d1/d1f/f30 [2307230,48955] 0 2024-03-23T15:44:53.531 INFO:tasks.workunit.client.0.smithi076.stdout:8/379: truncate d8/f1e 1497424 0 2024-03-23T15:44:53.531 INFO:tasks.workunit.client.0.smithi076.stdout:8/380: stat d8/d2e/c41 0 2024-03-23T15:44:53.532 INFO:tasks.workunit.client.0.smithi076.stdout:4/535: dread d8/d3a/d69/d73/d76/f78 [0,4194304] 0 2024-03-23T15:44:53.532 INFO:tasks.workunit.client.0.smithi076.stdout:1/285: readlink d0/d35/d52/l4c 0 2024-03-23T15:44:53.533 INFO:tasks.workunit.client.0.smithi076.stdout:1/286: chown d0/d35 153 1 2024-03-23T15:44:53.534 INFO:tasks.workunit.client.0.smithi076.stdout:7/542: creat d8/de/d13/d40/d5d/d3c/d8e/fb7 x:0 0 0 2024-03-23T15:44:53.534 INFO:tasks.workunit.client.0.smithi076.stdout:8/381: getdents d8/d2e/d30/d69 0 2024-03-23T15:44:53.539 INFO:tasks.workunit.client.0.smithi076.stdout:7/543: dread d8/de/d13/d18/f2b [0,4194304] 0 2024-03-23T15:44:53.539 INFO:tasks.workunit.client.0.smithi076.stdout:7/544: symlink d8/d4e/d8d/lb8 0 2024-03-23T15:44:53.540 INFO:tasks.workunit.client.0.smithi076.stdout:4/536: dread d8/f3f [0,4194304] 0 2024-03-23T15:44:53.540 INFO:tasks.workunit.client.0.smithi076.stdout:8/382: truncate d8/f23 682843 0 2024-03-23T15:44:53.541 INFO:tasks.workunit.client.0.smithi076.stdout:7/545: mkdir d8/de/d13/d40/d5d/d3c/d46/db9 0 2024-03-23T15:44:53.544 INFO:tasks.workunit.client.0.smithi076.stdout:5/403: dwrite d1a/d33/f54 [0,4194304] 0 2024-03-23T15:44:53.545 INFO:tasks.workunit.client.0.smithi076.stdout:5/404: write d1a/d39/d65/f6b [968189,85597] 0 2024-03-23T15:44:53.545 INFO:tasks.workunit.client.0.smithi076.stdout:5/405: stat d1a/d2c 0 2024-03-23T15:44:53.547 INFO:tasks.workunit.client.0.smithi076.stdout:5/406: mknod d1a/d33/c86 0 2024-03-23T15:44:53.548 INFO:tasks.workunit.client.0.smithi076.stdout:4/537: mknod d8/d3a/d18/d1b/c9d 0 2024-03-23T15:44:53.551 INFO:tasks.workunit.client.0.smithi076.stdout:5/407: link d1a/f42 d1a/d2c/f87 0 2024-03-23T15:44:53.553 INFO:tasks.workunit.client.0.smithi076.stdout:5/408: creat d1a/d39/d65/f88 x:0 0 0 2024-03-23T15:44:53.553 INFO:tasks.workunit.client.0.smithi076.stdout:0/533: dwrite d2/d8/f4f [0,4194304] 0 2024-03-23T15:44:53.553 INFO:tasks.workunit.client.0.smithi076.stdout:4/538: link d8/d3a/d18/d52/c8b d8/d3a/d69/d73/d85/d88/c9e 0 2024-03-23T15:44:53.560 INFO:tasks.workunit.client.0.smithi076.stdout:3/408: link d5/d17/d1c/d32/d39/f53 d5/d17/d1c/d31/d6b/f8a 0 2024-03-23T15:44:53.565 INFO:tasks.workunit.client.0.smithi076.stdout:0/534: write d2/d8/f53 [2911401,44158] 0 2024-03-23T15:44:53.565 INFO:tasks.workunit.client.0.smithi076.stdout:2/497: dwrite d1/d4/d52/f70 [0,4194304] 0 2024-03-23T15:44:53.565 INFO:tasks.workunit.client.0.smithi076.stdout:0/535: truncate d2/de/d10/f61 5120709 0 2024-03-23T15:44:53.565 INFO:tasks.workunit.client.0.smithi076.stdout:0/536: readlink d2/l16 0 2024-03-23T15:44:53.571 INFO:tasks.workunit.client.0.smithi076.stdout:7/546: dwrite d8/f62 [0,4194304] 0 2024-03-23T15:44:53.571 INFO:tasks.workunit.client.0.smithi076.stdout:7/547: chown d8/de/d13/d40/d88/d5a/ca3 1 1 2024-03-23T15:44:53.571 INFO:tasks.workunit.client.0.smithi076.stdout:7/548: write d8/de/d13/d40/d5d/d3c/d8e/fb7 [707181,105810] 0 2024-03-23T15:44:53.571 INFO:tasks.workunit.client.0.smithi076.stdout:4/539: dread d8/f1a [0,4194304] 0 2024-03-23T15:44:53.571 INFO:tasks.workunit.client.0.smithi076.stdout:0/537: truncate d2/de/d10/d43/f50 511183 0 2024-03-23T15:44:53.571 INFO:tasks.workunit.client.0.smithi076.stdout:2/498: creat d1/d4/d64/d82/fa6 x:0 0 0 2024-03-23T15:44:53.571 INFO:tasks.workunit.client.0.smithi076.stdout:9/451: dwrite d1/d1f/d29/d2d/f6f [0,4194304] 0 2024-03-23T15:44:53.572 INFO:tasks.workunit.client.0.smithi076.stdout:5/409: dwrite d1a/d39/d65/f70 [0,4194304] 0 2024-03-23T15:44:53.574 INFO:tasks.workunit.client.0.smithi076.stdout:2/499: rmdir d1/d2e/d8a 39 2024-03-23T15:44:53.575 INFO:tasks.workunit.client.0.smithi076.stdout:9/452: read d1/d1f/d29/f54 [3777943,119031] 0 2024-03-23T15:44:53.577 INFO:tasks.workunit.client.0.smithi076.stdout:3/409: mknod d5/d17/d1c/d32/d4a/c8b 0 2024-03-23T15:44:53.579 INFO:tasks.workunit.client.0.smithi076.stdout:9/453: truncate d1/d2f/d3a/d53/f61 4201109 0 2024-03-23T15:44:53.580 INFO:tasks.workunit.client.0.smithi076.stdout:7/549: rename f3 to d8/de/d13/d40/d5d/d3c/d46/fba 0 2024-03-23T15:44:53.581 INFO:tasks.workunit.client.0.smithi076.stdout:3/410: mkdir d5/d17/d1c/d32/d70/d8c 0 2024-03-23T15:44:53.582 INFO:tasks.workunit.client.0.smithi076.stdout:3/411: rmdir d5/d17/d1c/d31/d47 39 2024-03-23T15:44:53.582 INFO:tasks.workunit.client.0.smithi076.stdout:5/410: unlink d1a/f76 0 2024-03-23T15:44:53.582 INFO:tasks.workunit.client.0.smithi076.stdout:5/411: fsync d1a/d33/f4d 0 2024-03-23T15:44:53.583 INFO:tasks.workunit.client.0.smithi076.stdout:8/383: link d8/f48 d8/d2e/d30/d64/f77 0 2024-03-23T15:44:53.586 INFO:tasks.workunit.client.0.smithi076.stdout:2/500: unlink d1/d4/l3a 0 2024-03-23T15:44:53.588 INFO:tasks.workunit.client.0.smithi076.stdout:2/501: mknod d1/d4/d64/d1b/ca7 0 2024-03-23T15:44:53.588 INFO:tasks.workunit.client.0.smithi076.stdout:2/502: fdatasync d1/d2e/d8a/f9a 0 2024-03-23T15:44:53.589 INFO:tasks.workunit.client.0.smithi076.stdout:1/287: link d0/d35/d40/d47/f9 d0/d35/d52/f5b 0 2024-03-23T15:44:53.589 INFO:tasks.workunit.client.0.smithi076.stdout:1/288: readlink d0/d35/d40/d47/d17/d20/l2d 0 2024-03-23T15:44:53.590 INFO:tasks.workunit.client.0.smithi076.stdout:2/503: creat d1/d4/d64/d1d/fa8 x:0 0 0 2024-03-23T15:44:53.591 INFO:tasks.workunit.client.0.smithi076.stdout:4/540: dwrite d8/d3a/d18/d1b/f67 [0,4194304] 0 2024-03-23T15:44:53.591 INFO:tasks.workunit.client.0.smithi076.stdout:4/541: fsync d8/d3a/d18/f2b 0 2024-03-23T15:44:53.591 INFO:tasks.workunit.client.0.smithi076.stdout:4/542: chown d8/d3a/d69/d73/l8f 6038944 1 2024-03-23T15:44:53.592 INFO:tasks.workunit.client.0.smithi076.stdout:4/543: write d8/d9/f2d [56469,119082] 0 2024-03-23T15:44:53.594 INFO:tasks.workunit.client.0.smithi076.stdout:4/544: creat d8/f9f x:0 0 0 2024-03-23T15:44:53.594 INFO:tasks.workunit.client.0.smithi076.stdout:4/545: truncate d8/d3a/d18/d1b/f7f 549922 0 2024-03-23T15:44:53.595 INFO:tasks.workunit.client.0.smithi076.stdout:2/504: creat d1/d4/d64/d1b/d32/d79/fa9 x:0 0 0 2024-03-23T15:44:53.595 INFO:tasks.workunit.client.0.smithi076.stdout:2/505: chown d1/d4/l56 1 1 2024-03-23T15:44:53.595 INFO:tasks.workunit.client.0.smithi076.stdout:4/546: creat d8/d41/fa0 x:0 0 0 2024-03-23T15:44:53.596 INFO:tasks.workunit.client.0.smithi076.stdout:6/334: sync 2024-03-23T15:44:53.597 INFO:tasks.workunit.client.0.smithi076.stdout:4/547: symlink d8/d3a/d18/d52/la1 0 2024-03-23T15:44:53.598 INFO:tasks.workunit.client.0.smithi076.stdout:6/335: creat d1/d4/d10/f74 x:0 0 0 2024-03-23T15:44:53.599 INFO:tasks.workunit.client.0.smithi076.stdout:2/506: rename d1/f19 to d1/d4/d52/d6e/faa 0 2024-03-23T15:44:53.599 INFO:tasks.workunit.client.0.smithi076.stdout:2/507: fdatasync d1/d4/db/d3e/f77 0 2024-03-23T15:44:53.599 INFO:tasks.workunit.client.0.smithi076.stdout:4/548: mknod d8/d9/d32/ca2 0 2024-03-23T15:44:53.600 INFO:tasks.workunit.client.0.smithi076.stdout:1/289: dread d0/d2/d1f/f43 [0,4194304] 0 2024-03-23T15:44:53.601 INFO:tasks.workunit.client.0.smithi076.stdout:9/454: dwrite d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:53.601 INFO:tasks.workunit.client.0.smithi076.stdout:4/549: mknod d8/d80/ca3 0 2024-03-23T15:44:53.602 INFO:tasks.workunit.client.0.smithi076.stdout:9/455: symlink d1/d1f/d29/d7f/l93 0 2024-03-23T15:44:53.602 INFO:tasks.workunit.client.0.smithi076.stdout:9/456: readlink d1/d1f/d29/d2d/d37/d47/l76 0 2024-03-23T15:44:53.603 INFO:tasks.workunit.client.0.smithi076.stdout:4/550: symlink d8/d3a/la4 0 2024-03-23T15:44:53.605 INFO:tasks.workunit.client.0.smithi076.stdout:7/550: dwrite f1 [4194304,4194304] 0 2024-03-23T15:44:53.605 INFO:tasks.workunit.client.0.smithi076.stdout:7/551: write d8/de/d48/f84 [292039,50049] 0 2024-03-23T15:44:53.605 INFO:tasks.workunit.client.0.smithi076.stdout:4/551: creat d8/d3a/d18/d1b/fa5 x:0 0 0 2024-03-23T15:44:53.606 INFO:tasks.workunit.client.0.smithi076.stdout:6/336: mknod d1/dd/df/d5b/d62/c75 0 2024-03-23T15:44:53.606 INFO:tasks.workunit.client.0.smithi076.stdout:4/552: write d8/f71 [734622,13211] 0 2024-03-23T15:44:53.606 INFO:tasks.workunit.client.0.smithi076.stdout:4/553: stat d8/d3a/d69/d73/l8f 0 2024-03-23T15:44:53.606 INFO:tasks.workunit.client.0.smithi076.stdout:4/554: fdatasync d8/d3a/d18/d1b/f96 0 2024-03-23T15:44:53.607 INFO:tasks.workunit.client.0.smithi076.stdout:6/337: creat d1/d4/d10/d39/f76 x:0 0 0 2024-03-23T15:44:53.610 INFO:tasks.workunit.client.0.smithi076.stdout:6/338: creat d1/d2/d27/f77 x:0 0 0 2024-03-23T15:44:53.614 INFO:tasks.workunit.client.0.smithi076.stdout:9/457: dread d1/f32 [0,4194304] 0 2024-03-23T15:44:53.615 INFO:tasks.workunit.client.0.smithi076.stdout:4/555: write d8/d3a/d69/d73/d76/f77 [2265388,19269] 0 2024-03-23T15:44:53.617 INFO:tasks.workunit.client.0.smithi076.stdout:8/384: dwrite d8/d26/d33/f3d [0,4194304] 0 2024-03-23T15:44:53.617 INFO:tasks.workunit.client.0.smithi076.stdout:8/385: chown d8/d26/c6d 9018784 1 2024-03-23T15:44:53.618 INFO:tasks.workunit.client.0.smithi076.stdout:8/386: chown d8/fc 102484680 1 2024-03-23T15:44:53.618 INFO:tasks.workunit.client.0.smithi076.stdout:4/556: creat d8/d80/fa6 x:0 0 0 2024-03-23T15:44:53.621 INFO:tasks.workunit.client.0.smithi076.stdout:4/557: creat d8/d3a/d69/d73/fa7 x:0 0 0 2024-03-23T15:44:53.629 INFO:tasks.workunit.client.0.smithi076.stdout:1/290: dwrite d0/d35/f36 [4194304,4194304] 0 2024-03-23T15:44:53.629 INFO:tasks.workunit.client.0.smithi076.stdout:8/387: read d8/f67 [17889202,12759] 0 2024-03-23T15:44:53.630 INFO:tasks.workunit.client.0.smithi076.stdout:5/412: dwrite d1a/d4b/f50 [0,4194304] 0 2024-03-23T15:44:53.630 INFO:tasks.workunit.client.0.smithi076.stdout:5/413: write d1a/d33/f35 [274488,79269] 0 2024-03-23T15:44:53.633 INFO:tasks.workunit.client.0.smithi076.stdout:2/508: dwrite d1/d4/d52/d6e/faa [0,4194304] 0 2024-03-23T15:44:53.637 INFO:tasks.workunit.client.0.smithi076.stdout:4/558: dread d8/f42 [0,4194304] 0 2024-03-23T15:44:53.640 INFO:tasks.workunit.client.0.smithi076.stdout:4/559: truncate d8/d3a/d69/d73/d76/f78 2966996 0 2024-03-23T15:44:53.640 INFO:tasks.workunit.client.0.smithi076.stdout:4/560: truncate d8/d3a/d18/d1b/fa5 814293 0 2024-03-23T15:44:53.643 INFO:tasks.workunit.client.0.smithi076.stdout:4/561: mknod d8/ca8 0 2024-03-23T15:44:53.644 INFO:tasks.workunit.client.0.smithi076.stdout:2/509: write d1/d2e/f3f [1003016,18253] 0 2024-03-23T15:44:53.646 INFO:tasks.workunit.client.0.smithi076.stdout:4/562: mkdir d8/d3a/d69/d73/d85/da9 0 2024-03-23T15:44:53.646 INFO:tasks.workunit.client.0.smithi076.stdout:7/552: dwrite d8/de/f81 [0,4194304] 0 2024-03-23T15:44:53.647 INFO:tasks.workunit.client.0.smithi076.stdout:2/510: mknod d1/d2e/d8a/cab 0 2024-03-23T15:44:53.647 INFO:tasks.workunit.client.0.smithi076.stdout:7/553: symlink d8/de/d13/d3a/d85/db3/lbb 0 2024-03-23T15:44:53.649 INFO:tasks.workunit.client.0.smithi076.stdout:9/458: dread d1/f18 [0,4194304] 0 2024-03-23T15:44:53.649 INFO:tasks.workunit.client.0.smithi076.stdout:9/459: chown d1/d20/l38 2315 1 2024-03-23T15:44:53.649 INFO:tasks.workunit.client.0.smithi076.stdout:9/460: fdatasync d1/d1f/d46/f80 0 2024-03-23T15:44:53.649 INFO:tasks.workunit.client.0.smithi076.stdout:6/339: dwrite d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:53.649 INFO:tasks.workunit.client.0.smithi076.stdout:6/340: chown d1/d4/d10/d6b/l6f 133814 1 2024-03-23T15:44:53.649 INFO:tasks.workunit.client.0.smithi076.stdout:6/341: write d1/dd/df/f66 [348442,64985] 0 2024-03-23T15:44:53.649 INFO:tasks.workunit.client.0.smithi076.stdout:2/511: mkdir d1/d4/d64/d1b/d32/d79/dac 0 2024-03-23T15:44:53.650 INFO:tasks.workunit.client.0.smithi076.stdout:4/563: creat d8/d3a/d18/d75/faa x:0 0 0 2024-03-23T15:44:53.656 INFO:tasks.workunit.client.0.smithi076.stdout:6/342: creat d1/d4/d10/f78 x:0 0 0 2024-03-23T15:44:53.656 INFO:tasks.workunit.client.0.smithi076.stdout:6/343: readlink d1/dd/df/d5b/l40 0 2024-03-23T15:44:53.656 INFO:tasks.workunit.client.0.smithi076.stdout:6/344: stat d1/dd/l58 0 2024-03-23T15:44:53.656 INFO:tasks.workunit.client.0.smithi076.stdout:8/388: link d8/d2e/d30/d3c/c6c d8/d26/d27/d56/c78 0 2024-03-23T15:44:53.657 INFO:tasks.workunit.client.0.smithi076.stdout:2/512: dread d1/d4/d64/d1b/f6c [0,4194304] 0 2024-03-23T15:44:53.658 INFO:tasks.workunit.client.0.smithi076.stdout:8/389: chown d8/d26/d33/f59 1 1 2024-03-23T15:44:53.661 INFO:tasks.workunit.client.0.smithi076.stdout:9/461: mknod d1/d1f/d29/c94 0 2024-03-23T15:44:53.663 INFO:tasks.workunit.client.0.smithi076.stdout:8/390: mknod d8/d2e/d30/c79 0 2024-03-23T15:44:53.663 INFO:tasks.workunit.client.0.smithi076.stdout:7/554: dwrite d8/de/d13/d40/d5d/f29 [0,4194304] 0 2024-03-23T15:44:53.665 INFO:tasks.workunit.client.0.smithi076.stdout:8/391: creat d8/f7a x:0 0 0 2024-03-23T15:44:53.666 INFO:tasks.workunit.client.0.smithi076.stdout:9/462: rmdir d1/d1f 39 2024-03-23T15:44:53.668 INFO:tasks.workunit.client.0.smithi076.stdout:2/513: write d1/d4/db/d15/f89 [3855891,112330] 0 2024-03-23T15:44:53.670 INFO:tasks.workunit.client.0.smithi076.stdout:2/514: truncate d1/d4/d64/d82/f50 681159 0 2024-03-23T15:44:53.670 INFO:tasks.workunit.client.0.smithi076.stdout:2/515: write d1/f5 [5768057,13342] 0 2024-03-23T15:44:53.680 INFO:tasks.workunit.client.0.smithi076.stdout:1/291: rename d0/d35/d40/d47/d17/c1e to d0/d2/c5c 0 2024-03-23T15:44:53.680 INFO:tasks.workunit.client.0.smithi076.stdout:1/292: dread - d0/d2/f48 zero size 2024-03-23T15:44:53.680 INFO:tasks.workunit.client.0.smithi076.stdout:1/293: chown d0/d2/d1f/l24 507137 1 2024-03-23T15:44:53.680 INFO:tasks.workunit.client.0.smithi076.stdout:1/294: truncate d0/d35/d40/d47/d17/f54 434962 0 2024-03-23T15:44:53.681 INFO:tasks.workunit.client.0.smithi076.stdout:5/414: rename d1a/d4b/l4e to d1a/d39/d65/l89 0 2024-03-23T15:44:53.681 INFO:tasks.workunit.client.0.smithi076.stdout:5/415: fdatasync f5 0 2024-03-23T15:44:53.683 INFO:tasks.workunit.client.0.smithi076.stdout:5/416: creat d1a/d33/d3a/f8a x:0 0 0 2024-03-23T15:44:53.684 INFO:tasks.workunit.client.0.smithi076.stdout:6/345: rename d1/d4/c25 to d1/d4/d10/d6b/d72/c79 0 2024-03-23T15:44:53.684 INFO:tasks.workunit.client.0.smithi076.stdout:5/417: mkdir d1a/d33/d36/d8b 0 2024-03-23T15:44:53.686 INFO:tasks.workunit.client.0.smithi076.stdout:7/555: rename d8/de/d13/d40/d5d/la0 to d8/de/d13/d24/lbc 0 2024-03-23T15:44:53.687 INFO:tasks.workunit.client.0.smithi076.stdout:6/346: truncate d1/d2/f22 3157581 0 2024-03-23T15:44:53.689 INFO:tasks.workunit.client.0.smithi076.stdout:6/347: dread d1/d4/f35 [0,4194304] 0 2024-03-23T15:44:53.689 INFO:tasks.workunit.client.0.smithi076.stdout:6/348: chown d1/d4/d10/l4d 2 1 2024-03-23T15:44:53.691 INFO:tasks.workunit.client.0.smithi076.stdout:5/418: write f6 [3533113,8865] 0 2024-03-23T15:44:53.694 INFO:tasks.workunit.client.0.smithi076.stdout:1/295: dwrite d0/d2/f48 [0,4194304] 0 2024-03-23T15:44:53.696 INFO:tasks.workunit.client.0.smithi076.stdout:1/296: getdents d0/d2/d1f 0 2024-03-23T15:44:53.698 INFO:tasks.workunit.client.0.smithi076.stdout:5/419: dread d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:53.701 INFO:tasks.workunit.client.0.smithi076.stdout:8/392: rename d8/d2e/d30/c79 to d8/c7b 0 2024-03-23T15:44:53.703 INFO:tasks.workunit.client.0.smithi076.stdout:8/393: creat d8/d26/f7c x:0 0 0 2024-03-23T15:44:53.704 INFO:tasks.workunit.client.0.smithi076.stdout:8/394: mknod d8/d2e/c7d 0 2024-03-23T15:44:53.705 INFO:tasks.workunit.client.0.smithi076.stdout:8/395: link d8/f11 d8/d6b/f7e 0 2024-03-23T15:44:53.706 INFO:tasks.workunit.client.0.smithi076.stdout:7/556: dwrite d8/de/d13/f15 [0,4194304] 0 2024-03-23T15:44:53.707 INFO:tasks.workunit.client.0.smithi076.stdout:7/557: dread - d8/de/fad zero size 2024-03-23T15:44:53.710 INFO:tasks.workunit.client.0.smithi076.stdout:7/558: creat d8/de/d99/fbd x:0 0 0 2024-03-23T15:44:53.710 INFO:tasks.workunit.client.0.smithi076.stdout:7/559: read d8/de/d13/d3a/f89 [2730156,34057] 0 2024-03-23T15:44:53.710 INFO:tasks.workunit.client.0.smithi076.stdout:7/560: fsync d8/de/d13/d40/d88/fb2 0 2024-03-23T15:44:53.711 INFO:tasks.workunit.client.0.smithi076.stdout:7/561: getdents d8/de/d13/d40/d5d 0 2024-03-23T15:44:53.711 INFO:tasks.workunit.client.0.smithi076.stdout:6/349: dwrite d1/d2/f22 [0,4194304] 0 2024-03-23T15:44:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/3484897323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0107710 msgr2=0x7fe3e0107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/3484897323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0107710 0x7fe3e0107af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009930 tx=0x7fe3d402bc20 comp rx=0 tx=0).stop 2024-03-23T15:44:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/3484897323 shutdown_connections 2024-03-23T15:44:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/3484897323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e01020c0 0x7fe3e010e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/3484897323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3e0101700 0x7fe3e0101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/3484897323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0107710 0x7fe3e0107af0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/3484897323 >> 172.21.15.76:0/3484897323 conn(0x7fe3e00fd1d0 msgr2=0x7fe3e00ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/3484897323 shutdown_connections 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.709+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/3484897323 wait complete. 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 Processor -- start 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 -- start start 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0101700 0x7fe3e019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e01020c0 0x7fe3e019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3e0107710 0x7fe3e0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0198140 con 0x7fe3e0101700 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01982b0 con 0x7fe3e0107710 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e6c4e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0198470 con 0x7fe3e01020c0 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e49ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0101700 0x7fe3e019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e49ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0101700 0x7fe3e019d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48818/0 (socket says 172.21.15.76:48818) 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.710+0000 7fe3e49ea640 1 -- 172.21.15.76:0/2685138906 learned_addr learned my addr 172.21.15.76:0/2685138906 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e51eb640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3e0107710 0x7fe3e0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e49ea640 1 -- 172.21.15.76:0/2685138906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e01020c0 msgr2=0x7fe3e019de30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e01020c0 0x7fe3e019de30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e49ea640 1 -- 172.21.15.76:0/2685138906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3e0107710 msgr2=0x7fe3e0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3e0107710 0x7fe3e0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e49ea640 1 -- 172.21.15.76:0/2685138906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d4009560 con 0x7fe3e0101700 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e51eb640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3e0107710 0x7fe3e0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0101700 0x7fe3e019d8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000c00 tx=0x7fe3d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3ddffb640 1 -- 172.21.15.76:0/2685138906 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d402dea0 con 0x7fe3e0101700 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01986f0 con 0x7fe3e0101700 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3ddffb640 1 -- 172.21.15.76:0/2685138906 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d4004440 con 0x7fe3e0101700 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.711+0000 7fe3ddffb640 1 -- 172.21.15.76:0/2685138906 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4003c90 con 0x7fe3e0101700 2024-03-23T15:44:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.712+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01a4990 con 0x7fe3e0101700 2024-03-23T15:44:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.714+0000 7fe3ddffb640 1 -- 172.21.15.76:0/2685138906 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fe3d4004df0 con 0x7fe3e0101700 2024-03-23T15:44:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.714+0000 7fe3ddffb640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe3d00a22d0 0x7fe3d00a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.715+0000 7fe3ddffb640 1 -- 172.21.15.76:0/2685138906 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe3d40f0710 con 0x7fe3e0101700 2024-03-23T15:44:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.715+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe3d00a22d0 0x7fe3d00a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.721 INFO:tasks.workunit.client.0.smithi076.stdout:6/350: symlink d1/dd/df/l7a 0 2024-03-23T15:44:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.715+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe3d00a22d0 0x7fe3d00a4790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0106c50 tx=0x7fe3cc00b370 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.716+0000 7fe3e982d740 1 Processor -- start 2024-03-23T15:44:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.716+0000 7fe3e982d740 1 -- start start 2024-03-23T15:44:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.716+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d080 0x55697474d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.716+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d9a0 0x55697475aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.716+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 0x55697475d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d080 0x55697474d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d080 0x55697474d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48826/0 (socket says 172.21.15.76:48826) 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e982d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5569747081b0 con 0x55697474d080 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e982d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569746baf80 con 0x55697475b0e0 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e982d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569746b84b0 con 0x55697474d9a0 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 -- 172.21.15.76:0/1756060108 learned_addr learned my addr 172.21.15.76:0/1756060108 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 -- 172.21.15.76:0/1756060108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d9a0 msgr2=0x55697475aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/1756060108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d9a0 0x55697475aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 -- 172.21.15.76:0/1756060108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 msgr2=0x55697475d4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/1756060108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 0x55697475d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.717+0000 7fe3e49ea640 1 -- 172.21.15.76:0/1756060108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556974649810 con 0x55697474d080 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.718+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/1756060108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d080 0x55697474d460 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40b9be0 tx=0x7fe3d40bcc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd3a47200544140 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.718+0000 7fe3caffd640 1 -- 172.21.15.76:0/1756060108 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40c1170 con 0x55697474d080 2024-03-23T15:44:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.718+0000 7fe3caffd640 1 -- 172.21.15.76:0/1756060108 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d40c59b0 con 0x55697474d080 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.718+0000 7fe3caffd640 1 -- 172.21.15.76:0/1756060108 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40c5520 con 0x55697474d080 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 -- 172.21.15.76:0/1756060108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d080 msgr2=0x55697474d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1756060108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d080 0x55697474d460 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40b9be0 tx=0x7fe3d40bcc30 comp rx=0 tx=0).stop 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 -- 172.21.15.76:0/1756060108 shutdown_connections 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1756060108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 0x55697475d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1756060108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d9a0 0x55697475aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1756060108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d080 0x55697474d460 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 -- 172.21.15.76:0/1756060108 >> 172.21.15.76:0/1756060108 conn(0x556974747f00 msgr2=0x5569747482e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 -- 172.21.15.76:0/1756060108 shutdown_connections 2024-03-23T15:44:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.719+0000 7fe3e982d740 1 -- 172.21.15.76:0/1756060108 wait complete. 2024-03-23T15:44:53.725 INFO:tasks.workunit.client.0.smithi076.stdout:9/463: rename d1/d20/f90 to d1/d1f/d29/d7f/f95 0 2024-03-23T15:44:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.720+0000 7fe3e982d740 1 -- start start 2024-03-23T15:44:53.726 INFO:tasks.workunit.client.0.smithi076.stdout:5/420: dwrite d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.720+0000 7fe3e982d740 1 Processor -- start 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x556974758130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974758670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x556974758bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e982d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5569746b3c90 con 0x55697474d9a0 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e982d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556974658c50 con 0x55697474d080 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e982d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556974387250 con 0x55697475b0e0 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e982d740 10 client.0 _ll_register_callbacks cb 0x5569746a7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974758670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974758670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48834/0 (socket says 172.21.15.76:48834) 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3dffff640 1 -- 172.21.15.76:0/2415403322 learned_addr learned my addr 172.21.15.76:0/2415403322 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x556974758130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.721+0000 7fe3e51eb640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x556974758bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3dffff640 1 -- 172.21.15.76:0/2415403322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 msgr2=0x556974758bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x556974758bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3dffff640 1 -- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 msgr2=0x556974758130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x556974758130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3dffff640 1 -- 172.21.15.76:0/2415403322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01a4990 con 0x55697474d9a0 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x556974758130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974758670 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00dea0 tx=0x7fe3cc010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27910bf7290d8648 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:53.729 INFO:tasks.workunit.client.0.smithi076.stdout:5/421: truncate d1a/d33/f51 1105459 0 2024-03-23T15:44:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.722+0000 7fe3bffff640 1 -- 172.21.15.76:0/2415403322 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc01d640 con 0x55697474d9a0 2024-03-23T15:44:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.723+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5569746d8790 con 0x55697474d9a0 2024-03-23T15:44:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.723+0000 7fe3bffff640 1 -- 172.21.15.76:0/2415403322 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe3cc01d7a0 con 0x55697474d9a0 2024-03-23T15:44:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.723+0000 7fe3bffff640 1 -- 172.21.15.76:0/2415403322 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe3cc010e60 con 0x55697474d9a0 2024-03-23T15:44:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.723+0000 7fe3e982d740 10 client.15888 fetch_fsmap learned FSMap version 24 2024-03-23T15:44:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.723+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569746b2310 con 0x55697474d9a0 2024-03-23T15:44:53.731 INFO:tasks.workunit.client.0.smithi076.stdout:5/422: getdents d1a/d39/d5f 0 2024-03-23T15:44:53.731 INFO:tasks.workunit.client.0.smithi076.stdout:9/464: creat d1/d2f/d3a/f96 x:0 0 0 2024-03-23T15:44:53.731 INFO:tasks.workunit.client.0.smithi076.stdout:9/465: chown d1/d2f/d3f/c81 8663503 1 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3bffff640 1 -- 172.21.15.76:0/2415403322 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5569746d8790 con 0x55697474d9a0 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3bffff640 1 -- 172.21.15.76:0/2415403322 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe3cc00fc00 con 0x55697474d9a0 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3e982d740 10 client.15888 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3e982d740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3e982d740 20 client.15888 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x556974751850 0x556974751c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3e982d740 4 client.15888 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.724+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569743b3600 con 0x556974751850 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.725+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x556974751850 0x556974751c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.726+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x556974751850 0x556974751c60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c939f1c6d29f69 server_cookie=38e505f1a43f4b9d in_seq=0 out_seq=0 2024-03-23T15:44:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.726+0000 7fe3bffff640 10 client.15888 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:53.733 INFO:tasks.workunit.client.0.smithi076.stdout:8/396: dwrite d8/f13 [0,4194304] 0 2024-03-23T15:44:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3bffff640 1 -- 172.21.15.76:0/2415403322 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5569743b3600 con 0x556974751850 2024-03-23T15:44:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3bffff640 10 client.15888 handle_command_reply: tid=0 2024-03-23T15:44:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 client.15888 shutdown 2024-03-23T15:44:53.734 INFO:tasks.workunit.client.0.smithi076.stdout:9/466: dread d1/f19 [0,4194304] 0 2024-03-23T15:44:53.734 INFO:tasks.workunit.client.0.smithi076.stdout:9/467: fdatasync d1/f1a 0 2024-03-23T15:44:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 msgr2=0x556974758670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974758670 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00dea0 tx=0x7fe3cc010830 comp rx=0 tx=0).stop 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 shutdown_connections 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x556974758bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974758670 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x556974751850 0x556974751c60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2415403322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x556974758130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 >> 172.21.15.76:0/2415403322 conn(0x556974747f00 msgr2=0x556974749c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3bffff640 0 client.15888 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 shutdown_connections 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 1 -- 172.21.15.76:0/2415403322 wait complete. 2024-03-23T15:44:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.728+0000 7fe3e982d740 20 client.15888 trim_cache size 0 max 16384 2024-03-23T15:44:53.736 INFO:tasks.workunit.client.0.smithi076.stdout:7/562: dwrite d8/de/d13/d40/d5d/d3c/d46/fba [4194304,4194304] 0 2024-03-23T15:44:53.736 INFO:tasks.workunit.client.0.smithi076.stdout:8/397: creat d8/d2e/f7f x:0 0 0 2024-03-23T15:44:53.736 INFO:tasks.workunit.client.0.smithi076.stdout:9/468: creat d1/d20/f97 x:0 0 0 2024-03-23T15:44:53.744 INFO:tasks.workunit.client.0.smithi076.stdout:7/563: dread d8/de/d13/d40/d5d/d3c/f43 [0,4194304] 0 2024-03-23T15:44:53.745 INFO:tasks.workunit.client.0.smithi076.stdout:7/564: chown d8/de/d13/d3a 3094883 1 2024-03-23T15:44:53.747 INFO:tasks.workunit.client.0.smithi076.stdout:6/351: dwrite d1/d4/d10/f74 [0,4194304] 0 2024-03-23T15:44:53.748 INFO:tasks.workunit.client.0.smithi076.stdout:7/565: read d8/de/d13/d40/d5d/f6a [1784024,630] 0 2024-03-23T15:44:53.750 INFO:tasks.workunit.client.0.smithi076.stdout:7/566: mkdir d8/d4e/d8d/dbe 0 2024-03-23T15:44:53.750 INFO:tasks.workunit.client.0.smithi076.stdout:6/352: symlink d1/dd/l7b 0 2024-03-23T15:44:53.750 INFO:tasks.workunit.client.0.smithi076.stdout:7/567: write d8/de/d13/d40/d5d/d3c/f4f [657823,118234] 0 2024-03-23T15:44:53.751 INFO:tasks.workunit.client.0.smithi076.stdout:7/568: truncate d8/de/d13/d40/d88/f2d 1197697 0 2024-03-23T15:44:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 Processor -- start 2024-03-23T15:44:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 -- start start 2024-03-23T15:44:53.758 INFO:tasks.workunit.client.0.smithi076.stdout:0/538: sync 2024-03-23T15:44:53.758 INFO:tasks.workunit.client.0.smithi076.stdout:0/539: write d2/de/d10/d43/f79 [49094,127853] 0 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x55697475a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974752c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x5569747553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556974639660 con 0x55697474d9a0 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569746d8790 con 0x55697474d080 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3e982d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569747678c0 con 0x55697475b0e0 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974752c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.754+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974752c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48838/0 (socket says 172.21.15.76:48838) 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3dffff640 1 -- 172.21.15.76:0/1658478181 learned_addr learned my addr 172.21.15.76:0/1658478181 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3e51eb640 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x5569747553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3dffff640 1 -- 172.21.15.76:0/1658478181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 msgr2=0x5569747553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3dffff640 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x5569747553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3dffff640 1 -- 172.21.15.76:0/1658478181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 msgr2=0x55697475a860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T15:44:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3dffff640 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x55697475a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3dffff640 1 -- 172.21.15.76:0/1658478181 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55697464e810 con 0x55697474d9a0 2024-03-23T15:44:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3e51eb640 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x5569747553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:44:53.760 INFO:tasks.workunit.client.0.smithi076.stdout:1/297: rename d0/d35/d52/l12 to d0/d35/d40/d47/d17/d20/l5d 0 2024-03-23T15:44:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3dffff640 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974752c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00de70 tx=0x7fe3cc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfc62cad182d017 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:53.761 INFO:tasks.workunit.client.0.smithi076.stdout:0/540: mknod d2/d8/d52/c99 0 2024-03-23T15:44:53.761 INFO:tasks.workunit.client.0.smithi076.stdout:8/398: dwrite d8/f50 [0,4194304] 0 2024-03-23T15:44:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.755+0000 7fe3ca7fc640 1 -- 172.21.15.76:0/1658478181 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00da20 con 0x55697474d9a0 2024-03-23T15:44:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.756+0000 7fe3ca7fc640 1 -- 172.21.15.76:0/1658478181 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc01ce70 con 0x55697474d9a0 2024-03-23T15:44:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.756+0000 7fe3ca7fc640 1 -- 172.21.15.76:0/1658478181 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc004a00 con 0x55697474d9a0 2024-03-23T15:44:53.762 INFO:tasks.workunit.client.0.smithi076.stdout:6/353: write d1/dd/d2a/d23/f2d [3674975,70080] 0 2024-03-23T15:44:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 -- 172.21.15.76:0/1658478181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 msgr2=0x556974752c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974752c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00de70 tx=0x7fe3cc005970 comp rx=0 tx=0).stop 2024-03-23T15:44:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 -- 172.21.15.76:0/1658478181 shutdown_connections 2024-03-23T15:44:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697475b0e0 0x5569747553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974752c10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 --2- 172.21.15.76:0/1658478181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697474d080 0x55697475a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.763 INFO:tasks.workunit.client.0.smithi076.stdout:9/469: dwrite d1/d1f/d29/d7f/f95 [0,4194304] 0 2024-03-23T15:44:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 -- 172.21.15.76:0/1658478181 >> 172.21.15.76:0/1658478181 conn(0x556974747f00 msgr2=0x556974752500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 -- 172.21.15.76:0/1658478181 shutdown_connections 2024-03-23T15:44:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 -- 172.21.15.76:0/1658478181 wait complete. 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 -- start start 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.757+0000 7fe3e982d740 1 Processor -- start 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d080 0x556974777590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974777ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e982d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 0x556974772c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e982d740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556974773340 con 0x55697474d9a0 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e982d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569747734b0 con 0x55697475b0e0 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e982d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556974773670 con 0x55697474d080 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974777ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974777ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48850/0 (socket says 172.21.15.76:48850) 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3dffff640 1 -- 172.21.15.76:0/2091343705 learned_addr learned my addr 172.21.15.76:0/2091343705 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:44:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e51eb640 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 0x556974772c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.766 INFO:tasks.workunit.client.0.smithi076.stdout:0/541: mkdir d2/d9a 0 2024-03-23T15:44:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.758+0000 7fe3e982d740 10 client.0 _ll_register_callbacks cb 0x7fe3cc00b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:44:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3dffff640 1 -- 172.21.15.76:0/2091343705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d080 msgr2=0x556974777590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d080 0x556974777590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3dffff640 1 -- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 msgr2=0x556974772c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:44:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 0x556974772c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3dffff640 1 -- 172.21.15.76:0/2091343705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569746b2310 con 0x55697474d9a0 2024-03-23T15:44:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3dffff640 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974777ad0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d720 tx=0x7fe3cc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3147e7a88a54e60e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3bffff640 1 -- 172.21.15.76:0/2091343705 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc03f070 con 0x55697474d9a0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3bffff640 1 -- 172.21.15.76:0/2091343705 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe3cc010d30 con 0x55697474d9a0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d40c59b0 con 0x55697474d9a0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3bffff640 1 -- 172.21.15.76:0/2091343705 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe3cc01c990 con 0x55697474d9a0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3e982d740 10 client.15894 fetch_fsmap learned FSMap version 24 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569746745a0 con 0x55697474d9a0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.759+0000 7fe3bffff640 1 -- 172.21.15.76:0/2091343705 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d40c59b0 con 0x55697474d9a0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3bffff640 1 -- 172.21.15.76:0/2091343705 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe3cc042020 con 0x55697474d9a0 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3e982d740 10 client.15894 fetch_fsmap finished waiting for FSMap version 24 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3e982d740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3e982d740 20 client.15894 populate_metadata read hostname 'smithi076' 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5569747e6c20 0x5569747e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3e982d740 4 client.15894 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:44:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569744041b0 con 0x5569747e6c20 2024-03-23T15:44:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.760+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5569747e6c20 0x5569747e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:44:53.769 INFO:tasks.workunit.client.0.smithi076.stdout:8/399: mkdir d8/d4f/d80 0 2024-03-23T15:44:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.761+0000 7fe3e49ea640 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5569747e6c20 0x5569747e9040 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff08fab750e83748 server_cookie=a5d3b7af46345cdf in_seq=0 out_seq=0 2024-03-23T15:44:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:53.761+0000 7fe3bffff640 10 client.15894 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:53.770 INFO:tasks.workunit.client.0.smithi076.stdout:9/470: rmdir d1/d2f/d3f 39 2024-03-23T15:44:53.770 INFO:tasks.workunit.client.0.smithi076.stdout:9/471: chown d1/d1f/f22 476512107 1 2024-03-23T15:44:53.770 INFO:tasks.workunit.client.0.smithi076.stdout:0/542: creat d2/de/d10/d43/d56/f9b x:0 0 0 2024-03-23T15:44:53.770 INFO:tasks.workunit.client.0.smithi076.stdout:0/543: fsync d2/f1d 0 2024-03-23T15:44:53.770 INFO:tasks.workunit.client.0.smithi076.stdout:8/400: truncate d8/f67 18161863 0 2024-03-23T15:44:53.770 INFO:tasks.workunit.client.0.smithi076.stdout:6/354: dread d1/dd/df/f13 [4194304,4194304] 0 2024-03-23T15:44:53.770 INFO:tasks.workunit.client.0.smithi076.stdout:7/569: dwrite d8/de/d13/d40/d88/d5a/f7f [0,4194304] 0 2024-03-23T15:44:53.771 INFO:tasks.workunit.client.0.smithi076.stdout:9/472: mknod d1/c98 0 2024-03-23T15:44:53.771 INFO:tasks.workunit.client.0.smithi076.stdout:9/473: readlink d1/d2f/d3a/d74/l8a 0 2024-03-23T15:44:53.772 INFO:tasks.workunit.client.0.smithi076.stdout:7/570: unlink d8/de/d13/d40/d5d/f5b 0 2024-03-23T15:44:53.773 INFO:tasks.workunit.client.0.smithi076.stdout:9/474: symlink d1/d1f/d46/d4a/l99 0 2024-03-23T15:44:53.775 INFO:tasks.workunit.client.0.smithi076.stdout:7/571: stat d8/de/l1c 0 2024-03-23T15:44:53.775 INFO:tasks.workunit.client.0.smithi076.stdout:7/572: chown d8/de/d13/d40/d5d/f6a 54 1 2024-03-23T15:44:53.775 INFO:tasks.workunit.client.0.smithi076.stdout:7/573: write d8/de/d13/d40/d5d/d3c/f4f [165241,7046] 0 2024-03-23T15:44:53.775 INFO:tasks.workunit.client.0.smithi076.stdout:7/574: chown d8/de/d48/f84 3768087 1 2024-03-23T15:44:53.776 INFO:tasks.workunit.client.0.smithi076.stdout:9/475: symlink d1/d1f/d46/d4a/l9a 0 2024-03-23T15:44:53.780 INFO:tasks.workunit.client.0.smithi076.stdout:8/401: dread d8/f13 [0,4194304] 0 2024-03-23T15:44:53.782 INFO:tasks.workunit.client.0.smithi076.stdout:7/575: dread d8/de/d13/d40/d5d/d3c/d46/f51 [0,4194304] 0 2024-03-23T15:44:53.782 INFO:tasks.workunit.client.0.smithi076.stdout:8/402: getdents d8/d4f 0 2024-03-23T15:44:53.784 INFO:tasks.workunit.client.0.smithi076.stdout:8/403: creat d8/d2e/d30/d69/f81 x:0 0 0 2024-03-23T15:44:53.785 INFO:tasks.workunit.client.0.smithi076.stdout:8/404: write d8/d26/d33/f59 [376445,48299] 0 2024-03-23T15:44:53.787 INFO:tasks.workunit.client.0.smithi076.stdout:6/355: dwrite d1/f11 [0,4194304] 0 2024-03-23T15:44:53.790 INFO:tasks.workunit.client.0.smithi076.stdout:0/544: dwrite d2/de/d10/d3a/f5c [0,4194304] 0 2024-03-23T15:44:53.790 INFO:tasks.workunit.client.0.smithi076.stdout:9/476: write d1/d2f/d3a/f84 [1715742,64032] 0 2024-03-23T15:44:53.806 INFO:tasks.workunit.client.0.smithi076.stdout:7/576: dwrite d8/de/d13/d3a/f50 [0,4194304] 0 2024-03-23T15:44:53.806 INFO:tasks.workunit.client.0.smithi076.stdout:8/405: dwrite d8/d2e/d30/f6a [0,4194304] 0 2024-03-23T15:44:53.806 INFO:tasks.workunit.client.0.smithi076.stdout:7/577: mkdir d8/de/d13/d3a/dbf 0 2024-03-23T15:44:53.808 INFO:tasks.workunit.client.0.smithi076.stdout:7/578: rename d8/de/d13/d40/d5d/f20 to d8/de/d99/fc0 0 2024-03-23T15:44:53.814 INFO:tasks.workunit.client.0.smithi076.stdout:8/406: dread d8/f57 [0,4194304] 0 2024-03-23T15:44:53.814 INFO:tasks.workunit.client.0.smithi076.stdout:8/407: read d8/f19 [2133038,29469] 0 2024-03-23T15:44:53.815 INFO:tasks.workunit.client.0.smithi076.stdout:8/408: creat d8/d2e/d30/d69/f82 x:0 0 0 2024-03-23T15:44:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:53 smithi192 ceph-mon[36653]: pgmap v49: 97 pgs: 97 active+clean; 569 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 78 MiB/s wr, 2.08k op/s 2024-03-23T15:44:53.825 INFO:tasks.workunit.client.0.smithi076.stdout:7/579: dwrite d8/de/d13/d40/d88/f2c [4194304,4194304] 0 2024-03-23T15:44:53.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:53 smithi076 ceph-mon[32176]: pgmap v49: 97 pgs: 97 active+clean; 569 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 78 MiB/s wr, 2.08k op/s 2024-03-23T15:44:53.839 INFO:tasks.workunit.client.0.smithi076.stdout:7/580: dwrite d8/fd [4194304,4194304] 0 2024-03-23T15:44:53.840 INFO:tasks.workunit.client.0.smithi076.stdout:7/581: mknod d8/de/d13/d40/d5d/d3c/d8e/cc1 0 2024-03-23T15:44:53.842 INFO:tasks.workunit.client.0.smithi076.stdout:7/582: mknod d8/de/d13/d40/d5d/cc2 0 2024-03-23T15:44:53.843 INFO:tasks.workunit.client.0.smithi076.stdout:7/583: rmdir d8/de/d13/d3a/d82 39 2024-03-23T15:44:53.865 INFO:tasks.workunit.client.0.smithi076.stdout:7/584: dwrite d8/de/d13/d40/d5d/f3e [0,4194304] 0 2024-03-23T15:44:53.867 INFO:tasks.workunit.client.0.smithi076.stdout:7/585: getdents d8/de/d13/d3a/d85 0 2024-03-23T15:44:53.869 INFO:tasks.workunit.client.0.smithi076.stdout:7/586: unlink d8/de/d13/d40/d5d/f70 0 2024-03-23T15:44:53.870 INFO:tasks.workunit.client.0.smithi076.stdout:7/587: rmdir d8/de/d13/d3a/d82 39 2024-03-23T15:44:53.871 INFO:tasks.workunit.client.0.smithi076.stdout:7/588: chown d8/de/d13/d40/d5d/d3c/f73 636842546 1 2024-03-23T15:44:53.871 INFO:tasks.workunit.client.0.smithi076.stdout:7/589: creat d8/de/d48/fc3 x:0 0 0 2024-03-23T15:44:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:53 smithi183 ceph-mon[36296]: pgmap v49: 97 pgs: 97 active+clean; 569 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 78 MiB/s wr, 2.08k op/s 2024-03-23T15:44:53.899 INFO:tasks.workunit.client.0.smithi076.stdout:7/590: dwrite d8/de/d13/d40/d5d/d3c/d46/f93 [0,4194304] 0 2024-03-23T15:44:53.899 INFO:tasks.workunit.client.0.smithi076.stdout:7/591: write d8/de/d13/d40/f59 [569345,119813] 0 2024-03-23T15:44:53.900 INFO:tasks.workunit.client.0.smithi076.stdout:7/592: write d8/de/d13/d3a/f50 [5146806,66129] 0 2024-03-23T15:44:53.902 INFO:tasks.workunit.client.0.smithi076.stdout:7/593: truncate d8/de/d13/f2f 2010972 0 2024-03-23T15:44:54.167 INFO:tasks.workunit.client.0.smithi076.stdout:6/356: getdents d1 0 2024-03-23T15:44:54.169 INFO:tasks.workunit.client.0.smithi076.stdout:6/357: mkdir d1/d2/d27/d60/d7c 0 2024-03-23T15:44:54.207 INFO:tasks.workunit.client.0.smithi076.stdout:0/545: mknod d2/de/d10/d3a/d47/d69/d84/c9c 0 2024-03-23T15:44:54.410 INFO:tasks.workunit.client.0.smithi076.stdout:6/358: dwrite d1/d2/f38 [0,4194304] 0 2024-03-23T15:44:54.557 INFO:tasks.workunit.client.0.smithi076.stdout:0/546: dwrite d2/d8/f53 [0,4194304] 0 2024-03-23T15:44:54.557 INFO:tasks.workunit.client.0.smithi076.stdout:0/547: write d2/de/d10/d43/f88 [604441,110747] 0 2024-03-23T15:44:54.811 INFO:tasks.workunit.client.0.smithi076.stdout:0/548: rename d2/de/d10/f67 to d2/d1b/f9d 0 2024-03-23T15:44:54.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:54 smithi192 ceph-mon[36653]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:54.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:54 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:active 2024-03-23T15:44:54.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:54 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:44:54.417+0000 7fa1f8f14700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:54 smithi076 ceph-mon[32176]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:54.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:54 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:active 2024-03-23T15:44:54.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:54 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:54.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:54 smithi183 ceph-mon[36296]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-23T15:44:54.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:54 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] up:active 2024-03-23T15:44:54.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:54 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:54.939 INFO:tasks.workunit.client.0.smithi076.stdout:6/359: creat d1/dd/f7d x:0 0 0 2024-03-23T15:44:54.939 INFO:tasks.workunit.client.0.smithi076.stdout:6/360: write d1/dd/f7d [872745,17428] 0 2024-03-23T15:44:54.939 INFO:tasks.workunit.client.0.smithi076.stdout:6/361: chown d1/d2/f9 30777 1 2024-03-23T15:44:54.940 INFO:tasks.workunit.client.0.smithi076.stdout:0/549: rename d2/f1d to d2/de/d10/d12/f9e 0 2024-03-23T15:44:54.941 INFO:tasks.workunit.client.0.smithi076.stdout:6/362: creat d1/dd/df/d5b/d62/f7e x:0 0 0 2024-03-23T15:44:54.941 INFO:tasks.workunit.client.0.smithi076.stdout:6/363: chown d1/d2/d27 610068 1 2024-03-23T15:44:54.942 INFO:tasks.workunit.client.0.smithi076.stdout:6/364: read d1/dd/f7d [366053,55009] 0 2024-03-23T15:44:54.942 INFO:tasks.workunit.client.0.smithi076.stdout:6/365: chown d1/d4/d10/c3d 7615 1 2024-03-23T15:44:54.942 INFO:tasks.workunit.client.0.smithi076.stdout:6/366: chown d1/d4/l15 12389671 1 2024-03-23T15:44:54.944 INFO:tasks.workunit.client.0.smithi076.stdout:8/409: mkdir d8/d83 0 2024-03-23T15:44:54.950 INFO:tasks.workunit.client.0.smithi076.stdout:8/410: unlink d8/d26/d27/c2c 0 2024-03-23T15:44:54.950 INFO:tasks.workunit.client.0.smithi076.stdout:8/411: fsync d8/f18 0 2024-03-23T15:44:54.950 INFO:tasks.workunit.client.0.smithi076.stdout:0/550: dread d2/f7 [0,4194304] 0 2024-03-23T15:44:54.950 INFO:tasks.workunit.client.0.smithi076.stdout:0/551: fdatasync d2/de/f22 0 2024-03-23T15:44:54.950 INFO:tasks.workunit.client.0.smithi076.stdout:8/412: unlink d8/d2e/d30/c31 0 2024-03-23T15:44:54.951 INFO:tasks.workunit.client.0.smithi076.stdout:8/413: chown d8/d26/d33/c35 5764 1 2024-03-23T15:44:54.952 INFO:tasks.workunit.client.0.smithi076.stdout:0/552: creat d2/d8/d2d/f9f x:0 0 0 2024-03-23T15:44:54.952 INFO:tasks.workunit.client.0.smithi076.stdout:8/414: symlink d8/d2e/d30/d69/l84 0 2024-03-23T15:44:54.953 INFO:tasks.workunit.client.0.smithi076.stdout:8/415: dread d8/d2e/d30/f66 [0,4194304] 0 2024-03-23T15:44:54.953 INFO:tasks.workunit.client.0.smithi076.stdout:0/553: rename d2/d8/d52/c99 to d2/d8/d52/ca0 0 2024-03-23T15:44:54.953 INFO:tasks.workunit.client.0.smithi076.stdout:0/554: chown d2/de/d10/d12/c5b 1103553 1 2024-03-23T15:44:54.954 INFO:tasks.workunit.client.0.smithi076.stdout:0/555: write d2/de/d10/f68 [8389,121389] 0 2024-03-23T15:44:54.954 INFO:tasks.workunit.client.0.smithi076.stdout:0/556: readlink d2/d1b/l55 0 2024-03-23T15:44:54.954 INFO:tasks.workunit.client.0.smithi076.stdout:8/416: symlink d8/d26/d27/l85 0 2024-03-23T15:44:54.956 INFO:tasks.workunit.client.0.smithi076.stdout:8/417: creat d8/d26/d33/f86 x:0 0 0 2024-03-23T15:44:54.958 INFO:tasks.workunit.client.0.smithi076.stdout:9/477: symlink d1/l9b 0 2024-03-23T15:44:54.958 INFO:tasks.workunit.client.0.smithi076.stdout:9/478: readlink d1/d1f/d29/l79 0 2024-03-23T15:44:54.959 INFO:tasks.workunit.client.0.smithi076.stdout:8/418: creat d8/d26/d33/f87 x:0 0 0 2024-03-23T15:44:54.961 INFO:tasks.workunit.client.0.smithi076.stdout:8/419: mknod d8/d2e/d3e/c88 0 2024-03-23T15:44:54.961 INFO:tasks.workunit.client.0.smithi076.stdout:8/420: dread - d8/d26/f7c zero size 2024-03-23T15:44:54.962 INFO:tasks.workunit.client.0.smithi076.stdout:8/421: truncate d8/d2e/d30/f32 574760 0 2024-03-23T15:44:54.963 INFO:tasks.workunit.client.0.smithi076.stdout:9/479: unlink d1/d1f/d29/f54 0 2024-03-23T15:44:54.967 INFO:tasks.workunit.client.0.smithi076.stdout:9/480: unlink d1/d2f/d3f/l68 0 2024-03-23T15:44:54.968 INFO:tasks.workunit.client.0.smithi076.stdout:6/367: symlink d1/d2/l7f 0 2024-03-23T15:44:54.968 INFO:tasks.workunit.client.0.smithi076.stdout:6/368: getdents d1 0 2024-03-23T15:44:54.970 INFO:tasks.workunit.client.0.smithi076.stdout:0/557: symlink d2/de/d10/d3a/d47/la1 0 2024-03-23T15:44:54.974 INFO:tasks.workunit.client.0.smithi076.stdout:0/558: creat d2/de/d10/d3a/d47/d69/d84/fa2 x:0 0 0 2024-03-23T15:44:54.981 INFO:tasks.workunit.client.0.smithi076.stdout:0/559: unlink d2/de/d10/d43/d56/l74 0 2024-03-23T15:44:54.981 INFO:tasks.workunit.client.0.smithi076.stdout:0/560: truncate d2/d8/f28 5176430 0 2024-03-23T15:44:54.988 INFO:tasks.workunit.client.0.smithi076.stdout:9/481: dwrite d1/d2f/f42 [0,4194304] 0 2024-03-23T15:44:54.988 INFO:tasks.workunit.client.0.smithi076.stdout:8/422: dwrite d8/f1e [0,4194304] 0 2024-03-23T15:44:54.988 INFO:tasks.workunit.client.0.smithi076.stdout:8/423: chown d8/d2e/c41 95944 1 2024-03-23T15:44:54.989 INFO:tasks.workunit.client.0.smithi076.stdout:9/482: rmdir d1/d1f/d29/d2d/d37/d47 39 2024-03-23T15:44:54.989 INFO:tasks.workunit.client.0.smithi076.stdout:9/483: stat d1/d1f/d46/d4a/l4f 0 2024-03-23T15:44:54.991 INFO:tasks.workunit.client.0.smithi076.stdout:6/369: dwrite d1/dd/df/d5b/d62/f7e [0,4194304] 0 2024-03-23T15:44:54.991 INFO:tasks.workunit.client.0.smithi076.stdout:6/370: write d1/d2/f9 [966889,11087] 0 2024-03-23T15:44:54.991 INFO:tasks.workunit.client.0.smithi076.stdout:0/561: dread d2/de/d10/d3a/f5c [0,4194304] 0 2024-03-23T15:44:54.995 INFO:tasks.workunit.client.0.smithi076.stdout:3/412: sync 2024-03-23T15:44:54.995 INFO:tasks.workunit.client.0.smithi076.stdout:5/423: sync 2024-03-23T15:44:54.995 INFO:tasks.workunit.client.0.smithi076.stdout:4/564: sync 2024-03-23T15:44:54.995 INFO:tasks.workunit.client.0.smithi076.stdout:7/594: sync 2024-03-23T15:44:54.995 INFO:tasks.workunit.client.0.smithi076.stdout:1/298: sync 2024-03-23T15:44:54.995 INFO:tasks.workunit.client.0.smithi076.stdout:2/516: sync 2024-03-23T15:44:54.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/595: mkdir d8/de/d99/dc4 0 2024-03-23T15:44:54.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/596: write d8/de/d13/d40/d5d/d3c/d46/f93 [4290622,107490] 0 2024-03-23T15:44:54.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/597: chown d8/de/d13/d40/f59 1 1 2024-03-23T15:44:54.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/598: readlink d8/lb0 0 2024-03-23T15:44:55.001 INFO:tasks.workunit.client.0.smithi076.stdout:6/371: mknod d1/d4/d10/d6b/c80 0 2024-03-23T15:44:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.998+0000 7fe3bffff640 1 -- 172.21.15.76:0/2091343705 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569744041b0 con 0x5569747e6c20 2024-03-23T15:44:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.998+0000 7fe3bffff640 10 client.15894 handle_command_reply: tid=0 2024-03-23T15:44:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.998+0000 7fe3e982d740 1 client.15894 shutdown 2024-03-23T15:44:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.998+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 msgr2=0x556974777ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.998+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974777ad0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d720 tx=0x7fe3cc01c8a0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 shutdown_connections 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5569747e6c20 0x5569747e9040 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3bffff640 0 client.15894 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55697475b0e0 0x556974772c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55697474d9a0 0x556974777ad0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 --2- 172.21.15.76:0/2091343705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55697474d080 0x556974777590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 >> 172.21.15.76:0/2091343705 conn(0x556974747f00 msgr2=0x556974746610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 shutdown_connections 2024-03-23T15:44:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 1 -- 172.21.15.76:0/2091343705 wait complete. 2024-03-23T15:44:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:54.999+0000 7fe3e982d740 20 client.15894 trim_cache size 0 max 16384 2024-03-23T15:44:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:44:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:44:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:44:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:44:55.005 INFO:tasks.workunit.client.0.smithi076.stdout:1/299: creat d0/f5e x:0 0 0 2024-03-23T15:44:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe3d00a22d0 msgr2=0x7fe3d00a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe3d00a22d0 0x7fe3d00a4790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0106c50 tx=0x7fe3cc00b370 comp rx=0 tx=0).stop 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0101700 msgr2=0x7fe3e019d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0101700 0x7fe3e019d8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000c00 tx=0x7fe3d4004040 comp rx=0 tx=0).stop 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 shutdown_connections 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe3d00a22d0 0x7fe3d00a4790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3e0107710 0x7fe3e0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e01020c0 0x7fe3e019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 --2- 172.21.15.76:0/2685138906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3e0101700 0x7fe3e019d8f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 >> 172.21.15.76:0/2685138906 conn(0x7fe3e00fd1d0 msgr2=0x7fe3e00fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 shutdown_connections 2024-03-23T15:44:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:44:55.000+0000 7fe3e6c4e640 1 -- 172.21.15.76:0/2685138906 wait complete. 2024-03-23T15:44:55.007 INFO:tasks.workunit.client.0.smithi076.stdout:5/424: dread d1a/d33/d3a/f41 [0,4194304] 0 2024-03-23T15:44:55.007 INFO:tasks.workunit.client.0.smithi076.stdout:6/372: stat d1/d4/d10/d6b/c70 0 2024-03-23T15:44:55.007 INFO:tasks.workunit.client.0.smithi076.stdout:7/599: symlink d8/de/d13/d3a/d85/lc5 0 2024-03-23T15:44:55.008 INFO:tasks.workunit.client.0.smithi076.stdout:5/425: creat d1a/d39/d65/f8c x:0 0 0 2024-03-23T15:44:55.008 INFO:tasks.workunit.client.0.smithi076.stdout:6/373: mkdir d1/d81 0 2024-03-23T15:44:55.009 INFO:tasks.workunit.client.0.smithi076.stdout:7/600: mkdir d8/de/d99/dc4/dc6 0 2024-03-23T15:44:55.009 INFO:tasks.workunit.client.0.smithi076.stdout:7/601: fdatasync d8/de/d13/d3a/f50 0 2024-03-23T15:44:55.014 INFO:tasks.workunit.client.0.smithi076.stdout:5/426: mknod d1a/d39/d65/c8d 0 2024-03-23T15:44:55.014 INFO:tasks.workunit.client.0.smithi076.stdout:6/374: rmdir d1/d4 39 2024-03-23T15:44:55.016 INFO:tasks.workunit.client.0.smithi076.stdout:0/562: dwrite d2/de/d10/d3a/d47/d69/d84/fa2 [0,4194304] 0 2024-03-23T15:44:55.016 INFO:tasks.workunit.client.0.smithi076.stdout:0/563: read d2/de/f1a [381838,106234] 0 2024-03-23T15:44:55.019 INFO:tasks.workunit.client.0.smithi076.stdout:3/413: dwrite d5/d17/d1c/d32/d39/f44 [0,4194304] 0 2024-03-23T15:44:55.021 INFO:tasks.workunit.client.0.smithi076.stdout:5/427: unlink d1a/f3f 0 2024-03-23T15:44:55.021 INFO:tasks.workunit.client.0.smithi076.stdout:5/428: truncate d1a/d33/f73 677460 0 2024-03-23T15:44:55.021 INFO:tasks.workunit.client.0.smithi076.stdout:3/414: stat d5/d17/d1c/d32/d4a/l66 0 2024-03-23T15:44:55.023 INFO:tasks.workunit.client.0.smithi076.stdout:3/415: unlink d5/f11 0 2024-03-23T15:44:55.029 INFO:tasks.workunit.client.0.smithi076.stdout:9/484: dwrite d1/d20/f43 [0,4194304] 0 2024-03-23T15:44:55.029 INFO:tasks.workunit.client.0.smithi076.stdout:5/429: dread d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:55.030 INFO:tasks.workunit.client.0.smithi076.stdout:9/485: truncate d1/f35 754576 0 2024-03-23T15:44:55.032 INFO:tasks.workunit.client.0.smithi076.stdout:9/486: unlink d1/d1f/d29/d7f/l93 0 2024-03-23T15:44:55.033 INFO:tasks.workunit.client.0.smithi076.stdout:9/487: dread d1/f26 [0,4194304] 0 2024-03-23T15:44:55.035 INFO:tasks.workunit.client.0.smithi076.stdout:6/375: dwrite d1/d2/d27/f77 [0,4194304] 0 2024-03-23T15:44:55.035 INFO:tasks.workunit.client.0.smithi076.stdout:1/300: rename d0/d35/d40/d47/d17/l1a to d0/d35/d40/d47/d17/l5f 0 2024-03-23T15:44:55.035 INFO:tasks.workunit.client.0.smithi076.stdout:1/301: fsync d0/d3c/f57 0 2024-03-23T15:44:55.035 INFO:tasks.workunit.client.0.smithi076.stdout:1/302: dread - d0/d35/d40/d47/f4e zero size 2024-03-23T15:44:55.036 INFO:tasks.workunit.client.0.smithi076.stdout:9/488: mkdir d1/d1f/d82/d9c 0 2024-03-23T15:44:55.036 INFO:tasks.workunit.client.0.smithi076.stdout:9/489: write d1/d1f/d46/d4a/f50 [187870,53127] 0 2024-03-23T15:44:55.040 INFO:tasks.workunit.client.0.smithi076.stdout:0/564: dwrite d2/de/d10/d43/f63 [0,4194304] 0 2024-03-23T15:44:55.040 INFO:tasks.workunit.client.0.smithi076.stdout:3/416: dwrite d5/d17/d1c/d31/f64 [0,4194304] 0 2024-03-23T15:44:55.041 INFO:tasks.workunit.client.0.smithi076.stdout:3/417: chown d5/d17/l74 6653938 1 2024-03-23T15:44:55.042 INFO:tasks.workunit.client.0.smithi076.stdout:8/424: rename d8/d2e/d30/d3c to d8/d89 0 2024-03-23T15:44:55.042 INFO:tasks.workunit.client.0.smithi076.stdout:8/425: chown d8/f19 6158723 1 2024-03-23T15:44:55.045 INFO:tasks.workunit.client.0.smithi076.stdout:1/303: dread d0/f26 [0,4194304] 0 2024-03-23T15:44:55.045 INFO:tasks.workunit.client.0.smithi076.stdout:1/304: truncate d0/d35/d40/d47/d17/f54 1237627 0 2024-03-23T15:44:55.046 INFO:tasks.workunit.client.0.smithi076.stdout:3/418: getdents d5/d17/d1c/d32 0 2024-03-23T15:44:55.048 INFO:tasks.workunit.client.0.smithi076.stdout:3/419: creat d5/d17/d1c/d32/d39/d59/d79/f8d x:0 0 0 2024-03-23T15:44:55.048 INFO:tasks.workunit.client.0.smithi076.stdout:8/426: symlink d8/d26/d27/l8a 0 2024-03-23T15:44:55.048 INFO:tasks.workunit.client.0.smithi076.stdout:8/427: write d8/d26/d27/f37 [509958,17150] 0 2024-03-23T15:44:55.049 INFO:tasks.workunit.client.0.smithi076.stdout:5/430: dwrite d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:55.049 INFO:tasks.workunit.client.0.smithi076.stdout:9/490: dread d1/f32 [0,4194304] 0 2024-03-23T15:44:55.049 INFO:tasks.workunit.client.0.smithi076.stdout:9/491: getdents d1/d1f/d82/d9c 0 2024-03-23T15:44:55.049 INFO:tasks.workunit.client.0.smithi076.stdout:5/431: stat d1a/d20/c7f 0 2024-03-23T15:44:55.050 INFO:tasks.workunit.client.0.smithi076.stdout:3/420: rmdir d5/d17/d1c/d32/d5d/d89 0 2024-03-23T15:44:55.051 INFO:tasks.workunit.client.0.smithi076.stdout:9/492: truncate d1/f48 370228 0 2024-03-23T15:44:55.053 INFO:tasks.workunit.client.0.smithi076.stdout:9/493: mknod d1/d2f/d3f/c9d 0 2024-03-23T15:44:55.054 INFO:tasks.workunit.client.0.smithi076.stdout:5/432: dread d1a/d39/d65/f6b [0,4194304] 0 2024-03-23T15:44:55.054 INFO:tasks.workunit.client.0.smithi076.stdout:5/433: chown d1a/d39/c56 5 1 2024-03-23T15:44:55.054 INFO:tasks.workunit.client.0.smithi076.stdout:1/305: dread d0/d35/d40/d47/d17/f2c [0,4194304] 0 2024-03-23T15:44:55.056 INFO:tasks.workunit.client.0.smithi076.stdout:1/306: creat d0/d35/f60 x:0 0 0 2024-03-23T15:44:55.057 INFO:tasks.workunit.client.0.smithi076.stdout:5/434: creat d1a/d33/d3a/d67/d84/f8e x:0 0 0 2024-03-23T15:44:55.057 INFO:tasks.workunit.client.0.smithi076.stdout:0/565: dwrite d2/d8/d52/f5f [4194304,4194304] 0 2024-03-23T15:44:55.058 INFO:tasks.workunit.client.0.smithi076.stdout:5/435: mkdir d1a/d39/d5f/d8f 0 2024-03-23T15:44:55.071 INFO:tasks.workunit.client.0.smithi076.stdout:2/517: rename d1/f14 to d1/d4/db/d15/d29/d7d/fad 0 2024-03-23T15:44:55.072 INFO:tasks.workunit.client.0.smithi076.stdout:2/518: symlink d1/d4/d52/d6e/lae 0 2024-03-23T15:44:55.072 INFO:tasks.workunit.client.0.smithi076.stdout:2/519: chown d1/d4/d52/f70 4368987 1 2024-03-23T15:44:55.076 INFO:tasks.workunit.client.0.smithi076.stdout:5/436: dread f12 [0,4194304] 0 2024-03-23T15:44:55.078 INFO:tasks.workunit.client.0.smithi076.stdout:5/437: symlink d1a/d20/l90 0 2024-03-23T15:44:55.078 INFO:tasks.workunit.client.0.smithi076.stdout:0/566: write d2/d8/fa [1680524,21205] 0 2024-03-23T15:44:55.079 INFO:tasks.workunit.client.0.smithi076.stdout:7/602: rename d8/de/d13/d3a/d82 to d8/de/d13/d3a/dbf/dc7 0 2024-03-23T15:44:55.079 INFO:tasks.workunit.client.0.smithi076.stdout:7/603: write d8/de/d13/d18/f2b [5155719,70849] 0 2024-03-23T15:44:55.080 INFO:tasks.workunit.client.0.smithi076.stdout:7/604: dread - d8/d4e/f87 zero size 2024-03-23T15:44:55.080 INFO:tasks.workunit.client.0.smithi076.stdout:5/438: mknod d1a/d4b/c91 0 2024-03-23T15:44:55.080 INFO:tasks.workunit.client.0.smithi076.stdout:5/439: write d1a/d33/f54 [4668413,112019] 0 2024-03-23T15:44:55.080 INFO:tasks.workunit.client.0.smithi076.stdout:1/307: dwrite d0/f34 [0,4194304] 0 2024-03-23T15:44:55.081 INFO:tasks.workunit.client.0.smithi076.stdout:9/494: rmdir d1/d1f/d46 39 2024-03-23T15:44:55.082 INFO:tasks.workunit.client.0.smithi076.stdout:5/440: getdents d1a/d33/d3a 0 2024-03-23T15:44:55.084 INFO:tasks.workunit.client.0.smithi076.stdout:7/605: creat d8/de/d13/d3a/d85/daf/fc8 x:0 0 0 2024-03-23T15:44:55.084 INFO:tasks.workunit.client.0.smithi076.stdout:7/606: truncate d8/de/d13/f21 2236937 0 2024-03-23T15:44:55.084 INFO:tasks.workunit.client.0.smithi076.stdout:7/607: fsync d8/de/d13/d40/d88/f57 0 2024-03-23T15:44:55.092 INFO:tasks.workunit.client.0.smithi076.stdout:2/520: dwrite d1/d4/f98 [0,4194304] 0 2024-03-23T15:44:55.094 INFO:tasks.workunit.client.0.smithi076.stdout:2/521: truncate d1/d4/d64/f4e 2192779 0 2024-03-23T15:44:55.099 INFO:tasks.workunit.client.0.smithi076.stdout:7/608: read d8/de/d13/d40/d5d/f3e [1159020,1580] 0 2024-03-23T15:44:55.101 INFO:tasks.workunit.client.0.smithi076.stdout:2/522: dread d1/d4/d64/d1b/f63 [0,4194304] 0 2024-03-23T15:44:55.101 INFO:tasks.workunit.client.0.smithi076.stdout:2/523: chown d1/d4/db/d15/d29/l87 0 1 2024-03-23T15:44:55.101 INFO:tasks.workunit.client.0.smithi076.stdout:7/609: creat d8/de/d13/d18/fc9 x:0 0 0 2024-03-23T15:44:55.101 INFO:tasks.workunit.client.0.smithi076.stdout:7/610: stat d8/de/d13/d40/d5d/f34 0 2024-03-23T15:44:55.102 INFO:tasks.workunit.client.0.smithi076.stdout:2/524: symlink d1/d4/laf 0 2024-03-23T15:44:55.103 INFO:tasks.workunit.client.0.smithi076.stdout:2/525: fsync d1/d4/f98 0 2024-03-23T15:44:55.103 INFO:tasks.workunit.client.0.smithi076.stdout:2/526: readlink d1/d4/l8 0 2024-03-23T15:44:55.103 INFO:tasks.workunit.client.0.smithi076.stdout:2/527: chown d1/d4/d64/d1d 10322 1 2024-03-23T15:44:55.103 INFO:tasks.workunit.client.0.smithi076.stdout:9/495: link d1/d1f/d46/d4a/f50 d1/d1f/d82/f9e 0 2024-03-23T15:44:55.104 INFO:tasks.workunit.client.0.smithi076.stdout:2/528: symlink d1/d4/db/lb0 0 2024-03-23T15:44:55.104 INFO:tasks.workunit.client.0.smithi076.stdout:2/529: fsync d1/d2e/d8a/f94 0 2024-03-23T15:44:55.105 INFO:tasks.workunit.client.0.smithi076.stdout:5/441: dwrite f18 [0,4194304] 0 2024-03-23T15:44:55.106 INFO:tasks.workunit.client.0.smithi076.stdout:9/496: unlink d1/d2f/d3a/d64/f69 0 2024-03-23T15:44:55.108 INFO:tasks.workunit.client.0.smithi076.stdout:5/442: creat d1a/f92 x:0 0 0 2024-03-23T15:44:55.108 INFO:tasks.workunit.client.0.smithi076.stdout:5/443: fdatasync d1a/d33/d3a/d67/f6d 0 2024-03-23T15:44:55.108 INFO:tasks.workunit.client.0.smithi076.stdout:9/497: mkdir d1/d2f/d9f 0 2024-03-23T15:44:55.109 INFO:tasks.workunit.client.0.smithi076.stdout:2/530: creat d1/d4/d64/d1d/fb1 x:0 0 0 2024-03-23T15:44:55.110 INFO:tasks.workunit.client.0.smithi076.stdout:5/444: creat d1a/d39/f93 x:0 0 0 2024-03-23T15:44:55.112 INFO:tasks.workunit.client.0.smithi076.stdout:9/498: mkdir d1/d2f/d3f/da0 0 2024-03-23T15:44:55.112 INFO:tasks.workunit.client.0.smithi076.stdout:2/531: unlink d1/c3 0 2024-03-23T15:44:55.112 INFO:tasks.workunit.client.0.smithi076.stdout:9/499: dread - d1/d2f/d3a/d64/f91 zero size 2024-03-23T15:44:55.113 INFO:tasks.workunit.client.0.smithi076.stdout:2/532: truncate d1/d4/d64/d82/d7f/f8c 395590 0 2024-03-23T15:44:55.114 INFO:tasks.workunit.client.0.smithi076.stdout:9/500: mknod d1/d1f/d82/d9c/ca1 0 2024-03-23T15:44:55.114 INFO:tasks.workunit.client.0.smithi076.stdout:2/533: mknod d1/d2e/d6a/cb2 0 2024-03-23T15:44:55.114 INFO:tasks.workunit.client.0.smithi076.stdout:9/501: fsync d1/d20/f25 0 2024-03-23T15:44:55.114 INFO:tasks.workunit.client.0.smithi076.stdout:2/534: readlink d1/d4/d64/d7c/la5 0 2024-03-23T15:44:55.114 INFO:tasks.workunit.client.0.smithi076.stdout:2/535: chown d1/d4/d52/f91 14 1 2024-03-23T15:44:55.115 INFO:tasks.workunit.client.0.smithi076.stdout:2/536: creat d1/d4/db/d15/fb3 x:0 0 0 2024-03-23T15:44:55.116 INFO:tasks.workunit.client.0.smithi076.stdout:9/502: truncate d1/f18 4096930 0 2024-03-23T15:44:55.118 INFO:tasks.workunit.client.0.smithi076.stdout:6/376: rename d1/dd/df/d5b/d62/c75 to d1/d4/d10/d6b/c82 0 2024-03-23T15:44:55.119 INFO:tasks.workunit.client.0.smithi076.stdout:7/611: dwrite d8/de/d13/d24/f55 [0,4194304] 0 2024-03-23T15:44:55.120 INFO:tasks.workunit.client.0.smithi076.stdout:5/445: read d1a/d39/f44 [2103386,129195] 0 2024-03-23T15:44:55.120 INFO:tasks.workunit.client.0.smithi076.stdout:6/377: getdents d1/dd/df 0 2024-03-23T15:44:55.123 INFO:tasks.workunit.client.0.smithi076.stdout:8/428: rename d8/d2e/f7f to d8/d26/d27/d56/f8b 0 2024-03-23T15:44:55.124 INFO:tasks.workunit.client.0.smithi076.stdout:2/537: write d1/d4/f98 [1636010,89667] 0 2024-03-23T15:44:55.127 INFO:tasks.workunit.client.0.smithi076.stdout:3/421: rename d5/d35/d4c/d5a/d48 to d5/d17/d1c/d32/d8e 0 2024-03-23T15:44:55.127 INFO:tasks.workunit.client.0.smithi076.stdout:9/503: write d1/d2f/f42 [2168912,4989] 0 2024-03-23T15:44:55.127 INFO:tasks.workunit.client.0.smithi076.stdout:9/504: write d1/d2f/d3a/d53/f61 [173434,10530] 0 2024-03-23T15:44:55.127 INFO:tasks.workunit.client.0.smithi076.stdout:2/538: link d1/d4/d64/d82/d7f/c9c d1/d2e/cb4 0 2024-03-23T15:44:55.127 INFO:tasks.workunit.client.0.smithi076.stdout:2/539: dread - d1/d4/db/d15/fb3 zero size 2024-03-23T15:44:55.127 INFO:tasks.workunit.client.0.smithi076.stdout:2/540: write d1/d4/db/d15/d29/f61 [4893874,59573] 0 2024-03-23T15:44:55.129 INFO:tasks.workunit.client.0.smithi076.stdout:7/612: write d8/de/d13/d40/d88/d5a/f7f [690197,117428] 0 2024-03-23T15:44:55.129 INFO:tasks.workunit.client.0.smithi076.stdout:7/613: chown d8/de/d13/d3a/dbf/dc7 166879927 1 2024-03-23T15:44:55.130 INFO:tasks.workunit.client.0.smithi076.stdout:3/422: creat d5/d17/d1c/d32/d39/d59/d79/f8f x:0 0 0 2024-03-23T15:44:55.131 INFO:tasks.workunit.client.0.smithi076.stdout:0/567: rename d2/d1b/d98 to d2/de/d10/d3a/d47/d69/da3 0 2024-03-23T15:44:55.132 INFO:tasks.workunit.client.0.smithi076.stdout:2/541: creat d1/d4/d64/d1b/d32/d79/fb5 x:0 0 0 2024-03-23T15:44:55.134 INFO:tasks.workunit.client.0.smithi076.stdout:3/423: rmdir d5/d17/d1c/d32/d8e 39 2024-03-23T15:44:55.134 INFO:tasks.workunit.client.0.smithi076.stdout:2/542: creat d1/d4/d64/d82/fb6 x:0 0 0 2024-03-23T15:44:55.135 INFO:tasks.workunit.client.0.smithi076.stdout:9/505: read d1/d1f/d29/d2d/d37/f66 [4024105,109127] 0 2024-03-23T15:44:55.135 INFO:tasks.workunit.client.0.smithi076.stdout:2/543: truncate d1/d4/db/d15/f55 8400962 0 2024-03-23T15:44:55.136 INFO:tasks.workunit.client.0.smithi076.stdout:3/424: rmdir d5/d17/d1c/d31/d61 39 2024-03-23T15:44:55.136 INFO:tasks.workunit.client.0.smithi076.stdout:3/425: fsync d5/d17/d1c/d32/d39/f44 0 2024-03-23T15:44:55.136 INFO:tasks.workunit.client.0.smithi076.stdout:3/426: chown d5/d17/l58 0 1 2024-03-23T15:44:55.137 INFO:tasks.workunit.client.0.smithi076.stdout:8/429: mkdir d8/d26/d33/d8c 0 2024-03-23T15:44:55.138 INFO:tasks.workunit.client.0.smithi076.stdout:5/446: dwrite d1a/d33/d3a/f74 [0,4194304] 0 2024-03-23T15:44:55.138 INFO:tasks.workunit.client.0.smithi076.stdout:5/447: dread - d1a/f68 zero size 2024-03-23T15:44:55.140 INFO:tasks.workunit.client.0.smithi076.stdout:9/506: mknod d1/d1f/d29/d7f/ca2 0 2024-03-23T15:44:55.141 INFO:tasks.workunit.client.0.smithi076.stdout:1/308: rename d0/d35/f36 to d0/d35/d52/f61 0 2024-03-23T15:44:55.142 INFO:tasks.workunit.client.0.smithi076.stdout:6/378: link d1/dd/f54 d1/d2/d27/d60/d7c/f83 0 2024-03-23T15:44:55.142 INFO:tasks.workunit.client.0.smithi076.stdout:5/448: unlink d1a/d20/c27 0 2024-03-23T15:44:55.142 INFO:tasks.workunit.client.0.smithi076.stdout:5/449: write d1a/d33/d3a/d67/d84/f8e [208153,23341] 0 2024-03-23T15:44:55.144 INFO:tasks.workunit.client.0.smithi076.stdout:9/507: mknod d1/d1f/d29/d2d/ca3 0 2024-03-23T15:44:55.148 INFO:tasks.workunit.client.0.smithi076.stdout:8/430: link d8/d26/f46 d8/d2e/d3e/f8d 0 2024-03-23T15:44:55.148 INFO:tasks.workunit.client.0.smithi076.stdout:1/309: chown d0/d35/d52/l10 3331 1 2024-03-23T15:44:55.148 INFO:tasks.workunit.client.0.smithi076.stdout:9/508: write d1/d1f/d46/d83/f73 [501394,51634] 0 2024-03-23T15:44:55.148 INFO:tasks.workunit.client.0.smithi076.stdout:2/544: dread d1/d4/d52/d6e/faa [0,4194304] 0 2024-03-23T15:44:55.154 INFO:tasks.workunit.client.0.smithi076.stdout:1/310: dread d0/d35/d40/d47/d17/f1d [0,4194304] 0 2024-03-23T15:44:55.154 INFO:tasks.workunit.client.0.smithi076.stdout:2/545: unlink d1/d4/d64/d1b/c4d 0 2024-03-23T15:44:55.154 INFO:tasks.workunit.client.0.smithi076.stdout:2/546: stat d1/d2e/d8a/f94 0 2024-03-23T15:44:55.154 INFO:tasks.workunit.client.0.smithi076.stdout:8/431: dread d8/f11 [0,4194304] 0 2024-03-23T15:44:55.155 INFO:tasks.workunit.client.0.smithi076.stdout:1/311: mkdir d0/d2/d62 0 2024-03-23T15:44:55.157 INFO:tasks.workunit.client.0.smithi076.stdout:2/547: dread d1/d4/d64/d1b/f6c [0,4194304] 0 2024-03-23T15:44:55.158 INFO:tasks.workunit.client.0.smithi076.stdout:2/548: chown d1/d4/db/d15/la2 82206 1 2024-03-23T15:44:55.159 INFO:tasks.workunit.client.0.smithi076.stdout:2/549: link d1/d4/d64/d82/c7b d1/d2e/d6a/cb7 0 2024-03-23T15:44:55.160 INFO:tasks.workunit.client.0.smithi076.stdout:5/450: dread d1a/d39/d65/f70 [0,4194304] 0 2024-03-23T15:44:55.162 INFO:tasks.workunit.client.0.smithi076.stdout:8/432: write f2 [247396,21512] 0 2024-03-23T15:44:55.163 INFO:tasks.workunit.client.0.smithi076.stdout:0/568: dwrite d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:55.163 INFO:tasks.workunit.client.0.smithi076.stdout:7/614: dwrite d8/de/d13/d40/d88/f2d [0,4194304] 0 2024-03-23T15:44:55.163 INFO:tasks.workunit.client.0.smithi076.stdout:7/615: write d8/de/d13/f94 [2212799,2125] 0 2024-03-23T15:44:55.163 INFO:tasks.workunit.client.0.smithi076.stdout:8/433: mkdir d8/d6b/d8e 0 2024-03-23T15:44:55.163 INFO:tasks.workunit.client.0.smithi076.stdout:8/434: chown d8/d2e/c7d 270372 1 2024-03-23T15:44:55.165 INFO:tasks.workunit.client.0.smithi076.stdout:0/569: symlink d2/d1b/d77/la4 0 2024-03-23T15:44:55.165 INFO:tasks.workunit.client.0.smithi076.stdout:0/570: chown d2/de/d71/c80 642 1 2024-03-23T15:44:55.165 INFO:tasks.workunit.client.0.smithi076.stdout:8/435: creat d8/d2e/d30/d69/f8f x:0 0 0 2024-03-23T15:44:55.166 INFO:tasks.workunit.client.0.smithi076.stdout:7/616: mkdir d8/de/d13/d3a/d85/daf/dca 0 2024-03-23T15:44:55.168 INFO:tasks.workunit.client.0.smithi076.stdout:7/617: creat d8/fcb x:0 0 0 2024-03-23T15:44:55.168 INFO:tasks.workunit.client.0.smithi076.stdout:0/571: mknod d2/de/d10/d12/ca5 0 2024-03-23T15:44:55.170 INFO:tasks.workunit.client.0.smithi076.stdout:7/618: mkdir d8/de/d13/d18/dcc 0 2024-03-23T15:44:55.170 INFO:tasks.workunit.client.0.smithi076.stdout:0/572: mkdir d2/de/d10/d12/da6 0 2024-03-23T15:44:55.172 INFO:tasks.workunit.client.0.smithi076.stdout:5/451: dread d1a/f31 [0,4194304] 0 2024-03-23T15:44:55.173 INFO:tasks.workunit.client.0.smithi076.stdout:9/509: rename d1/d1f/d29/d2d/c4e to d1/d2f/d3f/da0/ca4 0 2024-03-23T15:44:55.173 INFO:tasks.workunit.client.0.smithi076.stdout:9/510: dread - d1/d1f/d29/d2d/f33 zero size 2024-03-23T15:44:55.175 INFO:tasks.workunit.client.0.smithi076.stdout:9/511: mkdir d1/d2f/d3a/d74/da5 0 2024-03-23T15:44:55.177 INFO:tasks.workunit.client.0.smithi076.stdout:9/512: creat d1/d2f/d3f/da0/fa6 x:0 0 0 2024-03-23T15:44:55.178 INFO:tasks.workunit.client.0.smithi076.stdout:2/550: dwrite d1/d4/d64/d1b/d32/d79/fb5 [0,4194304] 0 2024-03-23T15:44:55.178 INFO:tasks.workunit.client.0.smithi076.stdout:5/452: dread f12 [0,4194304] 0 2024-03-23T15:44:55.180 INFO:tasks.workunit.client.0.smithi076.stdout:3/427: dwrite d5/d17/d1c/d32/f5f [0,4194304] 0 2024-03-23T15:44:55.182 INFO:tasks.workunit.client.0.smithi076.stdout:1/312: dwrite d0/d35/d40/d47/d17/f49 [4194304,4194304] 0 2024-03-23T15:44:55.185 INFO:tasks.workunit.client.0.smithi076.stdout:5/453: write d1a/d39/f52 [2041220,93163] 0 2024-03-23T15:44:55.185 INFO:tasks.workunit.client.0.smithi076.stdout:1/313: symlink d0/d35/l63 0 2024-03-23T15:44:55.185 INFO:tasks.workunit.client.0.smithi076.stdout:1/314: dread - d0/d35/d40/d47/d17/f30 zero size 2024-03-23T15:44:55.186 INFO:tasks.workunit.client.0.smithi076.stdout:5/454: fsync d1a/f5a 0 2024-03-23T15:44:55.186 INFO:tasks.workunit.client.0.smithi076.stdout:9/513: mkdir d1/d1f/d46/d83/da7 0 2024-03-23T15:44:55.186 INFO:tasks.workunit.client.0.smithi076.stdout:9/514: chown d1/d2f/d3a/d74 7 1 2024-03-23T15:44:55.187 INFO:tasks.workunit.client.0.smithi076.stdout:5/455: symlink d1a/d33/d3a/d80/l94 0 2024-03-23T15:44:55.187 INFO:tasks.workunit.client.0.smithi076.stdout:2/551: symlink d1/d4/d64/d1d/lb8 0 2024-03-23T15:44:55.189 INFO:tasks.workunit.client.0.smithi076.stdout:7/619: dwrite d8/de/d13/d40/d5d/f6a [4194304,4194304] 0 2024-03-23T15:44:55.189 INFO:tasks.workunit.client.0.smithi076.stdout:9/515: creat d1/d20/fa8 x:0 0 0 2024-03-23T15:44:55.190 INFO:tasks.workunit.client.0.smithi076.stdout:5/456: rename d1a/d2c/l38 to d1a/d39/d65/l95 0 2024-03-23T15:44:55.190 INFO:tasks.workunit.client.0.smithi076.stdout:5/457: chown d1a/d33/d3a/f41 3 1 2024-03-23T15:44:55.192 INFO:tasks.workunit.client.0.smithi076.stdout:7/620: truncate d8/de/d13/d24/f55 3444724 0 2024-03-23T15:44:55.196 INFO:tasks.workunit.client.0.smithi076.stdout:7/621: rmdir d8 39 2024-03-23T15:44:55.199 INFO:tasks.workunit.client.0.smithi076.stdout:8/436: link d8/l1b d8/d26/d33/d8c/l90 0 2024-03-23T15:44:55.201 INFO:tasks.workunit.client.0.smithi076.stdout:7/622: creat d8/de/fcd x:0 0 0 2024-03-23T15:44:55.204 INFO:tasks.workunit.client.0.smithi076.stdout:7/623: symlink d8/de/d13/d3a/d85/lce 0 2024-03-23T15:44:55.205 INFO:tasks.workunit.client.0.smithi076.stdout:5/458: write d1a/d39/d65/f70 [283362,10025] 0 2024-03-23T15:44:55.206 INFO:tasks.workunit.client.0.smithi076.stdout:7/624: rmdir d8/de/d13/d40/d5d/d3c/d8e 39 2024-03-23T15:44:55.206 INFO:tasks.workunit.client.0.smithi076.stdout:7/625: readlink d8/de/d13/d3a/dbf/dc7/d8f/l9a 0 2024-03-23T15:44:55.206 INFO:tasks.workunit.client.0.smithi076.stdout:7/626: chown d8/d4e/d8d 22 1 2024-03-23T15:44:55.207 INFO:tasks.workunit.client.0.smithi076.stdout:5/459: mkdir d1a/d39/d5f/d8f/d96 0 2024-03-23T15:44:55.209 INFO:tasks.workunit.client.0.smithi076.stdout:8/437: dread f7 [4194304,4194304] 0 2024-03-23T15:44:55.210 INFO:tasks.workunit.client.0.smithi076.stdout:8/438: fdatasync d8/f16 0 2024-03-23T15:44:55.210 INFO:tasks.workunit.client.0.smithi076.stdout:0/573: dwrite d2/de/d10/d43/f79 [0,4194304] 0 2024-03-23T15:44:55.211 INFO:tasks.workunit.client.0.smithi076.stdout:8/439: mkdir d8/d6b/d91 0 2024-03-23T15:44:55.211 INFO:tasks.workunit.client.0.smithi076.stdout:8/440: chown d8/d2e/d30/f66 0 1 2024-03-23T15:44:55.213 INFO:tasks.workunit.client.0.smithi076.stdout:9/516: dwrite d1/d20/f25 [8388608,4194304] 0 2024-03-23T15:44:55.214 INFO:tasks.workunit.client.0.smithi076.stdout:2/552: dwrite d1/d4/f54 [0,4194304] 0 2024-03-23T15:44:55.214 INFO:tasks.workunit.client.0.smithi076.stdout:8/441: truncate f2 6076541 0 2024-03-23T15:44:55.214 INFO:tasks.workunit.client.0.smithi076.stdout:8/442: stat d8/d89/l58 0 2024-03-23T15:44:55.214 INFO:tasks.workunit.client.0.smithi076.stdout:8/443: chown d8/d2e/d30/d69/f81 119723 1 2024-03-23T15:44:55.215 INFO:tasks.workunit.client.0.smithi076.stdout:9/517: creat d1/d1f/d82/d9c/fa9 x:0 0 0 2024-03-23T15:44:55.215 INFO:tasks.workunit.client.0.smithi076.stdout:5/460: dread d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:55.216 INFO:tasks.workunit.client.0.smithi076.stdout:5/461: write d1a/d4b/f4c [1276979,100174] 0 2024-03-23T15:44:55.216 INFO:tasks.workunit.client.0.smithi076.stdout:7/627: write d8/de/d13/d40/d88/f2d [1669241,54318] 0 2024-03-23T15:44:55.217 INFO:tasks.workunit.client.0.smithi076.stdout:7/628: dread - d8/de/d13/d40/d88/fb2 zero size 2024-03-23T15:44:55.217 INFO:tasks.workunit.client.0.smithi076.stdout:9/518: mkdir d1/d1f/d46/d4a/d5a/daa 0 2024-03-23T15:44:55.217 INFO:tasks.workunit.client.0.smithi076.stdout:9/519: truncate d1/d2f/d3a/d64/f91 545719 0 2024-03-23T15:44:55.218 INFO:tasks.workunit.client.0.smithi076.stdout:7/629: mknod d8/de/d99/dc4/ccf 0 2024-03-23T15:44:55.218 INFO:tasks.workunit.client.0.smithi076.stdout:9/520: symlink d1/lab 0 2024-03-23T15:44:55.218 INFO:tasks.workunit.client.0.smithi076.stdout:9/521: chown d1/d1f/d29/d2d/l5f 6 1 2024-03-23T15:44:55.219 INFO:tasks.workunit.client.0.smithi076.stdout:0/574: dread d2/de/d10/d3a/d47/f59 [0,4194304] 0 2024-03-23T15:44:55.219 INFO:tasks.workunit.client.0.smithi076.stdout:7/630: unlink d8/de/d13/d40/d5d/d3c/d8e/cab 0 2024-03-23T15:44:55.219 INFO:tasks.workunit.client.0.smithi076.stdout:7/631: stat d8/de/d13/d24/f3b 0 2024-03-23T15:44:55.219 INFO:tasks.workunit.client.0.smithi076.stdout:1/315: dwrite d0/d35/d40/d47/f4e [0,4194304] 0 2024-03-23T15:44:55.221 INFO:tasks.workunit.client.0.smithi076.stdout:9/522: dread d1/f26 [0,4194304] 0 2024-03-23T15:44:55.221 INFO:tasks.workunit.client.0.smithi076.stdout:0/575: creat d2/fa7 x:0 0 0 2024-03-23T15:44:55.221 INFO:tasks.workunit.client.0.smithi076.stdout:7/632: creat d8/d4e/fd0 x:0 0 0 2024-03-23T15:44:55.222 INFO:tasks.workunit.client.0.smithi076.stdout:8/444: unlink d8/c7b 0 2024-03-23T15:44:55.222 INFO:tasks.workunit.client.0.smithi076.stdout:7/633: symlink d8/ld1 0 2024-03-23T15:44:55.225 INFO:tasks.workunit.client.0.smithi076.stdout:1/316: creat d0/d35/f64 x:0 0 0 2024-03-23T15:44:55.225 INFO:tasks.workunit.client.0.smithi076.stdout:7/634: truncate d8/de/d13/d40/d5d/f3e 2789750 0 2024-03-23T15:44:55.225 INFO:tasks.workunit.client.0.smithi076.stdout:7/635: readlink d8/de/d13/d18/l86 0 2024-03-23T15:44:55.225 INFO:tasks.workunit.client.0.smithi076.stdout:9/523: symlink d1/d2f/d3a/d74/da5/lac 0 2024-03-23T15:44:55.227 INFO:tasks.workunit.client.0.smithi076.stdout:9/524: getdents d1/d2f/d3a/d74 0 2024-03-23T15:44:55.228 INFO:tasks.workunit.client.0.smithi076.stdout:7/636: dread d8/de/d13/d18/f2b [0,4194304] 0 2024-03-23T15:44:55.229 INFO:tasks.workunit.client.0.smithi076.stdout:1/317: symlink d0/d35/d52/l65 0 2024-03-23T15:44:55.229 INFO:tasks.workunit.client.0.smithi076.stdout:9/525: getdents d1/d20 0 2024-03-23T15:44:55.231 INFO:tasks.workunit.client.0.smithi076.stdout:9/526: readlink d1/d2f/d3f/l6a 0 2024-03-23T15:44:55.232 INFO:tasks.workunit.client.0.smithi076.stdout:9/527: write d1/d2f/d3a/d64/f91 [137226,53773] 0 2024-03-23T15:44:55.236 INFO:tasks.workunit.client.0.smithi076.stdout:5/462: dwrite d1a/d39/d65/f70 [0,4194304] 0 2024-03-23T15:44:55.236 INFO:tasks.workunit.client.0.smithi076.stdout:9/528: write d1/d1f/d46/d83/f73 [1405110,42389] 0 2024-03-23T15:44:55.237 INFO:tasks.workunit.client.0.smithi076.stdout:5/463: creat d1a/d33/d3a/d67/f97 x:0 0 0 2024-03-23T15:44:55.239 INFO:tasks.workunit.client.0.smithi076.stdout:7/637: mkdir d8/de/d13/d3a/dd2 0 2024-03-23T15:44:55.239 INFO:tasks.workunit.client.0.smithi076.stdout:7/638: fdatasync d8/de/d13/d40/d5d/d3c/d46/f51 0 2024-03-23T15:44:55.239 INFO:tasks.workunit.client.0.smithi076.stdout:7/639: fsync f1 0 2024-03-23T15:44:55.242 INFO:tasks.workunit.client.0.smithi076.stdout:5/464: creat d1a/f98 x:0 0 0 2024-03-23T15:44:55.242 INFO:tasks.workunit.client.0.smithi076.stdout:5/465: dread - d1a/d33/d3a/d67/f97 zero size 2024-03-23T15:44:55.242 INFO:tasks.workunit.client.0.smithi076.stdout:2/553: dwrite d1/d4/d64/d1d/fb1 [0,4194304] 0 2024-03-23T15:44:55.242 INFO:tasks.workunit.client.0.smithi076.stdout:2/554: fsync d1/d4/f8d 0 2024-03-23T15:44:55.245 INFO:tasks.workunit.client.0.smithi076.stdout:5/466: rename d1a/d4b/c91 to d1a/d33/d3a/d67/c99 0 2024-03-23T15:44:55.245 INFO:tasks.workunit.client.0.smithi076.stdout:7/640: truncate d8/de/d13/f21 1141738 0 2024-03-23T15:44:55.249 INFO:tasks.workunit.client.0.smithi076.stdout:9/529: dread d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:55.249 INFO:tasks.workunit.client.0.smithi076.stdout:7/641: dread d8/de/d13/d40/d88/f2d [0,4194304] 0 2024-03-23T15:44:55.249 INFO:tasks.workunit.client.0.smithi076.stdout:7/642: dread - d8/de/d13/d18/fc9 zero size 2024-03-23T15:44:55.249 INFO:tasks.workunit.client.0.smithi076.stdout:7/643: truncate d8/de/fad 889554 0 2024-03-23T15:44:55.251 INFO:tasks.workunit.client.0.smithi076.stdout:7/644: creat d8/de/d13/d40/d5d/d3c/d8e/fd3 x:0 0 0 2024-03-23T15:44:55.252 INFO:tasks.workunit.client.0.smithi076.stdout:7/645: mknod d8/de/d13/d24/d91/cd4 0 2024-03-23T15:44:55.253 INFO:tasks.workunit.client.0.smithi076.stdout:9/530: rename d1/d2f/d3a/d65 to d1/d1f/d46/d83/dad 0 2024-03-23T15:44:55.253 INFO:tasks.workunit.client.0.smithi076.stdout:8/445: dwrite d8/d2e/d3e/f8d [0,4194304] 0 2024-03-23T15:44:55.255 INFO:tasks.workunit.client.0.smithi076.stdout:7/646: link d8/de/d13/d40/d5d/d3c/c69 d8/d4e/d98/cd5 0 2024-03-23T15:44:55.255 INFO:tasks.workunit.client.0.smithi076.stdout:7/647: fdatasync d8/fd 0 2024-03-23T15:44:55.255 INFO:tasks.workunit.client.0.smithi076.stdout:9/531: truncate d1/f12 2687165 0 2024-03-23T15:44:55.255 INFO:tasks.workunit.client.0.smithi076.stdout:0/576: dwrite d2/de/d10/d12/f9e [4194304,4194304] 0 2024-03-23T15:44:55.257 INFO:tasks.workunit.client.0.smithi076.stdout:7/648: creat d8/de/d99/dc4/fd6 x:0 0 0 2024-03-23T15:44:55.258 INFO:tasks.workunit.client.0.smithi076.stdout:8/446: rename d8/f13 to d8/d6b/d8e/f92 0 2024-03-23T15:44:55.258 INFO:tasks.workunit.client.0.smithi076.stdout:7/649: getdents d8/de/d13/d18 0 2024-03-23T15:44:55.260 INFO:tasks.workunit.client.0.smithi076.stdout:8/447: mkdir d8/d2e/d30/d69/d93 0 2024-03-23T15:44:55.260 INFO:tasks.workunit.client.0.smithi076.stdout:8/448: write d8/d26/f7c [879746,98755] 0 2024-03-23T15:44:55.260 INFO:tasks.workunit.client.0.smithi076.stdout:2/555: dwrite d1/d4/f5f [4194304,4194304] 0 2024-03-23T15:44:55.261 INFO:tasks.workunit.client.0.smithi076.stdout:9/532: getdents d1/d2f/d3a 0 2024-03-23T15:44:55.261 INFO:tasks.workunit.client.0.smithi076.stdout:8/449: symlink d8/d2e/d30/d64/l94 0 2024-03-23T15:44:55.262 INFO:tasks.workunit.client.0.smithi076.stdout:9/533: creat d1/d1f/d29/d2d/d37/d47/fae x:0 0 0 2024-03-23T15:44:55.263 INFO:tasks.workunit.client.0.smithi076.stdout:8/450: dread d8/d6b/f7e [0,4194304] 0 2024-03-23T15:44:55.263 INFO:tasks.workunit.client.0.smithi076.stdout:2/556: rename d1/d2e/d6a/f7e to d1/d4/fb9 0 2024-03-23T15:44:55.265 INFO:tasks.workunit.client.0.smithi076.stdout:0/577: creat d2/d1b/fa8 x:0 0 0 2024-03-23T15:44:55.266 INFO:tasks.workunit.client.0.smithi076.stdout:7/650: dread d8/de/d13/f94 [0,4194304] 0 2024-03-23T15:44:55.266 INFO:tasks.workunit.client.0.smithi076.stdout:2/557: creat d1/d2e/d8a/fba x:0 0 0 2024-03-23T15:44:55.269 INFO:tasks.workunit.client.0.smithi076.stdout:8/451: rename d8/fc to d8/d83/f95 0 2024-03-23T15:44:55.271 INFO:tasks.workunit.client.0.smithi076.stdout:7/651: rmdir d8/de/d13 39 2024-03-23T15:44:55.273 INFO:tasks.workunit.client.0.smithi076.stdout:5/467: dwrite f18 [0,4194304] 0 2024-03-23T15:44:55.273 INFO:tasks.workunit.client.0.smithi076.stdout:0/578: rename d2/de/d10/d12/f6a to d2/fa9 0 2024-03-23T15:44:55.275 INFO:tasks.workunit.client.0.smithi076.stdout:5/468: creat d1a/d39/d5f/d8f/f9a x:0 0 0 2024-03-23T15:44:55.275 INFO:tasks.workunit.client.0.smithi076.stdout:7/652: creat d8/de/d13/d3a/d85/fd7 x:0 0 0 2024-03-23T15:44:55.275 INFO:tasks.workunit.client.0.smithi076.stdout:7/653: chown d8/de/d13/d18/dcc 28974004 1 2024-03-23T15:44:55.276 INFO:tasks.workunit.client.0.smithi076.stdout:5/469: creat d1a/d39/d5f/f9b x:0 0 0 2024-03-23T15:44:55.281 INFO:tasks.workunit.client.0.smithi076.stdout:5/470: unlink f6 0 2024-03-23T15:44:55.282 INFO:tasks.workunit.client.0.smithi076.stdout:7/654: dread d8/de/d13/d40/d5d/d3c/d46/f93 [0,4194304] 0 2024-03-23T15:44:55.284 INFO:tasks.workunit.client.0.smithi076.stdout:7/655: mkdir d8/de/d48/dd8 0 2024-03-23T15:44:55.284 INFO:tasks.workunit.client.0.smithi076.stdout:5/471: symlink d1a/d33/d3a/l9c 0 2024-03-23T15:44:55.284 INFO:tasks.workunit.client.0.smithi076.stdout:9/534: dwrite d1/d2f/d3a/f96 [0,4194304] 0 2024-03-23T15:44:55.286 INFO:tasks.workunit.client.0.smithi076.stdout:5/472: symlink d1a/d39/d5f/l9d 0 2024-03-23T15:44:55.286 INFO:tasks.workunit.client.0.smithi076.stdout:5/473: truncate d1a/f68 712015 0 2024-03-23T15:44:55.286 INFO:tasks.workunit.client.0.smithi076.stdout:0/579: write d2/de/d10/d43/f79 [3039464,74378] 0 2024-03-23T15:44:55.286 INFO:tasks.workunit.client.0.smithi076.stdout:0/580: fdatasync d2/d8/d52/f64 0 2024-03-23T15:44:55.286 INFO:tasks.workunit.client.0.smithi076.stdout:2/558: dwrite d1/d4/d64/d1b/f1c [0,4194304] 0 2024-03-23T15:44:55.286 INFO:tasks.workunit.client.0.smithi076.stdout:7/656: unlink d8/l9 0 2024-03-23T15:44:55.286 INFO:tasks.workunit.client.0.smithi076.stdout:5/474: mknod d1a/d39/c9e 0 2024-03-23T15:44:55.288 INFO:tasks.workunit.client.0.smithi076.stdout:5/475: mknod d1a/d2c/c9f 0 2024-03-23T15:44:55.288 INFO:tasks.workunit.client.0.smithi076.stdout:5/476: write d1a/d39/d65/f70 [4832350,10021] 0 2024-03-23T15:44:55.288 INFO:tasks.workunit.client.0.smithi076.stdout:5/477: truncate d1a/d33/f35 1235944 0 2024-03-23T15:44:55.288 INFO:tasks.workunit.client.0.smithi076.stdout:0/581: creat d2/de/d10/d3a/d47/d69/faa x:0 0 0 2024-03-23T15:44:55.288 INFO:tasks.workunit.client.0.smithi076.stdout:2/559: creat d1/d4/d52/fbb x:0 0 0 2024-03-23T15:44:55.289 INFO:tasks.workunit.client.0.smithi076.stdout:5/478: creat d1a/d39/d5f/d8f/d96/fa0 x:0 0 0 2024-03-23T15:44:55.291 INFO:tasks.workunit.client.0.smithi076.stdout:2/560: getdents d1/d2e/d8a 0 2024-03-23T15:44:55.297 INFO:tasks.workunit.client.0.smithi076.stdout:2/561: mkdir d1/d4/d64/d1d/dbc 0 2024-03-23T15:44:55.297 INFO:tasks.workunit.client.0.smithi076.stdout:2/562: write d1/d4/d64/f48 [835155,62798] 0 2024-03-23T15:44:55.299 INFO:tasks.workunit.client.0.smithi076.stdout:2/563: mkdir d1/d4/db/d15/d29/d7d/dbd 0 2024-03-23T15:44:55.299 INFO:tasks.workunit.client.0.smithi076.stdout:2/564: stat d1/d4/d64/d1b/d32 0 2024-03-23T15:44:55.303 INFO:tasks.workunit.client.0.smithi076.stdout:5/479: dwrite d1a/d20/f34 [0,4194304] 0 2024-03-23T15:44:55.303 INFO:tasks.workunit.client.0.smithi076.stdout:5/480: write d1a/d33/f51 [212957,8068] 0 2024-03-23T15:44:55.305 INFO:tasks.workunit.client.0.smithi076.stdout:9/535: dwrite d1/d2f/d3a/d64/f85 [0,4194304] 0 2024-03-23T15:44:55.307 INFO:tasks.workunit.client.0.smithi076.stdout:5/481: mkdir d1a/d33/d36/da1 0 2024-03-23T15:44:55.307 INFO:tasks.workunit.client.0.smithi076.stdout:9/536: symlink d1/d1f/laf 0 2024-03-23T15:44:55.307 INFO:tasks.workunit.client.0.smithi076.stdout:9/537: chown d1/d1f/d46/d83/dad 0 1 2024-03-23T15:44:55.310 INFO:tasks.workunit.client.0.smithi076.stdout:9/538: creat d1/d1f/d29/d2d/d37/fb0 x:0 0 0 2024-03-23T15:44:55.316 INFO:tasks.workunit.client.0.smithi076.stdout:9/539: dread d1/d1f/f22 [0,4194304] 0 2024-03-23T15:44:55.320 INFO:tasks.workunit.client.0.smithi076.stdout:5/482: dwrite d1a/d33/f35 [0,4194304] 0 2024-03-23T15:44:55.325 INFO:tasks.workunit.client.0.smithi076.stdout:9/540: truncate d1/d20/f43 2107725 0 2024-03-23T15:44:55.330 INFO:tasks.workunit.client.0.smithi076.stdout:9/541: dread d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:55.330 INFO:tasks.workunit.client.0.smithi076.stdout:8/452: rename d8/d26/f7c to d8/d2e/d30/f96 0 2024-03-23T15:44:55.332 INFO:tasks.workunit.client.0.smithi076.stdout:8/453: dread d8/f23 [0,4194304] 0 2024-03-23T15:44:55.335 INFO:tasks.workunit.client.0.smithi076.stdout:9/542: dread d1/d2f/d3a/d64/f91 [0,4194304] 0 2024-03-23T15:44:55.335 INFO:tasks.workunit.client.0.smithi076.stdout:9/543: chown d1/d2f/c87 4140245 1 2024-03-23T15:44:55.335 INFO:tasks.workunit.client.0.smithi076.stdout:5/483: dwrite d1a/d39/f93 [0,4194304] 0 2024-03-23T15:44:55.335 INFO:tasks.workunit.client.0.smithi076.stdout:5/484: fdatasync d1a/d4b/f78 0 2024-03-23T15:44:55.335 INFO:tasks.workunit.client.0.smithi076.stdout:5/485: chown d1a/d39/d65/c8d 2644 1 2024-03-23T15:44:55.337 INFO:tasks.workunit.client.0.smithi076.stdout:9/544: dread d1/d1f/d29/d2d/d37/f66 [0,4194304] 0 2024-03-23T15:44:55.342 INFO:tasks.workunit.client.0.smithi076.stdout:5/486: creat d1a/d33/d36/fa2 x:0 0 0 2024-03-23T15:44:55.343 INFO:tasks.workunit.client.0.smithi076.stdout:5/487: rmdir d1a/d39/d5f 39 2024-03-23T15:44:55.344 INFO:tasks.workunit.client.0.smithi076.stdout:5/488: symlink d1a/d7e/la3 0 2024-03-23T15:44:55.347 INFO:tasks.workunit.client.0.smithi076.stdout:7/657: rename d8/de/d13/d3a/d85/db3 to d8/de/d13/d40/d88/d5a/dd9 0 2024-03-23T15:44:55.349 INFO:tasks.workunit.client.0.smithi076.stdout:0/582: rename d2/de/d10/f5d to d2/de/d10/d3a/d47/fab 0 2024-03-23T15:44:55.350 INFO:tasks.workunit.client.0.smithi076.stdout:0/583: truncate d2/d8/d2d/f91 727592 0 2024-03-23T15:44:55.351 INFO:tasks.workunit.client.0.smithi076.stdout:7/658: symlink d8/de/d13/d3a/dbf/dc7/lda 0 2024-03-23T15:44:55.351 INFO:tasks.workunit.client.0.smithi076.stdout:7/659: fdatasync d8/de/d13/d40/d5d/f3e 0 2024-03-23T15:44:55.351 INFO:tasks.workunit.client.0.smithi076.stdout:7/660: stat d8/de/d13/d40/d5d/d3c/d46/fba 0 2024-03-23T15:44:55.351 INFO:tasks.workunit.client.0.smithi076.stdout:7/661: truncate d8/de/d13/d40/d88/fb2 752323 0 2024-03-23T15:44:55.352 INFO:tasks.workunit.client.0.smithi076.stdout:0/584: getdents d2/d8/d52 0 2024-03-23T15:44:55.352 INFO:tasks.workunit.client.0.smithi076.stdout:7/662: creat d8/d4e/d8d/dbe/fdb x:0 0 0 2024-03-23T15:44:55.353 INFO:tasks.workunit.client.0.smithi076.stdout:7/663: write d8/de/d99/fc0 [2421227,42306] 0 2024-03-23T15:44:55.353 INFO:tasks.workunit.client.0.smithi076.stdout:7/664: write d8/d4e/f87 [455107,95820] 0 2024-03-23T15:44:55.353 INFO:tasks.workunit.client.0.smithi076.stdout:0/585: mknod d2/d8/d2d/cac 0 2024-03-23T15:44:55.353 INFO:tasks.workunit.client.0.smithi076.stdout:9/545: dwrite d1/d1f/d29/d7f/f95 [0,4194304] 0 2024-03-23T15:44:55.354 INFO:tasks.workunit.client.0.smithi076.stdout:0/586: creat d2/fad x:0 0 0 2024-03-23T15:44:55.356 INFO:tasks.workunit.client.0.smithi076.stdout:9/546: truncate d1/d2f/d3a/f96 4126440 0 2024-03-23T15:44:55.357 INFO:tasks.workunit.client.0.smithi076.stdout:9/547: symlink d1/d2f/d3a/d53/lb1 0 2024-03-23T15:44:55.357 INFO:tasks.workunit.client.0.smithi076.stdout:9/548: chown d1/d1f/d82/f9e 12731 1 2024-03-23T15:44:55.359 INFO:tasks.workunit.client.0.smithi076.stdout:9/549: link d1/ca d1/d2f/d3a/cb2 0 2024-03-23T15:44:55.359 INFO:tasks.workunit.client.0.smithi076.stdout:9/550: fdatasync d1/d1f/d29/d2d/d37/f59 0 2024-03-23T15:44:55.367 INFO:tasks.workunit.client.0.smithi076.stdout:9/551: read d1/f15 [854195,100809] 0 2024-03-23T15:44:55.368 INFO:tasks.workunit.client.0.smithi076.stdout:9/552: stat d1/ca 0 2024-03-23T15:44:55.370 INFO:tasks.workunit.client.0.smithi076.stdout:0/587: dwrite d2/d8/d32/f48 [0,4194304] 0 2024-03-23T15:44:55.370 INFO:tasks.workunit.client.0.smithi076.stdout:7/665: dwrite d8/d4e/fd0 [0,4194304] 0 2024-03-23T15:44:55.370 INFO:tasks.workunit.client.0.smithi076.stdout:7/666: stat d8/de/d13/d40/d88 0 2024-03-23T15:44:55.373 INFO:tasks.workunit.client.0.smithi076.stdout:0/588: unlink d2/de/d10/d43/l7d 0 2024-03-23T15:44:55.373 INFO:tasks.workunit.client.0.smithi076.stdout:0/589: fsync d2/de/f14 0 2024-03-23T15:44:55.374 INFO:tasks.workunit.client.0.smithi076.stdout:7/667: link d8/de/fcd d8/de/d13/d40/d5d/d3c/d46/fdc 0 2024-03-23T15:44:55.375 INFO:tasks.workunit.client.0.smithi076.stdout:7/668: rmdir d8/de/d13/d24 39 2024-03-23T15:44:55.376 INFO:tasks.workunit.client.0.smithi076.stdout:0/590: unlink d2/f9 0 2024-03-23T15:44:55.376 INFO:tasks.workunit.client.0.smithi076.stdout:2/565: rename d1/ld to d1/d4/d64/d7c/lbe 0 2024-03-23T15:44:55.377 INFO:tasks.workunit.client.0.smithi076.stdout:9/553: write d1/d1f/d29/d7f/f95 [3902961,33225] 0 2024-03-23T15:44:55.377 INFO:tasks.workunit.client.0.smithi076.stdout:9/554: truncate d1/d20/f43 2216862 0 2024-03-23T15:44:55.378 INFO:tasks.workunit.client.0.smithi076.stdout:2/566: mkdir d1/dbf 0 2024-03-23T15:44:55.378 INFO:tasks.workunit.client.0.smithi076.stdout:5/489: rename d1a/d33/d3a/d67/c99 to d1a/d2c/ca4 0 2024-03-23T15:44:55.380 INFO:tasks.workunit.client.0.smithi076.stdout:5/490: creat d1a/d33/d3a/fa5 x:0 0 0 2024-03-23T15:44:55.382 INFO:tasks.workunit.client.0.smithi076.stdout:9/555: write d1/f19 [1548747,10931] 0 2024-03-23T15:44:55.383 INFO:tasks.workunit.client.0.smithi076.stdout:2/567: creat d1/fc0 x:0 0 0 2024-03-23T15:44:55.383 INFO:tasks.workunit.client.0.smithi076.stdout:2/568: write d1/f5 [3168402,129054] 0 2024-03-23T15:44:55.385 INFO:tasks.workunit.client.0.smithi076.stdout:9/556: creat d1/d2f/d3a/d74/fb3 x:0 0 0 2024-03-23T15:44:55.387 INFO:tasks.workunit.client.0.smithi076.stdout:9/557: mknod d1/d1f/d46/d83/dad/cb4 0 2024-03-23T15:44:55.389 INFO:tasks.workunit.client.0.smithi076.stdout:5/491: write d1a/d2c/f87 [3215012,70333] 0 2024-03-23T15:44:55.389 INFO:tasks.workunit.client.0.smithi076.stdout:0/591: getdents d2/d8/d32 0 2024-03-23T15:44:55.390 INFO:tasks.workunit.client.0.smithi076.stdout:9/558: getdents d1/d2f/d3a/d74 0 2024-03-23T15:44:55.392 INFO:tasks.workunit.client.0.smithi076.stdout:9/559: truncate d1/d2f/d3a/f84 1613012 0 2024-03-23T15:44:55.392 INFO:tasks.workunit.client.0.smithi076.stdout:9/560: write d1/f19 [1081994,124055] 0 2024-03-23T15:44:55.395 INFO:tasks.workunit.client.0.smithi076.stdout:9/561: symlink d1/d1f/d46/d83/lb5 0 2024-03-23T15:44:55.401 INFO:tasks.workunit.client.0.smithi076.stdout:7/669: rename d8/f58 to d8/de/d13/d3a/dd2/fdd 0 2024-03-23T15:44:55.403 INFO:tasks.workunit.client.0.smithi076.stdout:9/562: dread d1/d20/f25 [0,4194304] 0 2024-03-23T15:44:55.403 INFO:tasks.workunit.client.0.smithi076.stdout:5/492: dwrite d1a/d39/f52 [0,4194304] 0 2024-03-23T15:44:55.403 INFO:tasks.workunit.client.0.smithi076.stdout:5/493: dread - d1a/d39/d5f/d8f/d96/fa0 zero size 2024-03-23T15:44:55.403 INFO:tasks.workunit.client.0.smithi076.stdout:7/670: mkdir d8/de/d13/d40/d88/d5a/dde 0 2024-03-23T15:44:55.403 INFO:tasks.workunit.client.0.smithi076.stdout:0/592: rename d2/de/d10/d3a/d47/d69/d84/l96 to d2/de/d10/d12/lae 0 2024-03-23T15:44:55.404 INFO:tasks.workunit.client.0.smithi076.stdout:7/671: fdatasync d8/de/d13/d3a/f8b 0 2024-03-23T15:44:55.406 INFO:tasks.workunit.client.0.smithi076.stdout:0/593: mkdir d2/daf 0 2024-03-23T15:44:55.406 INFO:tasks.workunit.client.0.smithi076.stdout:7/672: write d8/de/f8a [28594,120062] 0 2024-03-23T15:44:55.407 INFO:tasks.workunit.client.0.smithi076.stdout:7/673: mknod d8/de/d13/d40/d88/d5a/cdf 0 2024-03-23T15:44:55.407 INFO:tasks.workunit.client.0.smithi076.stdout:9/563: creat d1/d1f/d46/d83/fb6 x:0 0 0 2024-03-23T15:44:55.407 INFO:tasks.workunit.client.0.smithi076.stdout:9/564: write d1/f19 [3775271,88901] 0 2024-03-23T15:44:55.408 INFO:tasks.workunit.client.0.smithi076.stdout:7/674: truncate d8/de/f6f 178260 0 2024-03-23T15:44:55.408 INFO:tasks.workunit.client.0.smithi076.stdout:7/675: chown d8/de/d13/d40/d5d/f29 46 1 2024-03-23T15:44:55.409 INFO:tasks.workunit.client.0.smithi076.stdout:7/676: write d8/de/d13/d40/f59 [328225,2261] 0 2024-03-23T15:44:55.409 INFO:tasks.workunit.client.0.smithi076.stdout:9/565: creat d1/d1f/d29/d7f/fb7 x:0 0 0 2024-03-23T15:44:55.409 INFO:tasks.workunit.client.0.smithi076.stdout:0/594: creat d2/de/d10/fb0 x:0 0 0 2024-03-23T15:44:55.409 INFO:tasks.workunit.client.0.smithi076.stdout:0/595: fdatasync d2/de/d10/d43/f6f 0 2024-03-23T15:44:55.411 INFO:tasks.workunit.client.0.smithi076.stdout:7/677: mkdir d8/de/d13/d24/d91/d96/de0 0 2024-03-23T15:44:55.411 INFO:tasks.workunit.client.0.smithi076.stdout:7/678: read - d8/de/d13/d40/d88/fb5 zero size 2024-03-23T15:44:55.411 INFO:tasks.workunit.client.0.smithi076.stdout:5/494: write d1a/f2a [3653777,94979] 0 2024-03-23T15:44:55.411 INFO:tasks.workunit.client.0.smithi076.stdout:5/495: write d1a/f31 [100928,30064] 0 2024-03-23T15:44:55.411 INFO:tasks.workunit.client.0.smithi076.stdout:0/596: symlink d2/de/d10/d3a/d47/d69/lb1 0 2024-03-23T15:44:55.411 INFO:tasks.workunit.client.0.smithi076.stdout:5/496: write d1a/d33/d3a/fa5 [815616,64799] 0 2024-03-23T15:44:55.412 INFO:tasks.workunit.client.0.smithi076.stdout:7/679: creat d8/de/d13/d40/d88/fe1 x:0 0 0 2024-03-23T15:44:55.413 INFO:tasks.workunit.client.0.smithi076.stdout:7/680: creat d8/de/d13/d40/d88/d5a/dde/fe2 x:0 0 0 2024-03-23T15:44:55.414 INFO:tasks.workunit.client.0.smithi076.stdout:7/681: mknod d8/de/d13/d3a/dbf/dc7/ce3 0 2024-03-23T15:44:55.419 INFO:tasks.workunit.client.0.smithi076.stdout:0/597: dread d2/d8/d32/f48 [0,4194304] 0 2024-03-23T15:44:55.425 INFO:tasks.workunit.client.0.smithi076.stdout:0/598: dread d2/de/d10/d3a/d47/d69/d84/fa2 [0,4194304] 0 2024-03-23T15:44:55.428 INFO:tasks.workunit.client.0.smithi076.stdout:0/599: getdents d2/de/d10/d3a/d47/d69/d84 0 2024-03-23T15:44:55.429 INFO:tasks.workunit.client.0.smithi076.stdout:9/566: dwrite d1/d2f/d3f/f92 [0,4194304] 0 2024-03-23T15:44:55.429 INFO:tasks.workunit.client.0.smithi076.stdout:5/497: dwrite d1a/d33/d36/fa2 [0,4194304] 0 2024-03-23T15:44:55.430 INFO:tasks.workunit.client.0.smithi076.stdout:0/600: mknod d2/de/d10/cb2 0 2024-03-23T15:44:55.430 INFO:tasks.workunit.client.0.smithi076.stdout:0/601: readlink d2/de/d10/l66 0 2024-03-23T15:44:55.430 INFO:tasks.workunit.client.0.smithi076.stdout:0/602: readlink d2/de/d10/l24 0 2024-03-23T15:44:55.430 INFO:tasks.workunit.client.0.smithi076.stdout:9/567: rmdir d1/d1f/d29/d2d/d37 39 2024-03-23T15:44:55.431 INFO:tasks.workunit.client.0.smithi076.stdout:9/568: chown d1/d1f/d82/f9e 48 1 2024-03-23T15:44:55.431 INFO:tasks.workunit.client.0.smithi076.stdout:9/569: write d1/d2f/d3a/f84 [239415,1422] 0 2024-03-23T15:44:55.432 INFO:tasks.workunit.client.0.smithi076.stdout:9/570: creat d1/d1f/d46/d4a/d5a/daa/fb8 x:0 0 0 2024-03-23T15:44:55.433 INFO:tasks.workunit.client.0.smithi076.stdout:5/498: mkdir d1a/d33/d36/da1/da6 0 2024-03-23T15:44:55.436 INFO:tasks.workunit.client.0.smithi076.stdout:7/682: rename d8/de/d13/d3a/dbf/dc7/d8f to d8/de/d13/d24/de4 0 2024-03-23T15:44:55.436 INFO:tasks.workunit.client.0.smithi076.stdout:5/499: link d1a/d20/c23 d1a/d33/d36/ca7 0 2024-03-23T15:44:55.436 INFO:tasks.workunit.client.0.smithi076.stdout:7/683: truncate d8/de/d48/fc3 147637 0 2024-03-23T15:44:55.437 INFO:tasks.workunit.client.0.smithi076.stdout:5/500: mkdir d1a/d33/d3a/d67/da8 0 2024-03-23T15:44:55.438 INFO:tasks.workunit.client.0.smithi076.stdout:7/684: dread d8/de/d13/d3a/dbf/dc7/f6c [0,4194304] 0 2024-03-23T15:44:55.440 INFO:tasks.workunit.client.0.smithi076.stdout:7/685: symlink d8/de/d13/d3a/d85/daf/dca/le5 0 2024-03-23T15:44:55.440 INFO:tasks.workunit.client.0.smithi076.stdout:7/686: mknod d8/de/d13/d40/d88/d5a/dde/ce6 0 2024-03-23T15:44:55.441 INFO:tasks.workunit.client.0.smithi076.stdout:7/687: read d8/de/d13/d3a/dd2/fdd [426838,7194] 0 2024-03-23T15:44:55.447 INFO:tasks.workunit.client.0.smithi076.stdout:5/501: dread d1a/f2a [0,4194304] 0 2024-03-23T15:44:55.455 INFO:tasks.workunit.client.0.smithi076.stdout:9/571: dwrite d1/d2f/d3a/f96 [0,4194304] 0 2024-03-23T15:44:55.456 INFO:tasks.workunit.client.0.smithi076.stdout:9/572: symlink d1/d1f/d29/d7f/lb9 0 2024-03-23T15:44:55.457 INFO:tasks.workunit.client.0.smithi076.stdout:9/573: mknod d1/d1f/d29/d2d/d37/d47/cba 0 2024-03-23T15:44:55.457 INFO:tasks.workunit.client.0.smithi076.stdout:9/574: dread - d1/d1f/d82/d9c/fa9 zero size 2024-03-23T15:44:55.459 INFO:tasks.workunit.client.0.smithi076.stdout:7/688: dwrite d8/d4e/d8d/dbe/fdb [0,4194304] 0 2024-03-23T15:44:55.459 INFO:tasks.workunit.client.0.smithi076.stdout:7/689: dread - d8/de/d99/fbd zero size 2024-03-23T15:44:55.462 INFO:tasks.workunit.client.0.smithi076.stdout:7/690: creat d8/de/d13/d40/d88/d5a/dde/fe7 x:0 0 0 2024-03-23T15:44:55.465 INFO:tasks.workunit.client.0.smithi076.stdout:7/691: mkdir d8/de/de8 0 2024-03-23T15:44:55.466 INFO:tasks.workunit.client.0.smithi076.stdout:5/502: dwrite d1a/d4b/f50 [0,4194304] 0 2024-03-23T15:44:55.477 INFO:tasks.workunit.client.0.smithi076.stdout:9/575: dwrite d1/d1f/d29/d2d/d37/d47/fae [0,4194304] 0 2024-03-23T15:44:55.478 INFO:tasks.workunit.client.0.smithi076.stdout:0/603: rename d2/de/d10/d43/d56/f9b to d2/de/d71/fb3 0 2024-03-23T15:44:55.481 INFO:tasks.workunit.client.0.smithi076.stdout:0/604: creat d2/de/fb4 x:0 0 0 2024-03-23T15:44:55.481 INFO:tasks.workunit.client.0.smithi076.stdout:0/605: chown d2/d8/f4f 18226003 1 2024-03-23T15:44:55.481 INFO:tasks.workunit.client.0.smithi076.stdout:0/606: stat d2/d8/d2d/cac 0 2024-03-23T15:44:55.482 INFO:tasks.workunit.client.0.smithi076.stdout:5/503: dwrite f18 [4194304,4194304] 0 2024-03-23T15:44:55.483 INFO:tasks.workunit.client.0.smithi076.stdout:0/607: creat d2/d1b/fb5 x:0 0 0 2024-03-23T15:44:55.483 INFO:tasks.workunit.client.0.smithi076.stdout:7/692: link d8/de/d13/d3a/dbf/dc7/f6c d8/d4e/fe9 0 2024-03-23T15:44:55.484 INFO:tasks.workunit.client.0.smithi076.stdout:5/504: mknod d1a/d20/ca9 0 2024-03-23T15:44:55.485 INFO:tasks.workunit.client.0.smithi076.stdout:5/505: write d1a/d39/d65/f6b [650373,121033] 0 2024-03-23T15:44:55.485 INFO:tasks.workunit.client.0.smithi076.stdout:0/608: truncate d2/de/d10/d3a/d47/d69/d84/f93 3483907 0 2024-03-23T15:44:55.486 INFO:tasks.workunit.client.0.smithi076.stdout:7/693: rmdir d8/de/d13/d40 39 2024-03-23T15:44:55.488 INFO:tasks.workunit.client.0.smithi076.stdout:7/694: mkdir d8/d4e/d8d/dea 0 2024-03-23T15:44:55.497 INFO:tasks.workunit.client.0.smithi076.stdout:7/695: dread d8/de/d13/d40/d5d/f29 [0,4194304] 0 2024-03-23T15:44:55.501 INFO:tasks.workunit.client.0.smithi076.stdout:5/506: dwrite d1a/d33/d3a/f43 [4194304,4194304] 0 2024-03-23T15:44:55.502 INFO:tasks.workunit.client.0.smithi076.stdout:5/507: creat d1a/d4b/faa x:0 0 0 2024-03-23T15:44:55.503 INFO:tasks.workunit.client.0.smithi076.stdout:5/508: read d1a/d33/d3a/d67/d84/f8e [141342,5213] 0 2024-03-23T15:44:55.503 INFO:tasks.workunit.client.0.smithi076.stdout:9/576: rename d1/d1f/d29/d7f/fb7 to d1/d1f/fbb 0 2024-03-23T15:44:55.504 INFO:tasks.workunit.client.0.smithi076.stdout:9/577: fdatasync d1/d2f/d3a/f84 0 2024-03-23T15:44:55.505 INFO:tasks.workunit.client.0.smithi076.stdout:9/578: creat d1/d2f/fbc x:0 0 0 2024-03-23T15:44:55.507 INFO:tasks.workunit.client.0.smithi076.stdout:9/579: mknod d1/cbd 0 2024-03-23T15:44:55.507 INFO:tasks.workunit.client.0.smithi076.stdout:9/580: dread - d1/d2f/d3a/d74/fb3 zero size 2024-03-23T15:44:55.520 INFO:tasks.workunit.client.0.smithi076.stdout:5/509: dwrite d1a/d33/f35 [0,4194304] 0 2024-03-23T15:44:55.520 INFO:tasks.workunit.client.0.smithi076.stdout:9/581: dwrite d1/f19 [4194304,4194304] 0 2024-03-23T15:44:55.521 INFO:tasks.workunit.client.0.smithi076.stdout:5/510: write d1a/f3c [4354968,99925] 0 2024-03-23T15:44:55.522 INFO:tasks.workunit.client.0.smithi076.stdout:9/582: creat d1/d1f/d29/d2d/d37/d47/fbe x:0 0 0 2024-03-23T15:44:55.523 INFO:tasks.workunit.client.0.smithi076.stdout:9/583: getdents d1/d1f/d46/d4a 0 2024-03-23T15:44:55.525 INFO:tasks.workunit.client.0.smithi076.stdout:9/584: creat d1/d2f/fbf x:0 0 0 2024-03-23T15:44:55.525 INFO:tasks.workunit.client.0.smithi076.stdout:9/585: dread - d1/d20/f97 zero size 2024-03-23T15:44:55.537 INFO:tasks.workunit.client.0.smithi076.stdout:5/511: rename f12 to d1a/d7e/fab 0 2024-03-23T15:44:55.538 INFO:tasks.workunit.client.0.smithi076.stdout:5/512: symlink d1a/d33/d36/d8b/lac 0 2024-03-23T15:44:55.547 INFO:tasks.workunit.client.0.smithi076.stdout:5/513: write d1a/d33/f69 [3309962,79050] 0 2024-03-23T15:44:55.548 INFO:tasks.workunit.client.0.smithi076.stdout:5/514: creat d1a/d20/fad x:0 0 0 2024-03-23T15:44:55.551 INFO:tasks.workunit.client.0.smithi076.stdout:5/515: dread d1a/f2a [0,4194304] 0 2024-03-23T15:44:55.564 INFO:tasks.workunit.client.0.smithi076.stdout:5/516: dwrite d1a/f92 [0,4194304] 0 2024-03-23T15:44:55.566 INFO:tasks.workunit.client.0.smithi076.stdout:5/517: mknod d1a/d2c/cae 0 2024-03-23T15:44:55.567 INFO:tasks.workunit.client.0.smithi076.stdout:5/518: symlink d1a/d33/d3a/laf 0 2024-03-23T15:44:55.569 INFO:tasks.workunit.client.0.smithi076.stdout:5/519: symlink d1a/d20/lb0 0 2024-03-23T15:44:55.569 INFO:tasks.workunit.client.0.smithi076.stdout:5/520: fsync d1a/d39/d5f/f81 0 2024-03-23T15:44:55.570 INFO:tasks.workunit.client.0.smithi076.stdout:5/521: symlink d1a/d33/d3a/d67/d84/lb1 0 2024-03-23T15:44:55.571 INFO:tasks.workunit.client.0.smithi076.stdout:5/522: mknod d1a/d33/d36/da1/cb2 0 2024-03-23T15:44:55.571 INFO:tasks.workunit.client.0.smithi076.stdout:5/523: chown d1a/d33/d3a/d67/f97 43695 1 2024-03-23T15:44:55.572 INFO:tasks.workunit.client.0.smithi076.stdout:5/524: rename d1a/d20/l85 to d1a/d4b/lb3 0 2024-03-23T15:44:55.581 INFO:tasks.workunit.client.0.smithi076.stdout:5/525: read d1a/d33/f35 [319839,58545] 0 2024-03-23T15:44:55.581 INFO:tasks.workunit.client.0.smithi076.stdout:5/526: write d1a/d33/d3a/f74 [4596743,118575] 0 2024-03-23T15:44:55.584 INFO:tasks.workunit.client.0.smithi076.stdout:5/527: rename d1a/d20/f34 to d1a/d4b/fb4 0 2024-03-23T15:44:55.590 INFO:tasks.workunit.client.0.smithi076.stdout:5/528: read d1a/d2c/f4a [2405649,109279] 0 2024-03-23T15:44:55.590 INFO:tasks.workunit.client.0.smithi076.stdout:5/529: stat d1a/d20/lb0 0 2024-03-23T15:44:55.590 INFO:tasks.workunit.client.0.smithi076.stdout:5/530: chown d1a/d33/d36/da1/cb2 0 1 2024-03-23T15:44:55.606 INFO:tasks.workunit.client.0.smithi076.stdout:5/531: dwrite f7 [4194304,4194304] 0 2024-03-23T15:44:55.606 INFO:tasks.workunit.client.0.smithi076.stdout:5/532: stat d1a/d4b/f78 0 2024-03-23T15:44:55.606 INFO:tasks.workunit.client.0.smithi076.stdout:5/533: chown d1a/d33/d3a/laf 886727418 1 2024-03-23T15:44:55.620 INFO:tasks.workunit.client.0.smithi076.stdout:5/534: dwrite d1a/f29 [0,4194304] 0 2024-03-23T15:44:55.620 INFO:tasks.workunit.client.0.smithi076.stdout:5/535: truncate d1a/d33/f69 4827153 0 2024-03-23T15:44:55.621 INFO:tasks.workunit.client.0.smithi076.stdout:5/536: mknod d1a/d33/d36/cb5 0 2024-03-23T15:44:55.624 INFO:tasks.workunit.client.0.smithi076.stdout:5/537: mknod d1a/d7e/cb6 0 2024-03-23T15:44:55.636 INFO:tasks.workunit.client.0.smithi076.stdout:5/538: dwrite d1a/d4b/f4c [0,4194304] 0 2024-03-23T15:44:55.639 INFO:tasks.workunit.client.0.smithi076.stdout:5/539: creat d1a/d33/d3a/d80/fb7 x:0 0 0 2024-03-23T15:44:55.640 INFO:tasks.workunit.client.0.smithi076.stdout:5/540: truncate d1a/d2c/f87 2192589 0 2024-03-23T15:44:55.640 INFO:tasks.workunit.client.0.smithi076.stdout:5/541: chown d1a/d4b 378420813 1 2024-03-23T15:44:55.642 INFO:tasks.workunit.client.0.smithi076.stdout:5/542: creat d1a/d4b/fb8 x:0 0 0 2024-03-23T15:44:55.642 INFO:tasks.workunit.client.0.smithi076.stdout:5/543: chown d1a/d7e/fab 321 1 2024-03-23T15:44:55.643 INFO:tasks.workunit.client.0.smithi076.stdout:5/544: symlink d1a/d39/lb9 0 2024-03-23T15:44:55.643 INFO:tasks.workunit.client.0.smithi076.stdout:5/545: stat d1a/d33/d3a/d67/l83 0 2024-03-23T15:44:55.643 INFO:tasks.workunit.client.0.smithi076.stdout:5/546: dread - d1a/d33/d3a/f8a zero size 2024-03-23T15:44:55.643 INFO:tasks.workunit.client.0.smithi076.stdout:5/547: write d1a/d33/f73 [761010,65198] 0 2024-03-23T15:44:55.643 INFO:tasks.workunit.client.0.smithi076.stdout:5/548: readlink d1a/d33/d3a/d67/l83 0 2024-03-23T15:44:55.644 INFO:tasks.workunit.client.0.smithi076.stdout:5/549: creat d1a/d33/d36/d8b/fba x:0 0 0 2024-03-23T15:44:55.646 INFO:tasks.workunit.client.0.smithi076.stdout:5/550: creat d1a/d33/d3a/d67/da8/fbb x:0 0 0 2024-03-23T15:44:55.660 INFO:tasks.workunit.client.0.smithi076.stdout:5/551: dwrite d1a/f2a [4194304,4194304] 0 2024-03-23T15:44:55.662 INFO:tasks.workunit.client.0.smithi076.stdout:5/552: link d1a/d33/d3a/f43 d1a/d33/d36/da1/da6/fbc 0 2024-03-23T15:44:55.663 INFO:tasks.workunit.client.0.smithi076.stdout:5/553: creat d1a/d33/d36/d8b/fbd x:0 0 0 2024-03-23T15:44:55.664 INFO:tasks.workunit.client.0.smithi076.stdout:5/554: creat d1a/d2c/fbe x:0 0 0 2024-03-23T15:44:55.664 INFO:tasks.workunit.client.0.smithi076.stdout:5/555: write d1a/d33/d3a/f7a [921970,58638] 0 2024-03-23T15:44:55.665 INFO:tasks.workunit.client.0.smithi076.stdout:5/556: creat d1a/d2c/fbf x:0 0 0 2024-03-23T15:44:55.671 INFO:tasks.workunit.client.0.smithi076.stdout:5/557: dread d1a/f3c [0,4194304] 0 2024-03-23T15:44:55.672 INFO:tasks.workunit.client.0.smithi076.stdout:5/558: chown d1a/c2b 1271479 1 2024-03-23T15:44:55.673 INFO:tasks.workunit.client.0.smithi076.stdout:5/559: truncate d1a/d39/d65/f70 319216 0 2024-03-23T15:44:55.674 INFO:tasks.workunit.client.0.smithi076.stdout:5/560: creat d1a/d33/d36/fc0 x:0 0 0 2024-03-23T15:44:55.675 INFO:tasks.workunit.client.0.smithi076.stdout:5/561: truncate d1a/f42 1252911 0 2024-03-23T15:44:55.675 INFO:tasks.workunit.client.0.smithi076.stdout:5/562: chown d1a/d33/d3a/f74 94530 1 2024-03-23T15:44:55.695 INFO:tasks.workunit.client.0.smithi076.stdout:5/563: dwrite d1a/f1b [0,4194304] 0 2024-03-23T15:44:55.708 INFO:tasks.workunit.client.0.smithi076.stdout:5/564: dwrite d1a/f3c [0,4194304] 0 2024-03-23T15:44:55.709 INFO:tasks.workunit.client.0.smithi076.stdout:5/565: creat d1a/d39/d5f/d8f/d96/fc1 x:0 0 0 2024-03-23T15:44:55.710 INFO:tasks.workunit.client.0.smithi076.stdout:5/566: truncate d1a/d33/d36/f3d 2577367 0 2024-03-23T15:44:55.715 INFO:tasks.workunit.client.0.smithi076.stdout:5/567: truncate d1a/d33/f69 1561775 0 2024-03-23T15:44:55.717 INFO:tasks.workunit.client.0.smithi076.stdout:5/568: truncate d1a/d33/d36/f3d 512022 0 2024-03-23T15:44:55.719 INFO:tasks.workunit.client.0.smithi076.stdout:5/569: creat d1a/d33/d36/d8b/fc2 x:0 0 0 2024-03-23T15:44:55.719 INFO:tasks.workunit.client.0.smithi076.stdout:5/570: chown d1a/d2c/cae 145657 1 2024-03-23T15:44:55.722 INFO:tasks.workunit.client.0.smithi076.stdout:5/571: rename d1a/d33/d36/da1 to d1a/d33/dc3 0 2024-03-23T15:44:55.722 INFO:tasks.workunit.client.0.smithi076.stdout:5/572: chown d1a/d33/d36/d8b/fbd 1863 1 2024-03-23T15:44:55.734 INFO:tasks.workunit.client.0.smithi076.stdout:5/573: dwrite d1a/d2c/f4a [0,4194304] 0 2024-03-23T15:44:55.735 INFO:tasks.workunit.client.0.smithi076.stdout:5/574: getdents d1a/d2c 0 2024-03-23T15:44:55.737 INFO:tasks.workunit.client.0.smithi076.stdout:5/575: rename d1a/d39 to d1a/d33/d3a/d67/da8/dc4 0 2024-03-23T15:44:55.742 INFO:tasks.workunit.client.0.smithi076.stdout:5/576: dread d1a/d33/d3a/d67/da8/dc4/f44 [0,4194304] 0 2024-03-23T15:44:55.742 INFO:tasks.workunit.client.0.smithi076.stdout:5/577: write fc [7467374,79634] 0 2024-03-23T15:44:55.742 INFO:tasks.workunit.client.0.smithi076.stdout:5/578: chown d1a/d33/d3a/f41 94218 1 2024-03-23T15:44:55.742 INFO:tasks.workunit.client.0.smithi076.stdout:5/579: write d1a/d4b/fb8 [335676,101276] 0 2024-03-23T15:44:55.742 INFO:tasks.workunit.client.0.smithi076.stdout:5/580: getdents d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96 0 2024-03-23T15:44:55.744 INFO:tasks.workunit.client.0.smithi076.stdout:5/581: truncate d1a/d4b/fb4 1661073 0 2024-03-23T15:44:55.744 INFO:tasks.workunit.client.0.smithi076.stdout:5/582: write d1a/d33/d3a/d67/da8/fbb [361169,30246] 0 2024-03-23T15:44:55.745 INFO:tasks.workunit.client.0.smithi076.stdout:5/583: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/fc5 x:0 0 0 2024-03-23T15:44:55.745 INFO:tasks.workunit.client.0.smithi076.stdout:5/584: read d1a/d2c/f87 [261468,59804] 0 2024-03-23T15:44:55.751 INFO:tasks.workunit.client.0.smithi076.stdout:5/585: dread d1a/d4b/fb4 [0,4194304] 0 2024-03-23T15:44:55.752 INFO:tasks.workunit.client.0.smithi076.stdout:5/586: creat d1a/d4b/fc6 x:0 0 0 2024-03-23T15:44:55.753 INFO:tasks.workunit.client.0.smithi076.stdout:5/587: creat d1a/d33/dc3/fc7 x:0 0 0 2024-03-23T15:44:55.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:55 smithi192 ceph-mon[36653]: pgmap v50: 97 pgs: 97 active+clean; 569 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 77 MiB/s wr, 973 op/s 2024-03-23T15:44:55.820 INFO:tasks.workunit.client.0.smithi076.stdout:6/379: sync 2024-03-23T15:44:55.820 INFO:tasks.workunit.client.0.smithi076.stdout:4/565: sync 2024-03-23T15:44:55.821 INFO:tasks.workunit.client.0.smithi076.stdout:6/380: symlink d1/d2/d27/d60/l84 0 2024-03-23T15:44:55.821 INFO:tasks.workunit.client.0.smithi076.stdout:6/381: fdatasync d1/d2/f1c 0 2024-03-23T15:44:55.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:55 smithi076 ceph-mon[32176]: pgmap v50: 97 pgs: 97 active+clean; 569 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 77 MiB/s wr, 973 op/s 2024-03-23T15:44:55.827 INFO:tasks.workunit.client.0.smithi076.stdout:6/382: dread d1/d2/d27/f46 [0,4194304] 0 2024-03-23T15:44:55.832 INFO:tasks.workunit.client.0.smithi076.stdout:4/566: dwrite d8/f3f [0,4194304] 0 2024-03-23T15:44:55.832 INFO:tasks.workunit.client.0.smithi076.stdout:4/567: readlink d8/d3a/d69/d73/l8f 0 2024-03-23T15:44:55.833 INFO:tasks.workunit.client.0.smithi076.stdout:4/568: write d8/d3a/d69/f86 [257685,129885] 0 2024-03-23T15:44:55.833 INFO:tasks.workunit.client.0.smithi076.stdout:4/569: chown d8/d3a/d18/d75/d8d 989837 1 2024-03-23T15:44:55.835 INFO:tasks.workunit.client.0.smithi076.stdout:4/570: unlink d8/d3a/d18/f89 0 2024-03-23T15:44:55.837 INFO:tasks.workunit.client.0.smithi076.stdout:4/571: mkdir d8/d3a/dab 0 2024-03-23T15:44:55.839 INFO:tasks.workunit.client.0.smithi076.stdout:6/383: rename d1/dd/f7d to d1/f85 0 2024-03-23T15:44:55.839 INFO:tasks.workunit.client.0.smithi076.stdout:6/384: chown d1/d4/d10/l50 2305798 1 2024-03-23T15:44:55.841 INFO:tasks.workunit.client.0.smithi076.stdout:6/385: creat d1/d4/f86 x:0 0 0 2024-03-23T15:44:55.841 INFO:tasks.workunit.client.0.smithi076.stdout:6/386: chown d1/d2/f71 926050 1 2024-03-23T15:44:55.842 INFO:tasks.workunit.client.0.smithi076.stdout:6/387: creat d1/d2/f87 x:0 0 0 2024-03-23T15:44:55.843 INFO:tasks.workunit.client.0.smithi076.stdout:4/572: dread d8/d3a/d18/d1b/f67 [0,4194304] 0 2024-03-23T15:44:55.844 INFO:tasks.workunit.client.0.smithi076.stdout:4/573: symlink d8/d9/lac 0 2024-03-23T15:44:55.845 INFO:tasks.workunit.client.0.smithi076.stdout:4/574: write d8/d3a/d3b/f4c [786517,17511] 0 2024-03-23T15:44:55.846 INFO:tasks.workunit.client.0.smithi076.stdout:4/575: fsync d8/f28 0 2024-03-23T15:44:55.846 INFO:tasks.workunit.client.0.smithi076.stdout:4/576: chown d8/d3a/d18/d52/c8b 743066269 1 2024-03-23T15:44:55.856 INFO:tasks.workunit.client.0.smithi076.stdout:6/388: dwrite d1/d4/d10/f30 [0,4194304] 0 2024-03-23T15:44:55.857 INFO:tasks.workunit.client.0.smithi076.stdout:6/389: write d1/dd/d2a/f5c [135974,87451] 0 2024-03-23T15:44:55.863 INFO:tasks.workunit.client.0.smithi076.stdout:4/577: dwrite d8/d3a/d69/f86 [0,4194304] 0 2024-03-23T15:44:55.872 INFO:tasks.workunit.client.0.smithi076.stdout:4/578: dread d8/d3a/f48 [0,4194304] 0 2024-03-23T15:44:55.875 INFO:tasks.workunit.client.0.smithi076.stdout:4/579: write d8/d3a/d18/f21 [3540772,93279] 0 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:5/588: sync 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:2/569: sync 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:7/696: sync 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:9/586: sync 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:3/428: sync 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:1/318: sync 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:0/609: sync 2024-03-23T15:44:55.881 INFO:tasks.workunit.client.0.smithi076.stdout:8/454: sync 2024-03-23T15:44:55.882 INFO:tasks.workunit.client.0.smithi076.stdout:3/429: readlink d5/d17/d1c/d32/l2c 0 2024-03-23T15:44:55.882 INFO:tasks.workunit.client.0.smithi076.stdout:3/430: read d5/d35/f36 [3113857,104480] 0 2024-03-23T15:44:55.882 INFO:tasks.workunit.client.0.smithi076.stdout:1/319: write d0/f26 [7548160,34273] 0 2024-03-23T15:44:55.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:55 smithi183 ceph-mon[36296]: pgmap v50: 97 pgs: 97 active+clean; 569 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 57 MiB/s rd, 77 MiB/s wr, 973 op/s 2024-03-23T15:44:55.883 INFO:tasks.workunit.client.0.smithi076.stdout:8/455: link d8/d2e/d3e/c4b d8/d26/c97 0 2024-03-23T15:44:55.883 INFO:tasks.workunit.client.0.smithi076.stdout:8/456: chown d8/d26/f46 0 1 2024-03-23T15:44:55.885 INFO:tasks.workunit.client.0.smithi076.stdout:8/457: creat d8/d6b/d91/f98 x:0 0 0 2024-03-23T15:44:55.885 INFO:tasks.workunit.client.0.smithi076.stdout:7/697: rename d8/de/l63 to d8/de/d48/dd8/leb 0 2024-03-23T15:44:55.886 INFO:tasks.workunit.client.0.smithi076.stdout:2/570: mknod d1/d4/d64/d1b/d32/cc1 0 2024-03-23T15:44:55.886 INFO:tasks.workunit.client.0.smithi076.stdout:8/458: truncate d8/d2e/d30/f6a 1743390 0 2024-03-23T15:44:55.889 INFO:tasks.workunit.client.0.smithi076.stdout:8/459: symlink d8/d26/l99 0 2024-03-23T15:44:55.890 INFO:tasks.workunit.client.0.smithi076.stdout:8/460: chown d8/c62 3 1 2024-03-23T15:44:55.891 INFO:tasks.workunit.client.0.smithi076.stdout:0/610: rename d2/de/d71/fb3 to d2/de/d10/d43/fb6 0 2024-03-23T15:44:55.895 INFO:tasks.workunit.client.0.smithi076.stdout:4/580: dwrite d8/d9/f12 [0,4194304] 0 2024-03-23T15:44:55.895 INFO:tasks.workunit.client.0.smithi076.stdout:4/581: chown d8/c6e 12 1 2024-03-23T15:44:55.895 INFO:tasks.workunit.client.0.smithi076.stdout:4/582: fsync d8/d80/f8a 0 2024-03-23T15:44:55.895 INFO:tasks.workunit.client.0.smithi076.stdout:4/583: write d8/d9/d1d/f60 [3314022,48135] 0 2024-03-23T15:44:55.896 INFO:tasks.workunit.client.0.smithi076.stdout:4/584: chown d8/d3a/d18/d1b/f96 7667098 1 2024-03-23T15:44:55.897 INFO:tasks.workunit.client.0.smithi076.stdout:4/585: getdents d8/d9/d32 0 2024-03-23T15:44:55.897 INFO:tasks.workunit.client.0.smithi076.stdout:4/586: readlink d8/d3a/d18/d1b/l3c 0 2024-03-23T15:44:55.897 INFO:tasks.workunit.client.0.smithi076.stdout:4/587: write d8/d3a/d18/d52/f6d [563731,120204] 0 2024-03-23T15:44:55.898 INFO:tasks.workunit.client.0.smithi076.stdout:4/588: chown d8/d9/fc 51660 1 2024-03-23T15:44:55.898 INFO:tasks.workunit.client.0.smithi076.stdout:4/589: truncate d8/d3a/d18/d52/f61 1124487 0 2024-03-23T15:44:55.899 INFO:tasks.workunit.client.0.smithi076.stdout:4/590: truncate d8/d3a/d18/f20 1689414 0 2024-03-23T15:44:55.901 INFO:tasks.workunit.client.0.smithi076.stdout:4/591: creat d8/d3a/d18/d52/fad x:0 0 0 2024-03-23T15:44:55.903 INFO:tasks.workunit.client.0.smithi076.stdout:4/592: creat d8/fae x:0 0 0 2024-03-23T15:44:55.903 INFO:tasks.workunit.client.0.smithi076.stdout:4/593: dread - d8/d3a/f34 zero size 2024-03-23T15:44:55.904 INFO:tasks.workunit.client.0.smithi076.stdout:4/594: mknod d8/d3a/d18/d75/caf 0 2024-03-23T15:44:55.906 INFO:tasks.workunit.client.0.smithi076.stdout:4/595: mkdir d8/d3a/d18/d75/d8d/db0 0 2024-03-23T15:44:55.907 INFO:tasks.workunit.client.0.smithi076.stdout:4/596: dread d8/d3a/d18/f2b [0,4194304] 0 2024-03-23T15:44:55.908 INFO:tasks.workunit.client.0.smithi076.stdout:9/587: dwrite d1/f48 [0,4194304] 0 2024-03-23T15:44:55.909 INFO:tasks.workunit.client.0.smithi076.stdout:9/588: fsync d1/d1f/d29/d2d/d37/f59 0 2024-03-23T15:44:55.909 INFO:tasks.workunit.client.0.smithi076.stdout:9/589: chown d1/l2b 45684427 1 2024-03-23T15:44:55.909 INFO:tasks.workunit.client.0.smithi076.stdout:4/597: creat d8/d3a/d3b/fb1 x:0 0 0 2024-03-23T15:44:55.909 INFO:tasks.workunit.client.0.smithi076.stdout:4/598: write d8/f1a [3571724,129922] 0 2024-03-23T15:44:55.909 INFO:tasks.workunit.client.0.smithi076.stdout:4/599: fdatasync d8/d3a/f48 0 2024-03-23T15:44:55.910 INFO:tasks.workunit.client.0.smithi076.stdout:4/600: getdents d8/d3a/d18/d1b 0 2024-03-23T15:44:55.911 INFO:tasks.workunit.client.0.smithi076.stdout:3/431: link d5/d17/c54 d5/d17/d1c/d32/d39/d59/d79/c90 0 2024-03-23T15:44:55.911 INFO:tasks.workunit.client.0.smithi076.stdout:2/571: link d1/d4/d64/d7c/lbe d1/d4/d52/d6e/lc2 0 2024-03-23T15:44:55.916 INFO:tasks.workunit.client.0.smithi076.stdout:0/611: dwrite d2/de/d10/d3a/d47/f7c [0,4194304] 0 2024-03-23T15:44:55.921 INFO:tasks.workunit.client.0.smithi076.stdout:0/612: rmdir d2/de/d10/d3a/d47/d69 39 2024-03-23T15:44:55.921 INFO:tasks.workunit.client.0.smithi076.stdout:4/601: write d8/f49 [4155124,41175] 0 2024-03-23T15:44:55.924 INFO:tasks.workunit.client.0.smithi076.stdout:0/613: link d2/d8/f4f d2/fb7 0 2024-03-23T15:44:55.924 INFO:tasks.workunit.client.0.smithi076.stdout:7/698: dwrite d8/de/d13/f94 [0,4194304] 0 2024-03-23T15:44:55.925 INFO:tasks.workunit.client.0.smithi076.stdout:4/602: symlink d8/d3a/d18/d75/d8d/lb2 0 2024-03-23T15:44:55.926 INFO:tasks.workunit.client.0.smithi076.stdout:7/699: creat d8/fec x:0 0 0 2024-03-23T15:44:55.926 INFO:tasks.workunit.client.0.smithi076.stdout:0/614: mkdir d2/d9a/db8 0 2024-03-23T15:44:55.926 INFO:tasks.workunit.client.0.smithi076.stdout:0/615: readlink d2/de/d10/d12/lae 0 2024-03-23T15:44:55.928 INFO:tasks.workunit.client.0.smithi076.stdout:8/461: dwrite d8/d6b/d8e/f92 [0,4194304] 0 2024-03-23T15:44:55.928 INFO:tasks.workunit.client.0.smithi076.stdout:8/462: readlink d8/d2e/d3e/l4c 0 2024-03-23T15:44:55.928 INFO:tasks.workunit.client.0.smithi076.stdout:0/616: truncate d2/de/f1a 926260 0 2024-03-23T15:44:55.931 INFO:tasks.workunit.client.0.smithi076.stdout:7/700: rmdir d8/de/d99/dc4/dc6 0 2024-03-23T15:44:55.931 INFO:tasks.workunit.client.0.smithi076.stdout:0/617: creat d2/de/d10/d43/fb9 x:0 0 0 2024-03-23T15:44:55.933 INFO:tasks.workunit.client.0.smithi076.stdout:0/618: symlink d2/de/d10/d12/da6/lba 0 2024-03-23T15:44:55.933 INFO:tasks.workunit.client.0.smithi076.stdout:0/619: chown d2/c33 184455430 1 2024-03-23T15:44:55.933 INFO:tasks.workunit.client.0.smithi076.stdout:0/620: write d2/de/d10/d12/f85 [1305265,114874] 0 2024-03-23T15:44:55.933 INFO:tasks.workunit.client.0.smithi076.stdout:0/621: write d2/de/d10/d3a/d47/f7c [5130076,21041] 0 2024-03-23T15:44:55.933 INFO:tasks.workunit.client.0.smithi076.stdout:0/622: chown d2/d8/f4f 6774964 1 2024-03-23T15:44:55.933 INFO:tasks.workunit.client.0.smithi076.stdout:7/701: stat d8/de/f81 0 2024-03-23T15:44:55.934 INFO:tasks.workunit.client.0.smithi076.stdout:3/432: dwrite d5/d35/d4c/f51 [0,4194304] 0 2024-03-23T15:44:55.935 INFO:tasks.workunit.client.0.smithi076.stdout:8/463: dread d8/d26/d33/f3d [0,4194304] 0 2024-03-23T15:44:55.935 INFO:tasks.workunit.client.0.smithi076.stdout:0/623: getdents d2/d8 0 2024-03-23T15:44:55.936 INFO:tasks.workunit.client.0.smithi076.stdout:7/702: unlink d8/de/d13/d40/d5d/f6e 0 2024-03-23T15:44:55.937 INFO:tasks.workunit.client.0.smithi076.stdout:0/624: truncate d2/d8/d32/f4a 202291 0 2024-03-23T15:44:55.939 INFO:tasks.workunit.client.0.smithi076.stdout:9/590: dwrite d1/d1f/d46/d4a/d5a/daa/fb8 [0,4194304] 0 2024-03-23T15:44:55.940 INFO:tasks.workunit.client.0.smithi076.stdout:9/591: dread d1/d2f/d3a/d64/f91 [0,4194304] 0 2024-03-23T15:44:55.941 INFO:tasks.workunit.client.0.smithi076.stdout:2/572: dwrite d1/d4/db/d15/f21 [0,4194304] 0 2024-03-23T15:44:55.942 INFO:tasks.workunit.client.0.smithi076.stdout:7/703: read f1 [3889450,83504] 0 2024-03-23T15:44:55.942 INFO:tasks.workunit.client.0.smithi076.stdout:9/592: write d1/f35 [99328,58574] 0 2024-03-23T15:44:55.943 INFO:tasks.workunit.client.0.smithi076.stdout:8/464: dread d8/f1e [0,4194304] 0 2024-03-23T15:44:55.944 INFO:tasks.workunit.client.0.smithi076.stdout:2/573: creat d1/d4/d64/d7c/fc3 x:0 0 0 2024-03-23T15:44:55.944 INFO:tasks.workunit.client.0.smithi076.stdout:2/574: read - d1/d4/d52/fbb zero size 2024-03-23T15:44:55.946 INFO:tasks.workunit.client.0.smithi076.stdout:2/575: creat d1/d4/db/d15/fc4 x:0 0 0 2024-03-23T15:44:55.947 INFO:tasks.workunit.client.0.smithi076.stdout:9/593: dread - d1/d2f/d3f/f49 zero size 2024-03-23T15:44:55.947 INFO:tasks.workunit.client.0.smithi076.stdout:9/594: write d1/d2f/fbf [463226,13638] 0 2024-03-23T15:44:55.950 INFO:tasks.workunit.client.0.smithi076.stdout:2/576: dread d1/d4/d64/f4e [0,4194304] 0 2024-03-23T15:44:55.951 INFO:tasks.workunit.client.0.smithi076.stdout:2/577: mknod d1/d4/d64/d82/cc5 0 2024-03-23T15:44:55.951 INFO:tasks.workunit.client.0.smithi076.stdout:2/578: dread - d1/d4/d64/d82/fb6 zero size 2024-03-23T15:44:55.954 INFO:tasks.workunit.client.0.smithi076.stdout:9/595: mkdir d1/d2f/d3f/dc0 0 2024-03-23T15:44:55.955 INFO:tasks.workunit.client.0.smithi076.stdout:2/579: getdents d1/d4/d64/d7c 0 2024-03-23T15:44:55.955 INFO:tasks.workunit.client.0.smithi076.stdout:2/580: write d1/d2e/d8a/f9a [2458728,99178] 0 2024-03-23T15:44:55.955 INFO:tasks.workunit.client.0.smithi076.stdout:2/581: rename d1/d4/db/d3e to d1/d4/db/d3e/dc6 22 2024-03-23T15:44:55.956 INFO:tasks.workunit.client.0.smithi076.stdout:4/603: dwrite d8/d3a/d18/d52/f79 [0,4194304] 0 2024-03-23T15:44:55.957 INFO:tasks.workunit.client.0.smithi076.stdout:2/582: creat d1/d4/d64/fc7 x:0 0 0 2024-03-23T15:44:55.961 INFO:tasks.workunit.client.0.smithi076.stdout:9/596: unlink d1/c3b 0 2024-03-23T15:44:55.961 INFO:tasks.workunit.client.0.smithi076.stdout:9/597: dread - d1/d1f/d29/d2d/d37/d47/f52 zero size 2024-03-23T15:44:55.961 INFO:tasks.workunit.client.0.smithi076.stdout:2/583: mkdir d1/d4/db/dc8 0 2024-03-23T15:44:55.963 INFO:tasks.workunit.client.0.smithi076.stdout:2/584: creat d1/d4/db/d15/d29/d7d/fc9 x:0 0 0 2024-03-23T15:44:55.964 INFO:tasks.workunit.client.0.smithi076.stdout:7/704: dwrite d8/de/d13/d40/d88/d5a/f7f [4194304,4194304] 0 2024-03-23T15:44:55.964 INFO:tasks.workunit.client.0.smithi076.stdout:7/705: read d8/de/d13/d40/d5d/f34 [209151,78164] 0 2024-03-23T15:44:55.964 INFO:tasks.workunit.client.0.smithi076.stdout:7/706: write d8/de/d13/d3a/dd2/fdd [349080,127910] 0 2024-03-23T15:44:55.966 INFO:tasks.workunit.client.0.smithi076.stdout:7/707: mknod d8/de/d99/ced 0 2024-03-23T15:44:55.970 INFO:tasks.workunit.client.0.smithi076.stdout:7/708: rename d8/de/d99/dc4/fd6 to d8/de/d13/d40/d5d/d3c/fee 0 2024-03-23T15:44:55.970 INFO:tasks.workunit.client.0.smithi076.stdout:4/604: creat d8/d3a/dab/fb3 x:0 0 0 2024-03-23T15:44:55.971 INFO:tasks.workunit.client.0.smithi076.stdout:7/709: creat d8/de/d13/fef x:0 0 0 2024-03-23T15:44:55.972 INFO:tasks.workunit.client.0.smithi076.stdout:7/710: write d8/de/d13/d3a/dbf/dc7/f77 [948053,73297] 0 2024-03-23T15:44:55.972 INFO:tasks.workunit.client.0.smithi076.stdout:7/711: chown d8/de/d13/d3a/d85/lce 43415418 1 2024-03-23T15:44:55.972 INFO:tasks.workunit.client.0.smithi076.stdout:7/712: dread - d8/de/d13/d40/f90 zero size 2024-03-23T15:44:55.973 INFO:tasks.workunit.client.0.smithi076.stdout:7/713: link d8/de/d99/lb1 d8/de/d48/dd8/lf0 0 2024-03-23T15:44:55.976 INFO:tasks.workunit.client.0.smithi076.stdout:8/465: dwrite d8/fe [0,4194304] 0 2024-03-23T15:44:55.976 INFO:tasks.workunit.client.0.smithi076.stdout:4/605: dread d8/d3a/d69/f86 [0,4194304] 0 2024-03-23T15:44:55.976 INFO:tasks.workunit.client.0.smithi076.stdout:4/606: dread - d8/d80/f8a zero size 2024-03-23T15:44:55.978 INFO:tasks.workunit.client.0.smithi076.stdout:0/625: link d2/de/d10/d43/d56/c6c d2/de/d10/d3a/d47/d69/da3/cbb 0 2024-03-23T15:44:55.978 INFO:tasks.workunit.client.0.smithi076.stdout:0/626: stat d2/d1b/fa8 0 2024-03-23T15:44:55.980 INFO:tasks.workunit.client.0.smithi076.stdout:8/466: mkdir d8/d6b/d9a 0 2024-03-23T15:44:55.980 INFO:tasks.workunit.client.0.smithi076.stdout:8/467: getdents d8/d6b/d9a 0 2024-03-23T15:44:55.981 INFO:tasks.workunit.client.0.smithi076.stdout:8/468: read d8/d2e/f5c [147867,3411] 0 2024-03-23T15:44:55.982 INFO:tasks.workunit.client.0.smithi076.stdout:0/627: creat d2/d9a/fbc x:0 0 0 2024-03-23T15:44:55.982 INFO:tasks.workunit.client.0.smithi076.stdout:0/628: chown d2/de/d10/d12/c5b 57207 1 2024-03-23T15:44:55.982 INFO:tasks.workunit.client.0.smithi076.stdout:8/469: getdents d8/d83 0 2024-03-23T15:44:55.983 INFO:tasks.workunit.client.0.smithi076.stdout:0/629: mknod d2/d1b/cbd 0 2024-03-23T15:44:55.984 INFO:tasks.workunit.client.0.smithi076.stdout:8/470: dread d8/f23 [0,4194304] 0 2024-03-23T15:44:55.984 INFO:tasks.workunit.client.0.smithi076.stdout:8/471: readlink d8/l1b 0 2024-03-23T15:44:55.984 INFO:tasks.workunit.client.0.smithi076.stdout:0/630: creat d2/de/d10/d3a/d47/d69/fbe x:0 0 0 2024-03-23T15:44:55.985 INFO:tasks.workunit.client.0.smithi076.stdout:0/631: write d2/d8/fa [2192228,102725] 0 2024-03-23T15:44:55.985 INFO:tasks.workunit.client.0.smithi076.stdout:8/472: creat d8/d2e/d3e/f9b x:0 0 0 2024-03-23T15:44:55.985 INFO:tasks.workunit.client.0.smithi076.stdout:8/473: getdents d8/d4f 0 2024-03-23T15:44:55.986 INFO:tasks.workunit.client.0.smithi076.stdout:8/474: write d8/d2e/d30/d69/f8f [139791,1691] 0 2024-03-23T15:44:55.986 INFO:tasks.workunit.client.0.smithi076.stdout:0/632: mknod d2/daf/cbf 0 2024-03-23T15:44:55.989 INFO:tasks.workunit.client.0.smithi076.stdout:8/475: getdents d8/d6b 0 2024-03-23T15:44:55.994 INFO:tasks.workunit.client.0.smithi076.stdout:9/598: link d1/d1f/d29/d2d/d37/d47/l8d d1/d1f/d82/lc1 0 2024-03-23T15:44:55.996 INFO:tasks.workunit.client.0.smithi076.stdout:0/633: dread d2/de/d10/d12/f23 [0,4194304] 0 2024-03-23T15:44:55.996 INFO:tasks.workunit.client.0.smithi076.stdout:0/634: read - d2/d1b/fa8 zero size 2024-03-23T15:44:55.996 INFO:tasks.workunit.client.0.smithi076.stdout:9/599: symlink d1/d2f/d3f/dc0/lc2 0 2024-03-23T15:44:55.997 INFO:tasks.workunit.client.0.smithi076.stdout:2/585: link d1/d2e/d6a/f9b d1/d4/d64/d1d/fca 0 2024-03-23T15:44:55.997 INFO:tasks.workunit.client.0.smithi076.stdout:2/586: chown d1/d4/c35 306629 1 2024-03-23T15:44:55.997 INFO:tasks.workunit.client.0.smithi076.stdout:2/587: stat d1/d4/d64/d82/cc5 0 2024-03-23T15:44:55.997 INFO:tasks.workunit.client.0.smithi076.stdout:2/588: truncate d1/d2e/f42 4451923 0 2024-03-23T15:44:55.998 INFO:tasks.workunit.client.0.smithi076.stdout:2/589: write d1/d4/db/d15/d29/d7d/f9d [300879,37524] 0 2024-03-23T15:44:55.998 INFO:tasks.workunit.client.0.smithi076.stdout:9/600: symlink d1/d2f/d3f/dc0/lc3 0 2024-03-23T15:44:55.998 INFO:tasks.workunit.client.0.smithi076.stdout:9/601: truncate d1/d1f/d46/d83/fb6 915318 0 2024-03-23T15:44:55.999 INFO:tasks.workunit.client.0.smithi076.stdout:9/602: write d1/d1f/d46/d83/fb6 [73777,122620] 0 2024-03-23T15:44:55.999 INFO:tasks.workunit.client.0.smithi076.stdout:9/603: getdents d1/d1f/d46/d83/da7 0 2024-03-23T15:44:55.999 INFO:tasks.workunit.client.0.smithi076.stdout:8/476: dread d8/f1a [0,4194304] 0 2024-03-23T15:44:55.999 INFO:tasks.workunit.client.0.smithi076.stdout:8/477: chown d8/d26/d27/f37 457 1 2024-03-23T15:44:56.002 INFO:tasks.workunit.client.0.smithi076.stdout:9/604: truncate d1/d2f/f42 1388418 0 2024-03-23T15:44:56.003 INFO:tasks.workunit.client.0.smithi076.stdout:9/605: symlink d1/d1f/d29/d7f/lc4 0 2024-03-23T15:44:56.005 INFO:tasks.workunit.client.0.smithi076.stdout:8/478: unlink d8/d2e/f5c 0 2024-03-23T15:44:56.006 INFO:tasks.workunit.client.0.smithi076.stdout:8/479: mknod d8/d26/d33/d8c/c9c 0 2024-03-23T15:44:56.006 INFO:tasks.workunit.client.0.smithi076.stdout:8/480: chown d8/d2e/d30/f68 5 1 2024-03-23T15:44:56.007 INFO:tasks.workunit.client.0.smithi076.stdout:2/590: dread d1/d4/f54 [4194304,4194304] 0 2024-03-23T15:44:56.007 INFO:tasks.workunit.client.0.smithi076.stdout:2/591: write d1/d4/db/d15/d29/f9e [949709,98127] 0 2024-03-23T15:44:56.027 INFO:tasks.workunit.client.0.smithi076.stdout:9/606: rename d1/d1f/d29/d7f/f95 to d1/d1f/d46/d83/dad/fc5 0 2024-03-23T15:44:56.029 INFO:tasks.workunit.client.0.smithi076.stdout:8/481: dwrite d8/d2e/d3e/f9b [0,4194304] 0 2024-03-23T15:44:56.029 INFO:tasks.workunit.client.0.smithi076.stdout:9/607: mknod d1/d1f/d29/d2d/d37/d47/cc6 0 2024-03-23T15:44:56.029 INFO:tasks.workunit.client.0.smithi076.stdout:9/608: stat d1/d1f/d46/d4a/l9a 0 2024-03-23T15:44:56.032 INFO:tasks.workunit.client.0.smithi076.stdout:9/609: link d1/d20/l56 d1/d2f/d3a/d64/lc7 0 2024-03-23T15:44:56.032 INFO:tasks.workunit.client.0.smithi076.stdout:9/610: write d1/d1f/f22 [4063785,95149] 0 2024-03-23T15:44:56.033 INFO:tasks.workunit.client.0.smithi076.stdout:2/592: dwrite d1/d4/d64/f18 [0,4194304] 0 2024-03-23T15:44:56.033 INFO:tasks.workunit.client.0.smithi076.stdout:8/482: unlink d8/d2e/d30/f68 0 2024-03-23T15:44:56.033 INFO:tasks.workunit.client.0.smithi076.stdout:8/483: chown d8/d6b/d91/f98 0 1 2024-03-23T15:44:56.033 INFO:tasks.workunit.client.0.smithi076.stdout:8/484: chown d8/d26/c97 39 1 2024-03-23T15:44:56.033 INFO:tasks.workunit.client.0.smithi076.stdout:8/485: readlink d8/d26/d33/l74 0 2024-03-23T15:44:56.034 INFO:tasks.workunit.client.0.smithi076.stdout:2/593: symlink d1/d4/d64/d82/lcb 0 2024-03-23T15:44:56.034 INFO:tasks.workunit.client.0.smithi076.stdout:9/611: unlink d1/l2b 0 2024-03-23T15:44:56.035 INFO:tasks.workunit.client.0.smithi076.stdout:8/486: rmdir d8/d4f 39 2024-03-23T15:44:56.036 INFO:tasks.workunit.client.0.smithi076.stdout:2/594: mknod d1/d4/db/d3e/ccc 0 2024-03-23T15:44:56.036 INFO:tasks.workunit.client.0.smithi076.stdout:2/595: write d1/f2f [3964503,32464] 0 2024-03-23T15:44:56.036 INFO:tasks.workunit.client.0.smithi076.stdout:9/612: mknod d1/d2f/d3f/da0/cc8 0 2024-03-23T15:44:56.037 INFO:tasks.workunit.client.0.smithi076.stdout:6/390: sync 2024-03-23T15:44:56.037 INFO:tasks.workunit.client.0.smithi076.stdout:1/320: sync 2024-03-23T15:44:56.037 INFO:tasks.workunit.client.0.smithi076.stdout:5/589: sync 2024-03-23T15:44:56.038 INFO:tasks.workunit.client.0.smithi076.stdout:1/321: chown d0/d35/d40/d47/d17/l3b 37444126 1 2024-03-23T15:44:56.038 INFO:tasks.workunit.client.0.smithi076.stdout:9/613: truncate d1/d1f/d29/d2d/d37/d47/fae 3722728 0 2024-03-23T15:44:56.038 INFO:tasks.workunit.client.0.smithi076.stdout:5/590: creat d1a/d33/d3a/d67/da8/dc4/fc8 x:0 0 0 2024-03-23T15:44:56.042 INFO:tasks.workunit.client.0.smithi076.stdout:5/591: creat d1a/d33/fc9 x:0 0 0 2024-03-23T15:44:56.045 INFO:tasks.workunit.client.0.smithi076.stdout:8/487: dread d8/fa [0,4194304] 0 2024-03-23T15:44:56.048 INFO:tasks.workunit.client.0.smithi076.stdout:8/488: mkdir d8/d6b/d9d 0 2024-03-23T15:44:56.054 INFO:tasks.workunit.client.0.smithi076.stdout:6/391: read d1/dd/f26 [307122,127109] 0 2024-03-23T15:44:56.056 INFO:tasks.workunit.client.0.smithi076.stdout:6/392: truncate d1/d2/f1c 686024 0 2024-03-23T15:44:56.056 INFO:tasks.workunit.client.0.smithi076.stdout:1/322: dread d0/d35/d40/d47/f4e [0,4194304] 0 2024-03-23T15:44:56.056 INFO:tasks.workunit.client.0.smithi076.stdout:2/596: dwrite d1/d4/f9 [0,4194304] 0 2024-03-23T15:44:56.058 INFO:tasks.workunit.client.0.smithi076.stdout:2/597: rename d1/d4/c67 to d1/d4/d52/ccd 0 2024-03-23T15:44:56.058 INFO:tasks.workunit.client.0.smithi076.stdout:2/598: truncate d1/d4/db/d15/d29/f8f 1013226 0 2024-03-23T15:44:56.059 INFO:tasks.workunit.client.0.smithi076.stdout:2/599: mknod d1/d4/d64/d82/d7f/cce 0 2024-03-23T15:44:56.060 INFO:tasks.workunit.client.0.smithi076.stdout:2/600: write d1/d4/d64/f95 [508088,82734] 0 2024-03-23T15:44:56.061 INFO:tasks.workunit.client.0.smithi076.stdout:5/592: dwrite d1a/f2a [4194304,4194304] 0 2024-03-23T15:44:56.061 INFO:tasks.workunit.client.0.smithi076.stdout:2/601: dread - d1/d4/d64/d1d/fca zero size 2024-03-23T15:44:56.062 INFO:tasks.workunit.client.0.smithi076.stdout:2/602: creat d1/d4/d52/d6e/fcf x:0 0 0 2024-03-23T15:44:56.063 INFO:tasks.workunit.client.0.smithi076.stdout:5/593: creat d1a/d33/d3a/d80/fca x:0 0 0 2024-03-23T15:44:56.063 INFO:tasks.workunit.client.0.smithi076.stdout:5/594: chown d1a/d33/d3a/c57 9331628 1 2024-03-23T15:44:56.063 INFO:tasks.workunit.client.0.smithi076.stdout:6/393: dread d1/dd/df/f13 [0,4194304] 0 2024-03-23T15:44:56.063 INFO:tasks.workunit.client.0.smithi076.stdout:9/614: dwrite d1/d1f/d29/d2d/f6f [0,4194304] 0 2024-03-23T15:44:56.068 INFO:tasks.workunit.client.0.smithi076.stdout:2/603: creat d1/d4/d64/fd0 x:0 0 0 2024-03-23T15:44:56.068 INFO:tasks.workunit.client.0.smithi076.stdout:2/604: readlink d1/d4/d64/d7c/lbe 0 2024-03-23T15:44:56.068 INFO:tasks.workunit.client.0.smithi076.stdout:9/615: rmdir d1/d1f/d29/d2d/d37/d47 39 2024-03-23T15:44:56.068 INFO:tasks.workunit.client.0.smithi076.stdout:8/489: getdents d8/d2e/d30 0 2024-03-23T15:44:56.073 INFO:tasks.workunit.client.0.smithi076.stdout:1/323: link d0/d3c/l42 d0/d35/d40/d47/l66 0 2024-03-23T15:44:56.073 INFO:tasks.workunit.client.0.smithi076.stdout:1/324: truncate d0/d35/f64 705300 0 2024-03-23T15:44:56.075 INFO:tasks.workunit.client.0.smithi076.stdout:1/325: rmdir d0 39 2024-03-23T15:44:56.075 INFO:tasks.workunit.client.0.smithi076.stdout:1/326: write d0/d2/f48 [4901828,63571] 0 2024-03-23T15:44:56.075 INFO:tasks.workunit.client.0.smithi076.stdout:1/327: readlink d0/d35/d40/d47/d17/l5f 0 2024-03-23T15:44:56.077 INFO:tasks.workunit.client.0.smithi076.stdout:6/394: link d1/d4/d10/d6b/l6f d1/d81/l88 0 2024-03-23T15:44:56.081 INFO:tasks.workunit.client.0.smithi076.stdout:1/328: dread d0/d35/d40/d47/f9 [0,4194304] 0 2024-03-23T15:44:56.084 INFO:tasks.workunit.client.0.smithi076.stdout:1/329: mknod d0/d35/d40/c67 0 2024-03-23T15:44:56.084 INFO:tasks.workunit.client.0.smithi076.stdout:1/330: write d0/d35/d40/d47/d17/f30 [1023314,87242] 0 2024-03-23T15:44:56.084 INFO:tasks.workunit.client.0.smithi076.stdout:1/331: readlink d0/d35/d52/l59 0 2024-03-23T15:44:56.088 INFO:tasks.workunit.client.0.smithi076.stdout:9/616: dwrite d1/d2f/d3a/d74/fb3 [0,4194304] 0 2024-03-23T15:44:56.088 INFO:tasks.workunit.client.0.smithi076.stdout:9/617: write d1/d20/fa8 [494544,118530] 0 2024-03-23T15:44:56.089 INFO:tasks.workunit.client.0.smithi076.stdout:9/618: truncate d1/d1f/d29/d2d/f6f 667419 0 2024-03-23T15:44:56.091 INFO:tasks.workunit.client.0.smithi076.stdout:2/605: dwrite d1/d4/d64/fc7 [0,4194304] 0 2024-03-23T15:44:56.091 INFO:tasks.workunit.client.0.smithi076.stdout:2/606: stat d1/d4/db/d15/d29/d7d/fad 0 2024-03-23T15:44:56.091 INFO:tasks.workunit.client.0.smithi076.stdout:2/607: write d1/d4/f8d [390971,118262] 0 2024-03-23T15:44:56.092 INFO:tasks.workunit.client.0.smithi076.stdout:9/619: unlink d1/d1f/d29/d2d/d37/l71 0 2024-03-23T15:44:56.093 INFO:tasks.workunit.client.0.smithi076.stdout:9/620: rmdir d1/d1f/d82/d9c 39 2024-03-23T15:44:56.093 INFO:tasks.workunit.client.0.smithi076.stdout:5/595: dwrite d1a/d33/dc3/da6/fbc [4194304,4194304] 0 2024-03-23T15:44:56.094 INFO:tasks.workunit.client.0.smithi076.stdout:5/596: symlink d1a/d33/dc3/da6/lcb 0 2024-03-23T15:44:56.099 INFO:tasks.workunit.client.0.smithi076.stdout:6/395: dwrite d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:56.100 INFO:tasks.workunit.client.0.smithi076.stdout:9/621: write d1/f48 [3130298,29528] 0 2024-03-23T15:44:56.100 INFO:tasks.workunit.client.0.smithi076.stdout:9/622: truncate d1/d2f/d3a/f84 2161427 0 2024-03-23T15:44:56.100 INFO:tasks.workunit.client.0.smithi076.stdout:9/623: getdents d1/d20 0 2024-03-23T15:44:56.100 INFO:tasks.workunit.client.0.smithi076.stdout:2/608: unlink d1/d4/d64/d1d/f3c 0 2024-03-23T15:44:56.101 INFO:tasks.workunit.client.0.smithi076.stdout:2/609: getdents d1/d4/db 0 2024-03-23T15:44:56.101 INFO:tasks.workunit.client.0.smithi076.stdout:6/396: symlink d1/d81/l89 0 2024-03-23T15:44:56.101 INFO:tasks.workunit.client.0.smithi076.stdout:6/397: fdatasync d1/dd/df/f66 0 2024-03-23T15:44:56.102 INFO:tasks.workunit.client.0.smithi076.stdout:6/398: fdatasync d1/d4/d10/f30 0 2024-03-23T15:44:56.103 INFO:tasks.workunit.client.0.smithi076.stdout:5/597: write d1a/f92 [1973794,59475] 0 2024-03-23T15:44:56.103 INFO:tasks.workunit.client.0.smithi076.stdout:6/399: truncate d1/d4/d10/f34 1972155 0 2024-03-23T15:44:56.104 INFO:tasks.workunit.client.0.smithi076.stdout:2/610: write d1/f5 [4411694,14600] 0 2024-03-23T15:44:56.105 INFO:tasks.workunit.client.0.smithi076.stdout:5/598: truncate d1a/d7e/fab 794815 0 2024-03-23T15:44:56.107 INFO:tasks.workunit.client.0.smithi076.stdout:2/611: truncate d1/d4/db/d15/f21 2806662 0 2024-03-23T15:44:56.107 INFO:tasks.workunit.client.0.smithi076.stdout:2/612: read d1/d4/db/d15/d29/d7d/fad [421787,10596] 0 2024-03-23T15:44:56.107 INFO:tasks.workunit.client.0.smithi076.stdout:2/613: dread - d1/d2e/d6a/f9b zero size 2024-03-23T15:44:56.107 INFO:tasks.workunit.client.0.smithi076.stdout:2/614: chown d1/d2e/cb4 126622 1 2024-03-23T15:44:56.107 INFO:tasks.workunit.client.0.smithi076.stdout:2/615: dread - d1/d4/db/d3e/f84 zero size 2024-03-23T15:44:56.107 INFO:tasks.workunit.client.0.smithi076.stdout:8/490: rename d8/d26/d33/f59 to d8/d26/d27/f9e 0 2024-03-23T15:44:56.107 INFO:tasks.workunit.client.0.smithi076.stdout:8/491: stat d8/f2a 0 2024-03-23T15:44:56.108 INFO:tasks.workunit.client.0.smithi076.stdout:5/599: creat d1a/d33/d36/fcc x:0 0 0 2024-03-23T15:44:56.108 INFO:tasks.workunit.client.0.smithi076.stdout:5/600: chown d1a/f2a 2267 1 2024-03-23T15:44:56.108 INFO:tasks.workunit.client.0.smithi076.stdout:2/616: mkdir d1/d2e/dd1 0 2024-03-23T15:44:56.110 INFO:tasks.workunit.client.0.smithi076.stdout:5/601: symlink d1a/d33/dc3/lcd 0 2024-03-23T15:44:56.110 INFO:tasks.workunit.client.0.smithi076.stdout:5/602: chown d1a/d33/d3a/f43 900 1 2024-03-23T15:44:56.112 INFO:tasks.workunit.client.0.smithi076.stdout:5/603: creat d1a/d2c/fce x:0 0 0 2024-03-23T15:44:56.112 INFO:tasks.workunit.client.0.smithi076.stdout:3/433: sync 2024-03-23T15:44:56.114 INFO:tasks.workunit.client.0.smithi076.stdout:1/332: rename d0/d2/d1f/f31 to d0/d35/f68 0 2024-03-23T15:44:56.114 INFO:tasks.workunit.client.0.smithi076.stdout:5/604: mkdir d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf 0 2024-03-23T15:44:56.115 INFO:tasks.workunit.client.0.smithi076.stdout:5/605: mknod d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/cd0 0 2024-03-23T15:44:56.116 INFO:tasks.workunit.client.0.smithi076.stdout:7/714: sync 2024-03-23T15:44:56.117 INFO:tasks.workunit.client.0.smithi076.stdout:2/617: rename d1/d4/c35 to d1/d4/d64/d82/d7f/cd2 0 2024-03-23T15:44:56.118 INFO:tasks.workunit.client.0.smithi076.stdout:5/606: symlink d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/ld1 0 2024-03-23T15:44:56.118 INFO:tasks.workunit.client.0.smithi076.stdout:2/618: dread d1/d4/fb9 [0,4194304] 0 2024-03-23T15:44:56.118 INFO:tasks.workunit.client.0.smithi076.stdout:9/624: getdents d1/d1f/d29 0 2024-03-23T15:44:56.119 INFO:tasks.workunit.client.0.smithi076.stdout:1/333: symlink d0/d35/d40/d47/d55/l69 0 2024-03-23T15:44:56.119 INFO:tasks.workunit.client.0.smithi076.stdout:2/619: write d1/d2e/d8a/f94 [281682,103519] 0 2024-03-23T15:44:56.120 INFO:tasks.workunit.client.0.smithi076.stdout:1/334: symlink d0/d35/d40/l6a 0 2024-03-23T15:44:56.121 INFO:tasks.workunit.client.0.smithi076.stdout:5/607: rename d1a/d33/d36/f5e to d1a/d33/d3a/d67/da8/dc4/d5f/fd2 0 2024-03-23T15:44:56.121 INFO:tasks.workunit.client.0.smithi076.stdout:5/608: stat d1a/d33/d3a/d67/da8/dc4/d65 0 2024-03-23T15:44:56.121 INFO:tasks.workunit.client.0.smithi076.stdout:5/609: chown d1a/d33/dc3 21776 1 2024-03-23T15:44:56.122 INFO:tasks.workunit.client.0.smithi076.stdout:3/434: rmdir d5/d17/d1c/d32 39 2024-03-23T15:44:56.124 INFO:tasks.workunit.client.0.smithi076.stdout:5/610: creat d1a/d33/d3a/d67/d84/fd3 x:0 0 0 2024-03-23T15:44:56.124 INFO:tasks.workunit.client.0.smithi076.stdout:5/611: stat d1a/d33/f54 0 2024-03-23T15:44:56.124 INFO:tasks.workunit.client.0.smithi076.stdout:3/435: symlink d5/d17/d1c/d32/d39/l91 0 2024-03-23T15:44:56.126 INFO:tasks.workunit.client.0.smithi076.stdout:5/612: symlink d1a/d4b/ld4 0 2024-03-23T15:44:56.126 INFO:tasks.workunit.client.0.smithi076.stdout:5/613: write d1a/d33/d36/fc0 [338762,25065] 0 2024-03-23T15:44:56.126 INFO:tasks.workunit.client.0.smithi076.stdout:3/436: mkdir d5/d17/d1c/d31/d61/d92 0 2024-03-23T15:44:56.126 INFO:tasks.workunit.client.0.smithi076.stdout:2/620: dread d1/d4/d52/d6e/faa [4194304,4194304] 0 2024-03-23T15:44:56.127 INFO:tasks.workunit.client.0.smithi076.stdout:8/492: dwrite d8/f16 [0,4194304] 0 2024-03-23T15:44:56.127 INFO:tasks.workunit.client.0.smithi076.stdout:3/437: dread - d5/d17/d1c/d32/d39/f3e zero size 2024-03-23T15:44:56.127 INFO:tasks.workunit.client.0.smithi076.stdout:3/438: chown d5/d17/d1c/d32/d39/d59/f60 1771260 1 2024-03-23T15:44:56.127 INFO:tasks.workunit.client.0.smithi076.stdout:2/621: rmdir d1/d4/d64/d1b/d32 39 2024-03-23T15:44:56.128 INFO:tasks.workunit.client.0.smithi076.stdout:5/614: rename d1a/d4b/f4c to d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fd5 0 2024-03-23T15:44:56.128 INFO:tasks.workunit.client.0.smithi076.stdout:7/715: dread d8/de/d13/f94 [0,4194304] 0 2024-03-23T15:44:56.129 INFO:tasks.workunit.client.0.smithi076.stdout:2/622: creat d1/dbf/fd3 x:0 0 0 2024-03-23T15:44:56.130 INFO:tasks.workunit.client.0.smithi076.stdout:3/439: creat d5/d17/d1c/d32/d5d/f93 x:0 0 0 2024-03-23T15:44:56.130 INFO:tasks.workunit.client.0.smithi076.stdout:6/400: dwrite d1/d2/f71 [0,4194304] 0 2024-03-23T15:44:56.130 INFO:tasks.workunit.client.0.smithi076.stdout:9/625: dread d1/f1a [0,4194304] 0 2024-03-23T15:44:56.131 INFO:tasks.workunit.client.0.smithi076.stdout:2/623: mkdir d1/d4/d64/d1b/dd4 0 2024-03-23T15:44:56.131 INFO:tasks.workunit.client.0.smithi076.stdout:5/615: unlink d1a/d33/d3a/d67/d84/lb1 0 2024-03-23T15:44:56.132 INFO:tasks.workunit.client.0.smithi076.stdout:8/493: write d8/d2e/d30/f66 [3022,102901] 0 2024-03-23T15:44:56.132 INFO:tasks.workunit.client.0.smithi076.stdout:8/494: fsync d8/d26/d27/f37 0 2024-03-23T15:44:56.132 INFO:tasks.workunit.client.0.smithi076.stdout:3/440: truncate d5/d17/d1c/d32/d4a/f55 1982991 0 2024-03-23T15:44:56.133 INFO:tasks.workunit.client.0.smithi076.stdout:6/401: dread d1/d2/d27/f46 [0,4194304] 0 2024-03-23T15:44:56.134 INFO:tasks.workunit.client.0.smithi076.stdout:2/624: creat d1/d2e/fd5 x:0 0 0 2024-03-23T15:44:56.134 INFO:tasks.workunit.client.0.smithi076.stdout:2/625: fsync d1/d4/db/d15/d29/d7d/fad 0 2024-03-23T15:44:56.136 INFO:tasks.workunit.client.0.smithi076.stdout:3/441: creat d5/d35/d3c/f94 x:0 0 0 2024-03-23T15:44:56.136 INFO:tasks.workunit.client.0.smithi076.stdout:6/402: symlink d1/d2/d27/d60/l8a 0 2024-03-23T15:44:56.136 INFO:tasks.workunit.client.0.smithi076.stdout:6/403: write d1/dd/df/d5b/f3e [4655094,37689] 0 2024-03-23T15:44:56.136 INFO:tasks.workunit.client.0.smithi076.stdout:1/335: dwrite d0/d35/f64 [0,4194304] 0 2024-03-23T15:44:56.139 INFO:tasks.workunit.client.0.smithi076.stdout:2/626: creat d1/d2e/d8a/fd6 x:0 0 0 2024-03-23T15:44:56.139 INFO:tasks.workunit.client.0.smithi076.stdout:2/627: write d1/d4/db/d15/d29/d7d/fc9 [998434,22799] 0 2024-03-23T15:44:56.139 INFO:tasks.workunit.client.0.smithi076.stdout:3/442: symlink d5/d35/d6d/l95 0 2024-03-23T15:44:56.141 INFO:tasks.workunit.client.0.smithi076.stdout:8/495: getdents d8/d26/d33 0 2024-03-23T15:44:56.144 INFO:tasks.workunit.client.0.smithi076.stdout:2/628: symlink d1/d2e/ld7 0 2024-03-23T15:44:56.145 INFO:tasks.workunit.client.0.smithi076.stdout:7/716: unlink d8/de/d13/d3a/dd2/fdd 0 2024-03-23T15:44:56.145 INFO:tasks.workunit.client.0.smithi076.stdout:7/717: fdatasync d8/d4e/d8d/dbe/fdb 0 2024-03-23T15:44:56.146 INFO:tasks.workunit.client.0.smithi076.stdout:8/496: truncate d8/f16 2286007 0 2024-03-23T15:44:56.146 INFO:tasks.workunit.client.0.smithi076.stdout:2/629: mkdir d1/d4/d64/d1b/d32/d79/dac/dd8 0 2024-03-23T15:44:56.147 INFO:tasks.workunit.client.0.smithi076.stdout:7/718: mkdir d8/de/d99/dc4/df1 0 2024-03-23T15:44:56.148 INFO:tasks.workunit.client.0.smithi076.stdout:3/443: unlink d5/d17/d1c/d31/d6b/l86 0 2024-03-23T15:44:56.149 INFO:tasks.workunit.client.0.smithi076.stdout:3/444: chown d5/ff 0 1 2024-03-23T15:44:56.149 INFO:tasks.workunit.client.0.smithi076.stdout:7/719: link d8/de/d99/dc4/ccf d8/cf2 0 2024-03-23T15:44:56.149 INFO:tasks.workunit.client.0.smithi076.stdout:5/616: dread f4 [0,4194304] 0 2024-03-23T15:44:56.149 INFO:tasks.workunit.client.0.smithi076.stdout:7/720: mknod d8/d4e/d8d/dbe/cf3 0 2024-03-23T15:44:56.150 INFO:tasks.workunit.client.0.smithi076.stdout:7/721: dread - d8/fcb zero size 2024-03-23T15:44:56.150 INFO:tasks.workunit.client.0.smithi076.stdout:8/497: mknod d8/d2e/d3e/c9f 0 2024-03-23T15:44:56.150 INFO:tasks.workunit.client.0.smithi076.stdout:5/617: truncate d1a/f31 1058884 0 2024-03-23T15:44:56.151 INFO:tasks.workunit.client.0.smithi076.stdout:1/336: dwrite d0/d35/d40/d47/d17/d20/f46 [0,4194304] 0 2024-03-23T15:44:56.151 INFO:tasks.workunit.client.0.smithi076.stdout:2/630: truncate d1/d4/d64/d1d/fa8 744167 0 2024-03-23T15:44:56.151 INFO:tasks.workunit.client.0.smithi076.stdout:2/631: chown d1/d4/db/d15/la2 3 1 2024-03-23T15:44:56.152 INFO:tasks.workunit.client.0.smithi076.stdout:9/626: dwrite d1/d20/f97 [0,4194304] 0 2024-03-23T15:44:56.155 INFO:tasks.workunit.client.0.smithi076.stdout:5/618: creat d1a/d33/dc3/fd6 x:0 0 0 2024-03-23T15:44:56.156 INFO:tasks.workunit.client.0.smithi076.stdout:8/498: symlink d8/d6b/d8e/la0 0 2024-03-23T15:44:56.156 INFO:tasks.workunit.client.0.smithi076.stdout:8/499: chown d8/f10 139646 1 2024-03-23T15:44:56.156 INFO:tasks.workunit.client.0.smithi076.stdout:8/500: chown d8/d89/l58 420 1 2024-03-23T15:44:56.157 INFO:tasks.workunit.client.0.smithi076.stdout:9/627: getdents d1/d1f/d29/d2d/d37 0 2024-03-23T15:44:56.157 INFO:tasks.workunit.client.0.smithi076.stdout:9/628: write d1/d2f/d3a/f96 [5239739,73045] 0 2024-03-23T15:44:56.159 INFO:tasks.workunit.client.0.smithi076.stdout:3/445: rmdir d5/d17/d1c/d32/d39/d59 39 2024-03-23T15:44:56.159 INFO:tasks.workunit.client.0.smithi076.stdout:8/501: mknod d8/d4f/ca1 0 2024-03-23T15:44:56.159 INFO:tasks.workunit.client.0.smithi076.stdout:8/502: truncate d8/d2e/d30/d69/f8f 811982 0 2024-03-23T15:44:56.160 INFO:tasks.workunit.client.0.smithi076.stdout:5/619: mknod d1a/d33/d3a/d67/cd7 0 2024-03-23T15:44:56.161 INFO:tasks.workunit.client.0.smithi076.stdout:3/446: mkdir d5/d17/d1c/d31/d61/d96 0 2024-03-23T15:44:56.161 INFO:tasks.workunit.client.0.smithi076.stdout:3/447: write d5/f1f [502383,35470] 0 2024-03-23T15:44:56.166 INFO:tasks.workunit.client.0.smithi076.stdout:7/722: dwrite d8/de/d13/d40/d88/f33 [0,4194304] 0 2024-03-23T15:44:56.168 INFO:tasks.workunit.client.0.smithi076.stdout:7/723: mkdir d8/de/d13/d40/d5d/d3c/df4 0 2024-03-23T15:44:56.168 INFO:tasks.workunit.client.0.smithi076.stdout:7/724: write d8/de/d13/d40/d88/f57 [5095395,46406] 0 2024-03-23T15:44:56.168 INFO:tasks.workunit.client.0.smithi076.stdout:7/725: write d8/de/f6f [1092446,5870] 0 2024-03-23T15:44:56.169 INFO:tasks.workunit.client.0.smithi076.stdout:8/503: creat d8/d2e/d30/fa2 x:0 0 0 2024-03-23T15:44:56.170 INFO:tasks.workunit.client.0.smithi076.stdout:7/726: write d8/de/d13/d40/d88/fb2 [183907,127667] 0 2024-03-23T15:44:56.170 INFO:tasks.workunit.client.0.smithi076.stdout:3/448: rename d5/d17/d1c/d32/f34 to d5/d17/d1c/d32/f97 0 2024-03-23T15:44:56.171 INFO:tasks.workunit.client.0.smithi076.stdout:7/727: write d8/de/d13/d3a/f8b [312323,118680] 0 2024-03-23T15:44:56.172 INFO:tasks.workunit.client.0.smithi076.stdout:1/337: dwrite d0/d35/d40/d47/d17/f38 [0,4194304] 0 2024-03-23T15:44:56.172 INFO:tasks.workunit.client.0.smithi076.stdout:8/504: mkdir d8/d4f/d80/da3 0 2024-03-23T15:44:56.172 INFO:tasks.workunit.client.0.smithi076.stdout:1/338: readlink d0/d2/d1f/l24 0 2024-03-23T15:44:56.172 INFO:tasks.workunit.client.0.smithi076.stdout:7/728: readlink d8/de/d13/d3a/l60 0 2024-03-23T15:44:56.176 INFO:tasks.workunit.client.0.smithi076.stdout:7/729: rmdir d8/de/d13/d40/d5d/d3c/d8e 39 2024-03-23T15:44:56.177 INFO:tasks.workunit.client.0.smithi076.stdout:8/505: stat d8/l39 0 2024-03-23T15:44:56.177 INFO:tasks.workunit.client.0.smithi076.stdout:8/506: write d8/d26/d33/f86 [618792,69504] 0 2024-03-23T15:44:56.178 INFO:tasks.workunit.client.0.smithi076.stdout:7/730: creat d8/de/d13/d24/d91/ff5 x:0 0 0 2024-03-23T15:44:56.179 INFO:tasks.workunit.client.0.smithi076.stdout:8/507: symlink d8/d26/d27/d56/la4 0 2024-03-23T15:44:56.180 INFO:tasks.workunit.client.0.smithi076.stdout:7/731: truncate d8/de/d13/f15 629866 0 2024-03-23T15:44:56.181 INFO:tasks.workunit.client.0.smithi076.stdout:7/732: mkdir d8/d4e/d98/df6 0 2024-03-23T15:44:56.181 INFO:tasks.workunit.client.0.smithi076.stdout:7/733: chown d8/de/d13/d3a/f50 3 1 2024-03-23T15:44:56.182 INFO:tasks.workunit.client.0.smithi076.stdout:8/508: dread d8/d26/d33/f3a [0,4194304] 0 2024-03-23T15:44:56.182 INFO:tasks.workunit.client.0.smithi076.stdout:5/620: dwrite d1a/d33/d3a/f8a [0,4194304] 0 2024-03-23T15:44:56.184 INFO:tasks.workunit.client.0.smithi076.stdout:3/449: dread d5/d35/d4c/f51 [0,4194304] 0 2024-03-23T15:44:56.184 INFO:tasks.workunit.client.0.smithi076.stdout:7/734: unlink d8/de/d99/fc0 0 2024-03-23T15:44:56.186 INFO:tasks.workunit.client.0.smithi076.stdout:9/629: dwrite d1/d1f/d82/f9e [0,4194304] 0 2024-03-23T15:44:56.186 INFO:tasks.workunit.client.0.smithi076.stdout:8/509: creat d8/d26/d27/fa5 x:0 0 0 2024-03-23T15:44:56.188 INFO:tasks.workunit.client.0.smithi076.stdout:9/630: dread d1/d2f/fbf [0,4194304] 0 2024-03-23T15:44:56.188 INFO:tasks.workunit.client.0.smithi076.stdout:9/631: readlink d1/l9b 0 2024-03-23T15:44:56.189 INFO:tasks.workunit.client.0.smithi076.stdout:8/510: creat d8/d6b/d9a/fa6 x:0 0 0 2024-03-23T15:44:56.190 INFO:tasks.workunit.client.0.smithi076.stdout:0/635: sync 2024-03-23T15:44:56.190 INFO:tasks.workunit.client.0.smithi076.stdout:4/607: sync 2024-03-23T15:44:56.193 INFO:tasks.workunit.client.0.smithi076.stdout:8/511: dread d8/d2e/d30/f6a [0,4194304] 0 2024-03-23T15:44:56.195 INFO:tasks.workunit.client.0.smithi076.stdout:4/608: mkdir d8/d3a/d69/d73/d76/db4 0 2024-03-23T15:44:56.195 INFO:tasks.workunit.client.0.smithi076.stdout:4/609: truncate d8/d3a/f34 162942 0 2024-03-23T15:44:56.198 INFO:tasks.workunit.client.0.smithi076.stdout:0/636: getdents d2/de/d10/d3a/d47/d69 0 2024-03-23T15:44:56.199 INFO:tasks.workunit.client.0.smithi076.stdout:4/610: symlink d8/d3a/d18/d52/lb5 0 2024-03-23T15:44:56.199 INFO:tasks.workunit.client.0.smithi076.stdout:4/611: getdents d8/d3a/d18/d75/d8d/db0 0 2024-03-23T15:44:56.201 INFO:tasks.workunit.client.0.smithi076.stdout:5/621: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fa0 [0,4194304] 0 2024-03-23T15:44:56.201 INFO:tasks.workunit.client.0.smithi076.stdout:1/339: rmdir d0/d35/d40/d47 39 2024-03-23T15:44:56.203 INFO:tasks.workunit.client.0.smithi076.stdout:3/450: read d5/d35/d4c/d5a/f41 [1703233,89145] 0 2024-03-23T15:44:56.204 INFO:tasks.workunit.client.0.smithi076.stdout:1/340: creat d0/d35/d40/d47/f6b x:0 0 0 2024-03-23T15:44:56.206 INFO:tasks.workunit.client.0.smithi076.stdout:3/451: truncate d5/d17/d1c/f52 740046 0 2024-03-23T15:44:56.206 INFO:tasks.workunit.client.0.smithi076.stdout:3/452: chown d5/d35/d3c/l4e 6602 1 2024-03-23T15:44:56.206 INFO:tasks.workunit.client.0.smithi076.stdout:3/453: write d5/d17/d1c/d31/d6b/f8a [3725962,123501] 0 2024-03-23T15:44:56.207 INFO:tasks.workunit.client.0.smithi076.stdout:4/612: unlink d8/d3a/d3b/l50 0 2024-03-23T15:44:56.207 INFO:tasks.workunit.client.0.smithi076.stdout:4/613: stat d8/c31 0 2024-03-23T15:44:56.209 INFO:tasks.workunit.client.0.smithi076.stdout:4/614: dread d8/d3a/d3b/f25 [0,4194304] 0 2024-03-23T15:44:56.209 INFO:tasks.workunit.client.0.smithi076.stdout:4/615: fsync d8/f71 0 2024-03-23T15:44:56.209 INFO:tasks.workunit.client.0.smithi076.stdout:1/341: creat d0/d3c/f6c x:0 0 0 2024-03-23T15:44:56.209 INFO:tasks.workunit.client.0.smithi076.stdout:1/342: truncate d0/f5e 549565 0 2024-03-23T15:44:56.210 INFO:tasks.workunit.client.0.smithi076.stdout:3/454: link d5/d17/d1c/d31/d61/l77 d5/d17/d1c/d32/d8e/l98 0 2024-03-23T15:44:56.212 INFO:tasks.workunit.client.0.smithi076.stdout:4/616: truncate d8/d3a/d18/f20 896601 0 2024-03-23T15:44:56.218 INFO:tasks.workunit.client.0.smithi076.stdout:8/512: dwrite d8/f10 [0,4194304] 0 2024-03-23T15:44:56.218 INFO:tasks.workunit.client.0.smithi076.stdout:5/622: dwrite d1a/f92 [0,4194304] 0 2024-03-23T15:44:56.220 INFO:tasks.workunit.client.0.smithi076.stdout:8/513: unlink d8/f67 0 2024-03-23T15:44:56.221 INFO:tasks.workunit.client.0.smithi076.stdout:5/623: rename d1a/d33/d3a/d67/f97 to d1a/d33/d3a/d80/fd8 0 2024-03-23T15:44:56.221 INFO:tasks.workunit.client.0.smithi076.stdout:5/624: readlink d1a/d33/d3a/d80/l94 0 2024-03-23T15:44:56.221 INFO:tasks.workunit.client.0.smithi076.stdout:5/625: truncate d1a/f42 1707682 0 2024-03-23T15:44:56.222 INFO:tasks.workunit.client.0.smithi076.stdout:5/626: creat d1a/d33/dc3/da6/fd9 x:0 0 0 2024-03-23T15:44:56.225 INFO:tasks.workunit.client.0.smithi076.stdout:8/514: rename d8/d26/d33/f87 to d8/d2e/d30/d69/fa7 0 2024-03-23T15:44:56.226 INFO:tasks.workunit.client.0.smithi076.stdout:8/515: creat d8/d26/d27/fa8 x:0 0 0 2024-03-23T15:44:56.243 INFO:tasks.workunit.client.0.smithi076.stdout:8/516: dwrite d8/f48 [0,4194304] 0 2024-03-23T15:44:56.255 INFO:tasks.workunit.client.0.smithi076.stdout:8/517: dwrite d8/f1a [0,4194304] 0 2024-03-23T15:44:56.256 INFO:tasks.workunit.client.0.smithi076.stdout:8/518: mkdir d8/d26/d27/da9 0 2024-03-23T15:44:56.257 INFO:tasks.workunit.client.0.smithi076.stdout:7/735: dwrite d8/de/d13/d40/d5d/d3c/f73 [0,4194304] 0 2024-03-23T15:44:56.257 INFO:tasks.workunit.client.0.smithi076.stdout:7/736: fsync d8/de/d13/d40/d88/d5a/dde/fe7 0 2024-03-23T15:44:56.257 INFO:tasks.workunit.client.0.smithi076.stdout:8/519: write d8/d2e/d30/f32 [132615,4088] 0 2024-03-23T15:44:56.259 INFO:tasks.workunit.client.0.smithi076.stdout:7/737: mkdir d8/de/d13/d24/d91/df7 0 2024-03-23T15:44:56.261 INFO:tasks.workunit.client.0.smithi076.stdout:7/738: rename d8/de/d13/d24/d91/d96/d7a/c7b to d8/de/d48/cf8 0 2024-03-23T15:44:56.263 INFO:tasks.workunit.client.0.smithi076.stdout:7/739: creat d8/de/d13/d3a/dbf/dc7/ff9 x:0 0 0 2024-03-23T15:44:56.264 INFO:tasks.workunit.client.0.smithi076.stdout:7/740: rename d8/de/d13/d40/d5d/d3c/c69 to d8/de/d13/d40/d5d/cfa 0 2024-03-23T15:44:56.266 INFO:tasks.workunit.client.0.smithi076.stdout:7/741: link d8/de/d13/l1f d8/de/de8/lfb 0 2024-03-23T15:44:56.267 INFO:tasks.workunit.client.0.smithi076.stdout:7/742: mkdir d8/d4e/d8d/dea/dfc 0 2024-03-23T15:44:56.267 INFO:tasks.workunit.client.0.smithi076.stdout:7/743: readlink d8/de/d13/d24/d91/d96/d7a/lb6 0 2024-03-23T15:44:56.268 INFO:tasks.workunit.client.0.smithi076.stdout:7/744: rmdir d8/de/d13/d24/d91/d96 39 2024-03-23T15:44:56.268 INFO:tasks.workunit.client.0.smithi076.stdout:7/745: stat d8/de/d13/d24/f3b 0 2024-03-23T15:44:56.268 INFO:tasks.workunit.client.0.smithi076.stdout:7/746: truncate d8/fd 5693954 0 2024-03-23T15:44:56.268 INFO:tasks.workunit.client.0.smithi076.stdout:7/747: fdatasync d8/f83 0 2024-03-23T15:44:56.270 INFO:tasks.workunit.client.0.smithi076.stdout:7/748: mkdir d8/de/d13/d18/dfd 0 2024-03-23T15:44:56.272 INFO:tasks.workunit.client.0.smithi076.stdout:7/749: mknod d8/de/d13/d3a/d85/cfe 0 2024-03-23T15:44:56.272 INFO:tasks.workunit.client.0.smithi076.stdout:7/750: dread - d8/de/fcd zero size 2024-03-23T15:44:56.272 INFO:tasks.workunit.client.0.smithi076.stdout:7/751: write d8/de/d13/d40/d5d/f65 [3370573,34328] 0 2024-03-23T15:44:56.272 INFO:tasks.workunit.client.0.smithi076.stdout:7/752: creat d8/de/d99/fff x:0 0 0 2024-03-23T15:44:56.274 INFO:tasks.workunit.client.0.smithi076.stdout:7/753: creat d8/de/d99/dc4/f100 x:0 0 0 2024-03-23T15:44:56.274 INFO:tasks.workunit.client.0.smithi076.stdout:7/754: fsync d8/f83 0 2024-03-23T15:44:56.274 INFO:tasks.workunit.client.0.smithi076.stdout:8/520: dwrite d8/f48 [0,4194304] 0 2024-03-23T15:44:56.274 INFO:tasks.workunit.client.0.smithi076.stdout:8/521: write d8/d2e/d30/f32 [419056,26038] 0 2024-03-23T15:44:56.275 INFO:tasks.workunit.client.0.smithi076.stdout:7/755: truncate d8/de/d13/d40/d5d/d3c/f73 1617515 0 2024-03-23T15:44:56.276 INFO:tasks.workunit.client.0.smithi076.stdout:7/756: creat d8/de/d13/d40/d5d/f101 x:0 0 0 2024-03-23T15:44:56.287 INFO:tasks.workunit.client.0.smithi076.stdout:3/455: fsync d5/d17/d1c/d31/d6b/f8a 0 2024-03-23T15:44:56.287 INFO:tasks.workunit.client.0.smithi076.stdout:3/456: write d5/d35/d4c/d5a/f41 [3506058,8084] 0 2024-03-23T15:44:56.291 INFO:tasks.workunit.client.0.smithi076.stdout:1/343: getdents d0/d35/d40/d47 0 2024-03-23T15:44:56.292 INFO:tasks.workunit.client.0.smithi076.stdout:3/457: creat d5/d35/d6d/f99 x:0 0 0 2024-03-23T15:44:56.292 INFO:tasks.workunit.client.0.smithi076.stdout:5/627: write d1a/d33/dc3/da6/fbc [8259731,72990] 0 2024-03-23T15:44:56.292 INFO:tasks.workunit.client.0.smithi076.stdout:3/458: write d5/d17/d1c/d32/d4a/f72 [43804,109363] 0 2024-03-23T15:44:56.293 INFO:tasks.workunit.client.0.smithi076.stdout:3/459: write d5/d17/d1c/d32/d39/d59/d79/f8f [179106,57887] 0 2024-03-23T15:44:56.294 INFO:tasks.workunit.client.0.smithi076.stdout:3/460: write d5/f28 [987102,96006] 0 2024-03-23T15:44:56.295 INFO:tasks.workunit.client.0.smithi076.stdout:7/757: dwrite d8/de/d13/d40/d88/fe1 [0,4194304] 0 2024-03-23T15:44:56.296 INFO:tasks.workunit.client.0.smithi076.stdout:8/522: dwrite d8/fe [0,4194304] 0 2024-03-23T15:44:56.296 INFO:tasks.workunit.client.0.smithi076.stdout:0/637: link d2/d8/d2d/l8d d2/d8/d32/lc0 0 2024-03-23T15:44:56.298 INFO:tasks.workunit.client.0.smithi076.stdout:7/758: mkdir d8/de/d13/d40/d88/d5a/dde/d102 0 2024-03-23T15:44:56.300 INFO:tasks.workunit.client.0.smithi076.stdout:7/759: unlink d8/l27 0 2024-03-23T15:44:56.301 INFO:tasks.workunit.client.0.smithi076.stdout:7/760: symlink d8/de/d13/d24/d91/l103 0 2024-03-23T15:44:56.302 INFO:tasks.workunit.client.0.smithi076.stdout:1/344: write d0/d2/f48 [3281948,55749] 0 2024-03-23T15:44:56.305 INFO:tasks.workunit.client.0.smithi076.stdout:5/628: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fd5 [0,4194304] 0 2024-03-23T15:44:56.311 INFO:tasks.workunit.client.0.smithi076.stdout:3/461: unlink d5/d17/d1c/d32/d5d/f93 0 2024-03-23T15:44:56.312 INFO:tasks.workunit.client.0.smithi076.stdout:5/629: write d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fd5 [2459306,48684] 0 2024-03-23T15:44:56.313 INFO:tasks.workunit.client.0.smithi076.stdout:8/523: truncate d8/d2e/d3e/f8d 1875340 0 2024-03-23T15:44:56.315 INFO:tasks.workunit.client.0.smithi076.stdout:5/630: unlink d1a/d33/d36/d8b/lac 0 2024-03-23T15:44:56.315 INFO:tasks.workunit.client.0.smithi076.stdout:5/631: dread - d1a/d33/d3a/d80/fca zero size 2024-03-23T15:44:56.316 INFO:tasks.workunit.client.0.smithi076.stdout:0/638: mknod d2/d8/d52/cc1 0 2024-03-23T15:44:56.316 INFO:tasks.workunit.client.0.smithi076.stdout:0/639: chown d2/de/d71/l75 27246778 1 2024-03-23T15:44:56.317 INFO:tasks.workunit.client.0.smithi076.stdout:5/632: dread d1a/f25 [0,4194304] 0 2024-03-23T15:44:56.317 INFO:tasks.workunit.client.0.smithi076.stdout:0/640: symlink d2/de/d10/d3a/d47/d69/lc2 0 2024-03-23T15:44:56.317 INFO:tasks.workunit.client.0.smithi076.stdout:0/641: chown d2/de/d10/d3a/d47/d69/c6b 29462297 1 2024-03-23T15:44:56.321 INFO:tasks.workunit.client.0.smithi076.stdout:6/404: sync 2024-03-23T15:44:56.321 INFO:tasks.workunit.client.0.smithi076.stdout:2/632: sync 2024-03-23T15:44:56.321 INFO:tasks.workunit.client.0.smithi076.stdout:1/345: dwrite d0/d35/d40/d47/f6b [0,4194304] 0 2024-03-23T15:44:56.324 INFO:tasks.workunit.client.0.smithi076.stdout:2/633: creat d1/fd9 x:0 0 0 2024-03-23T15:44:56.325 INFO:tasks.workunit.client.0.smithi076.stdout:3/462: write d5/d35/d4c/f51 [2372103,102211] 0 2024-03-23T15:44:56.325 INFO:tasks.workunit.client.0.smithi076.stdout:6/405: creat d1/dd/df/d5b/d62/f8b x:0 0 0 2024-03-23T15:44:56.326 INFO:tasks.workunit.client.0.smithi076.stdout:0/642: write d2/d8/f28 [717681,26388] 0 2024-03-23T15:44:56.326 INFO:tasks.workunit.client.0.smithi076.stdout:0/643: dread - d2/de/d10/d3a/d47/d69/fbe zero size 2024-03-23T15:44:56.327 INFO:tasks.workunit.client.0.smithi076.stdout:2/634: mknod d1/cda 0 2024-03-23T15:44:56.328 INFO:tasks.workunit.client.0.smithi076.stdout:3/463: dread d5/d17/d1c/f52 [0,4194304] 0 2024-03-23T15:44:56.328 INFO:tasks.workunit.client.0.smithi076.stdout:2/635: rmdir d1/d4/d64/d1b/d32/d79/dac 39 2024-03-23T15:44:56.329 INFO:tasks.workunit.client.0.smithi076.stdout:6/406: creat d1/dd/d2a/f8c x:0 0 0 2024-03-23T15:44:56.329 INFO:tasks.workunit.client.0.smithi076.stdout:6/407: fdatasync d1/d4/f86 0 2024-03-23T15:44:56.329 INFO:tasks.workunit.client.0.smithi076.stdout:1/346: dread d0/d35/d40/d47/d17/d20/f46 [0,4194304] 0 2024-03-23T15:44:56.329 INFO:tasks.workunit.client.0.smithi076.stdout:3/464: truncate d5/f7 3644152 0 2024-03-23T15:44:56.330 INFO:tasks.workunit.client.0.smithi076.stdout:8/524: dread d8/d2e/f4a [0,4194304] 0 2024-03-23T15:44:56.330 INFO:tasks.workunit.client.0.smithi076.stdout:8/525: stat d8/d2e/d3e/f49 0 2024-03-23T15:44:56.330 INFO:tasks.workunit.client.0.smithi076.stdout:8/526: write d8/d26/d27/fa8 [742531,27460] 0 2024-03-23T15:44:56.330 INFO:tasks.workunit.client.0.smithi076.stdout:8/527: truncate d8/d26/f46 2768039 0 2024-03-23T15:44:56.331 INFO:tasks.workunit.client.0.smithi076.stdout:7/761: rename d8/de/d13 to d8/d4e/d98/df6/d104 0 2024-03-23T15:44:56.332 INFO:tasks.workunit.client.0.smithi076.stdout:8/528: creat d8/d2e/d30/d69/faa x:0 0 0 2024-03-23T15:44:56.333 INFO:tasks.workunit.client.0.smithi076.stdout:8/529: chown d8/d4f/d80 564850 1 2024-03-23T15:44:56.333 INFO:tasks.workunit.client.0.smithi076.stdout:7/762: creat d8/de/f105 x:0 0 0 2024-03-23T15:44:56.333 INFO:tasks.workunit.client.0.smithi076.stdout:0/644: mknod d2/d8/d32/cc3 0 2024-03-23T15:44:56.333 INFO:tasks.workunit.client.0.smithi076.stdout:5/633: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fc1 [0,4194304] 0 2024-03-23T15:44:56.334 INFO:tasks.workunit.client.0.smithi076.stdout:8/530: symlink d8/d2e/lab 0 2024-03-23T15:44:56.334 INFO:tasks.workunit.client.0.smithi076.stdout:2/636: unlink d1/d4/db/d15/d29/d7d/l86 0 2024-03-23T15:44:56.335 INFO:tasks.workunit.client.0.smithi076.stdout:2/637: getdents d1/d4/d64/d1d/dbc 0 2024-03-23T15:44:56.339 INFO:tasks.workunit.client.0.smithi076.stdout:0/645: dread d2/d8/f4f [0,4194304] 0 2024-03-23T15:44:56.340 INFO:tasks.workunit.client.0.smithi076.stdout:5/634: creat d1a/d33/d3a/fda x:0 0 0 2024-03-23T15:44:56.341 INFO:tasks.workunit.client.0.smithi076.stdout:5/635: truncate d1a/d33/d3a/d67/da8/dc4/f93 1027801 0 2024-03-23T15:44:56.342 INFO:tasks.workunit.client.0.smithi076.stdout:1/347: dwrite d0/d35/d40/d47/d17/d20/f25 [0,4194304] 0 2024-03-23T15:44:56.344 INFO:tasks.workunit.client.0.smithi076.stdout:5/636: mkdir d1a/d33/d3a/d67/da8/dc4/d5f/ddb 0 2024-03-23T15:44:56.344 INFO:tasks.workunit.client.0.smithi076.stdout:5/637: readlink d1a/d33/d3a/laf 0 2024-03-23T15:44:56.345 INFO:tasks.workunit.client.0.smithi076.stdout:0/646: creat d2/d1b/fc4 x:0 0 0 2024-03-23T15:44:56.347 INFO:tasks.workunit.client.0.smithi076.stdout:5/638: getdents d1a/d33 0 2024-03-23T15:44:56.347 INFO:tasks.workunit.client.0.smithi076.stdout:5/639: chown d1a/d33/c86 100984 1 2024-03-23T15:44:56.347 INFO:tasks.workunit.client.0.smithi076.stdout:5/640: write d1a/d20/fad [524375,81073] 0 2024-03-23T15:44:56.347 INFO:tasks.workunit.client.0.smithi076.stdout:0/647: mkdir d2/de/d10/d3a/d47/d69/d84/dc5 0 2024-03-23T15:44:56.350 INFO:tasks.workunit.client.0.smithi076.stdout:5/641: symlink d1a/d33/d36/d8b/ldc 0 2024-03-23T15:44:56.350 INFO:tasks.workunit.client.0.smithi076.stdout:5/642: fsync d1a/d33/d3a/d67/da8/dc4/f52 0 2024-03-23T15:44:56.351 INFO:tasks.workunit.client.0.smithi076.stdout:0/648: mkdir d2/de/d10/dc6 0 2024-03-23T15:44:56.351 INFO:tasks.workunit.client.0.smithi076.stdout:5/643: mknod d1a/d33/d3a/d80/cdd 0 2024-03-23T15:44:56.352 INFO:tasks.workunit.client.0.smithi076.stdout:5/644: write d1a/d33/d3a/fa5 [570886,82012] 0 2024-03-23T15:44:56.353 INFO:tasks.workunit.client.0.smithi076.stdout:0/649: unlink d2/de/d10/d3a/d47/c7e 0 2024-03-23T15:44:56.353 INFO:tasks.workunit.client.0.smithi076.stdout:0/650: chown d2/de/d10/d43/c89 72065 1 2024-03-23T15:44:56.366 INFO:tasks.workunit.client.0.smithi076.stdout:3/465: link d5/d17/d1c/d32/d39/c6f d5/d35/d6d/d7f/c9a 0 2024-03-23T15:44:56.368 INFO:tasks.workunit.client.0.smithi076.stdout:3/466: getdents d5/d35 0 2024-03-23T15:44:56.369 INFO:tasks.workunit.client.0.smithi076.stdout:0/651: dwrite d2/d1b/fc4 [0,4194304] 0 2024-03-23T15:44:56.369 INFO:tasks.workunit.client.0.smithi076.stdout:0/652: readlink d2/de/d10/d12/da6/lba 0 2024-03-23T15:44:56.369 INFO:tasks.workunit.client.0.smithi076.stdout:0/653: fdatasync d2/fad 0 2024-03-23T15:44:56.369 INFO:tasks.workunit.client.0.smithi076.stdout:0/654: fdatasync d2/d8/f25 0 2024-03-23T15:44:56.370 INFO:tasks.workunit.client.0.smithi076.stdout:5/645: dwrite d1a/d2c/fbe [0,4194304] 0 2024-03-23T15:44:56.370 INFO:tasks.workunit.client.0.smithi076.stdout:2/638: link d1/d4/d64/d1d/c28 d1/d4/cdb 0 2024-03-23T15:44:56.372 INFO:tasks.workunit.client.0.smithi076.stdout:0/655: symlink d2/d8/lc7 0 2024-03-23T15:44:56.372 INFO:tasks.workunit.client.0.smithi076.stdout:0/656: stat d2/de/d10/d12/f23 0 2024-03-23T15:44:56.372 INFO:tasks.workunit.client.0.smithi076.stdout:0/657: write d2/fa7 [826578,8993] 0 2024-03-23T15:44:56.372 INFO:tasks.workunit.client.0.smithi076.stdout:2/639: mkdir d1/d2e/dd1/ddc 0 2024-03-23T15:44:56.372 INFO:tasks.workunit.client.0.smithi076.stdout:2/640: read - d1/d4/db/d15/fb3 zero size 2024-03-23T15:44:56.373 INFO:tasks.workunit.client.0.smithi076.stdout:2/641: link d1/d4/c5c d1/d4/d64/d1b/d32/cdd 0 2024-03-23T15:44:56.373 INFO:tasks.workunit.client.0.smithi076.stdout:5/646: mknod d1a/d7e/cde 0 2024-03-23T15:44:56.378 INFO:tasks.workunit.client.0.smithi076.stdout:2/642: getdents d1/d4 0 2024-03-23T15:44:56.378 INFO:tasks.workunit.client.0.smithi076.stdout:5/647: mkdir d1a/d33/d36/d8b/ddf 0 2024-03-23T15:44:56.378 INFO:tasks.workunit.client.0.smithi076.stdout:3/467: dread d5/d17/d1c/d32/d39/d59/f88 [0,4194304] 0 2024-03-23T15:44:56.378 INFO:tasks.workunit.client.0.smithi076.stdout:2/643: mknod d1/d4/d52/d6e/cde 0 2024-03-23T15:44:56.379 INFO:tasks.workunit.client.0.smithi076.stdout:6/408: rename d1/d2/d27 to d1/d4/d10/d6b/d8d 0 2024-03-23T15:44:56.381 INFO:tasks.workunit.client.0.smithi076.stdout:3/468: getdents d5/d35/d3c 0 2024-03-23T15:44:56.381 INFO:tasks.workunit.client.0.smithi076.stdout:7/763: rename d8/d4e/d98/df6/d104/d40/d88/d5a/ca3 to d8/de/d48/d92/c106 0 2024-03-23T15:44:56.381 INFO:tasks.workunit.client.0.smithi076.stdout:7/764: write d8/d4e/d98/df6/d104/d40/d88/f57 [5979837,70380] 0 2024-03-23T15:44:56.381 INFO:tasks.workunit.client.0.smithi076.stdout:7/765: readlink d8/d4e/d98/df6/d104/d40/l6b 0 2024-03-23T15:44:56.381 INFO:tasks.workunit.client.0.smithi076.stdout:7/766: write d8/d4e/d98/df6/d104/d40/d5d/f101 [229347,116497] 0 2024-03-23T15:44:56.381 INFO:tasks.workunit.client.0.smithi076.stdout:4/617: sync 2024-03-23T15:44:56.381 INFO:tasks.workunit.client.0.smithi076.stdout:9/632: sync 2024-03-23T15:44:56.382 INFO:tasks.workunit.client.0.smithi076.stdout:2/644: creat d1/d4/d64/d1d/dbc/fdf x:0 0 0 2024-03-23T15:44:56.382 INFO:tasks.workunit.client.0.smithi076.stdout:7/767: symlink d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/db9/l107 0 2024-03-23T15:44:56.384 INFO:tasks.workunit.client.0.smithi076.stdout:4/618: creat d8/d3a/d69/d73/d85/da9/fb6 x:0 0 0 2024-03-23T15:44:56.385 INFO:tasks.workunit.client.0.smithi076.stdout:4/619: write d8/f1a [8452515,114785] 0 2024-03-23T15:44:56.385 INFO:tasks.workunit.client.0.smithi076.stdout:4/620: write d8/f56 [942521,52272] 0 2024-03-23T15:44:56.385 INFO:tasks.workunit.client.0.smithi076.stdout:9/633: stat d1/d20/l56 0 2024-03-23T15:44:56.388 INFO:tasks.workunit.client.0.smithi076.stdout:7/768: mkdir d8/d4e/d98/df6/d104/d3a/dbf/d108 0 2024-03-23T15:44:56.389 INFO:tasks.workunit.client.0.smithi076.stdout:0/658: dwrite d2/d1b/fa8 [0,4194304] 0 2024-03-23T15:44:56.389 INFO:tasks.workunit.client.0.smithi076.stdout:4/621: link d8/f1a d8/d41/d97/fb7 0 2024-03-23T15:44:56.391 INFO:tasks.workunit.client.0.smithi076.stdout:9/634: creat d1/d2f/d3f/da0/fc9 x:0 0 0 2024-03-23T15:44:56.392 INFO:tasks.workunit.client.0.smithi076.stdout:2/645: dread d1/d4/f98 [4194304,4194304] 0 2024-03-23T15:44:56.392 INFO:tasks.workunit.client.0.smithi076.stdout:2/646: chown d1/d4/d64/d7c/fc3 5073 1 2024-03-23T15:44:56.393 INFO:tasks.workunit.client.0.smithi076.stdout:5/648: dwrite d1a/d2c/f4a [0,4194304] 0 2024-03-23T15:44:56.393 INFO:tasks.workunit.client.0.smithi076.stdout:3/469: write d5/d17/d1c/d32/f5f [3757697,9391] 0 2024-03-23T15:44:56.394 INFO:tasks.workunit.client.0.smithi076.stdout:5/649: symlink d1a/d2c/le0 0 2024-03-23T15:44:56.395 INFO:tasks.workunit.client.0.smithi076.stdout:4/622: creat d8/fb8 x:0 0 0 2024-03-23T15:44:56.396 INFO:tasks.workunit.client.0.smithi076.stdout:6/409: dwrite d1/dd/df/d5b/d62/f6e [0,4194304] 0 2024-03-23T15:44:56.398 INFO:tasks.workunit.client.0.smithi076.stdout:6/410: symlink d1/dd/df/l8e 0 2024-03-23T15:44:56.398 INFO:tasks.workunit.client.0.smithi076.stdout:6/411: write d1/d4/f35 [357213,3873] 0 2024-03-23T15:44:56.400 INFO:tasks.workunit.client.0.smithi076.stdout:3/470: dread d5/d17/d1c/f1e [0,4194304] 0 2024-03-23T15:44:56.400 INFO:tasks.workunit.client.0.smithi076.stdout:6/412: mknod d1/dd/df/c8f 0 2024-03-23T15:44:56.402 INFO:tasks.workunit.client.0.smithi076.stdout:5/650: read f18 [4319501,69996] 0 2024-03-23T15:44:56.402 INFO:tasks.workunit.client.0.smithi076.stdout:7/769: symlink d8/d4e/d98/df6/d104/d40/d88/d5a/dd9/l109 0 2024-03-23T15:44:56.402 INFO:tasks.workunit.client.0.smithi076.stdout:7/770: dread - d8/d4e/d98/df6/d104/d40/f90 zero size 2024-03-23T15:44:56.403 INFO:tasks.workunit.client.0.smithi076.stdout:7/771: stat d8/d4e/d98/df6/d104/d3a/f8b 0 2024-03-23T15:44:56.403 INFO:tasks.workunit.client.0.smithi076.stdout:5/651: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/fe1 x:0 0 0 2024-03-23T15:44:56.404 INFO:tasks.workunit.client.0.smithi076.stdout:2/647: creat d1/d4/d64/d1d/fe0 x:0 0 0 2024-03-23T15:44:56.404 INFO:tasks.workunit.client.0.smithi076.stdout:5/652: chown d1a/l5d 12762 1 2024-03-23T15:44:56.405 INFO:tasks.workunit.client.0.smithi076.stdout:3/471: mknod d5/d17/d1c/d31/d47/c9b 0 2024-03-23T15:44:56.405 INFO:tasks.workunit.client.0.smithi076.stdout:3/472: getdents d5/d35/d3c 0 2024-03-23T15:44:56.406 INFO:tasks.workunit.client.0.smithi076.stdout:3/473: symlink d5/d17/d1c/d31/l9c 0 2024-03-23T15:44:56.406 INFO:tasks.workunit.client.0.smithi076.stdout:3/474: chown d5/d35/d4c/d5a/f41 192978360 1 2024-03-23T15:44:56.406 INFO:tasks.workunit.client.0.smithi076.stdout:3/475: chown d5/d35/d4c/d5a/l4b 3033 1 2024-03-23T15:44:56.407 INFO:tasks.workunit.client.0.smithi076.stdout:3/476: rmdir d5/d17/d1c/d31/d61 39 2024-03-23T15:44:56.409 INFO:tasks.workunit.client.0.smithi076.stdout:3/477: mkdir d5/d35/d4c/d5a/d9d 0 2024-03-23T15:44:56.410 INFO:tasks.workunit.client.0.smithi076.stdout:3/478: chown d5/d35/d4c/d5a/l46 52448570 1 2024-03-23T15:44:56.411 INFO:tasks.workunit.client.0.smithi076.stdout:3/479: unlink d5/d17/d1c/d32/l5b 0 2024-03-23T15:44:56.413 INFO:tasks.workunit.client.0.smithi076.stdout:3/480: symlink d5/d35/l9e 0 2024-03-23T15:44:56.417 INFO:tasks.workunit.client.0.smithi076.stdout:3/481: creat d5/d35/d6d/f9f x:0 0 0 2024-03-23T15:44:56.419 INFO:tasks.workunit.client.0.smithi076.stdout:3/482: symlink d5/d17/d1c/d31/d61/la0 0 2024-03-23T15:44:56.420 INFO:tasks.workunit.client.0.smithi076.stdout:3/483: write d5/d17/d1c/d32/d4a/f72 [1008376,125161] 0 2024-03-23T15:44:56.420 INFO:tasks.workunit.client.0.smithi076.stdout:3/484: readlink d5/d17/d1c/d32/l2c 0 2024-03-23T15:44:56.422 INFO:tasks.workunit.client.0.smithi076.stdout:7/772: dwrite d8/de/f8a [0,4194304] 0 2024-03-23T15:44:56.425 INFO:tasks.workunit.client.0.smithi076.stdout:2/648: dwrite d1/d4/db/f3d [0,4194304] 0 2024-03-23T15:44:56.425 INFO:tasks.workunit.client.0.smithi076.stdout:5/653: dwrite f18 [8388608,4194304] 0 2024-03-23T15:44:56.426 INFO:tasks.workunit.client.0.smithi076.stdout:5/654: symlink d1a/d33/le2 0 2024-03-23T15:44:56.426 INFO:tasks.workunit.client.0.smithi076.stdout:2/649: truncate d1/d4/d64/f18 3846129 0 2024-03-23T15:44:56.426 INFO:tasks.workunit.client.0.smithi076.stdout:7/773: dread d8/d4e/d98/df6/d104/d40/d5d/f29 [0,4194304] 0 2024-03-23T15:44:56.428 INFO:tasks.workunit.client.0.smithi076.stdout:2/650: link d1/d4/f9 d1/d4/d64/d1b/fe1 0 2024-03-23T15:44:56.429 INFO:tasks.workunit.client.0.smithi076.stdout:2/651: mknod d1/d4/db/ce2 0 2024-03-23T15:44:56.431 INFO:tasks.workunit.client.0.smithi076.stdout:2/652: truncate d1/d4/db/d15/f55 6061587 0 2024-03-23T15:44:56.431 INFO:tasks.workunit.client.0.smithi076.stdout:2/653: fdatasync d1/d4/db/f3d 0 2024-03-23T15:44:56.431 INFO:tasks.workunit.client.0.smithi076.stdout:9/635: unlink d1/d1f/d82/f9e 0 2024-03-23T15:44:56.431 INFO:tasks.workunit.client.0.smithi076.stdout:9/636: chown d1/d2f/d3f/c9d 2 1 2024-03-23T15:44:56.431 INFO:tasks.workunit.client.0.smithi076.stdout:9/637: write d1/d1f/fbb [518907,72543] 0 2024-03-23T15:44:56.432 INFO:tasks.workunit.client.0.smithi076.stdout:9/638: mknod d1/d2f/d3a/d74/cca 0 2024-03-23T15:44:56.432 INFO:tasks.workunit.client.0.smithi076.stdout:9/639: fdatasync d1/f35 0 2024-03-23T15:44:56.433 INFO:tasks.workunit.client.0.smithi076.stdout:2/654: dread d1/d4/db/d15/d29/d7d/f9d [0,4194304] 0 2024-03-23T15:44:56.434 INFO:tasks.workunit.client.0.smithi076.stdout:8/531: rename d8/d26/d33/c47 to d8/d4f/d80/cac 0 2024-03-23T15:44:56.440 INFO:tasks.workunit.client.0.smithi076.stdout:2/655: truncate d1/d4/d64/d1d/fa8 121045 0 2024-03-23T15:44:56.440 INFO:tasks.workunit.client.0.smithi076.stdout:6/413: link d1/dd/df/f13 d1/d4/d10/f90 0 2024-03-23T15:44:56.441 INFO:tasks.workunit.client.0.smithi076.stdout:5/655: dwrite d1a/d33/d3a/f7a [0,4194304] 0 2024-03-23T15:44:56.444 INFO:tasks.workunit.client.0.smithi076.stdout:6/414: creat d1/dd/d2a/f91 x:0 0 0 2024-03-23T15:44:56.444 INFO:tasks.workunit.client.0.smithi076.stdout:6/415: fsync d1/dd/d2a/d23/f2d 0 2024-03-23T15:44:56.445 INFO:tasks.workunit.client.0.smithi076.stdout:2/656: mkdir d1/d4/db/d15/d29/d7d/dbd/de3 0 2024-03-23T15:44:56.451 INFO:tasks.workunit.client.0.smithi076.stdout:6/416: rmdir d1/dd/df 39 2024-03-23T15:44:56.453 INFO:tasks.workunit.client.0.smithi076.stdout:6/417: mknod d1/d4/d10/d6b/d8d/d60/d7c/c92 0 2024-03-23T15:44:56.455 INFO:tasks.workunit.client.0.smithi076.stdout:9/640: dwrite d1/d2f/f42 [0,4194304] 0 2024-03-23T15:44:56.456 INFO:tasks.workunit.client.0.smithi076.stdout:6/418: unlink d1/dd/df/c49 0 2024-03-23T15:44:56.456 INFO:tasks.workunit.client.0.smithi076.stdout:8/532: dwrite d8/d26/d33/f3d [4194304,4194304] 0 2024-03-23T15:44:56.457 INFO:tasks.workunit.client.0.smithi076.stdout:9/641: creat d1/d1f/d46/d4a/d5a/daa/fcb x:0 0 0 2024-03-23T15:44:56.458 INFO:tasks.workunit.client.0.smithi076.stdout:9/642: chown d1/d1f/d29/d2d/d37 0 1 2024-03-23T15:44:56.458 INFO:tasks.workunit.client.0.smithi076.stdout:8/533: symlink d8/d2e/d30/d64/lad 0 2024-03-23T15:44:56.459 INFO:tasks.workunit.client.0.smithi076.stdout:9/643: mknod d1/d1f/d46/d83/da7/ccc 0 2024-03-23T15:44:56.459 INFO:tasks.workunit.client.0.smithi076.stdout:9/644: truncate d1/d1f/d29/d2d/d37/d47/fbe 3547 0 2024-03-23T15:44:56.459 INFO:tasks.workunit.client.0.smithi076.stdout:9/645: write d1/d1f/d29/d2d/d37/fb0 [999798,22492] 0 2024-03-23T15:44:56.462 INFO:tasks.workunit.client.0.smithi076.stdout:6/419: mknod d1/dd/c93 0 2024-03-23T15:44:56.463 INFO:tasks.workunit.client.0.smithi076.stdout:6/420: getdents d1/d4/d10/d6b/d8d/d60 0 2024-03-23T15:44:56.463 INFO:tasks.workunit.client.0.smithi076.stdout:8/534: mkdir d8/d26/d33/dae 0 2024-03-23T15:44:56.464 INFO:tasks.workunit.client.0.smithi076.stdout:2/657: dwrite d1/f2f [0,4194304] 0 2024-03-23T15:44:56.466 INFO:tasks.workunit.client.0.smithi076.stdout:1/348: rename d0/d35/d52/cb to d0/d35/d40/d47/d17/c6d 0 2024-03-23T15:44:56.468 INFO:tasks.workunit.client.0.smithi076.stdout:0/659: rename d2/de/d10/d43/f6f to d2/d9a/fc8 0 2024-03-23T15:44:56.469 INFO:tasks.workunit.client.0.smithi076.stdout:1/349: mkdir d0/d3c/d6e 0 2024-03-23T15:44:56.470 INFO:tasks.workunit.client.0.smithi076.stdout:2/658: read d1/d4/d64/d1d/fb1 [428894,55965] 0 2024-03-23T15:44:56.472 INFO:tasks.workunit.client.0.smithi076.stdout:4/623: rename d8/d3a/d18/d1b/f7f to d8/d41/d97/fb9 0 2024-03-23T15:44:56.474 INFO:tasks.workunit.client.0.smithi076.stdout:0/660: truncate d2/de/d10/f8c 1409440 0 2024-03-23T15:44:56.478 INFO:tasks.workunit.client.0.smithi076.stdout:3/485: rename d5/f2a to d5/d17/d1c/d31/fa1 0 2024-03-23T15:44:56.478 INFO:tasks.workunit.client.0.smithi076.stdout:6/421: dwrite d1/dd/f54 [0,4194304] 0 2024-03-23T15:44:56.479 INFO:tasks.workunit.client.0.smithi076.stdout:6/422: dread - d1/dd/df/d5b/d62/f8b zero size 2024-03-23T15:44:56.485 INFO:tasks.workunit.client.0.smithi076.stdout:4/624: write d8/f42 [3423607,93378] 0 2024-03-23T15:44:56.486 INFO:tasks.workunit.client.0.smithi076.stdout:0/661: link d2/fb7 d2/d1b/fc9 0 2024-03-23T15:44:56.488 INFO:tasks.workunit.client.0.smithi076.stdout:2/659: dread d1/d4/d64/d1d/f73 [0,4194304] 0 2024-03-23T15:44:56.488 INFO:tasks.workunit.client.0.smithi076.stdout:1/350: write d0/f34 [2068279,48975] 0 2024-03-23T15:44:56.489 INFO:tasks.workunit.client.0.smithi076.stdout:6/423: dread d1/dd/d2a/d23/f2d [0,4194304] 0 2024-03-23T15:44:56.489 INFO:tasks.workunit.client.0.smithi076.stdout:2/660: dread d1/d4/d64/f95 [0,4194304] 0 2024-03-23T15:44:56.491 INFO:tasks.workunit.client.0.smithi076.stdout:0/662: creat d2/de/d10/d3a/fca x:0 0 0 2024-03-23T15:44:56.491 INFO:tasks.workunit.client.0.smithi076.stdout:0/663: truncate d2/de/f95 688555 0 2024-03-23T15:44:56.492 INFO:tasks.workunit.client.0.smithi076.stdout:2/661: unlink d1/d4/d64/f95 0 2024-03-23T15:44:56.493 INFO:tasks.workunit.client.0.smithi076.stdout:2/662: dread d1/d4/fb9 [0,4194304] 0 2024-03-23T15:44:56.493 INFO:tasks.workunit.client.0.smithi076.stdout:0/664: symlink d2/d9a/lcb 0 2024-03-23T15:44:56.495 INFO:tasks.workunit.client.0.smithi076.stdout:4/625: unlink d8/ca8 0 2024-03-23T15:44:56.495 INFO:tasks.workunit.client.0.smithi076.stdout:4/626: fdatasync d8/d9/f5a 0 2024-03-23T15:44:56.496 INFO:tasks.workunit.client.0.smithi076.stdout:6/424: rmdir d1/dd/d2a 39 2024-03-23T15:44:56.496 INFO:tasks.workunit.client.0.smithi076.stdout:6/425: write d1/d2/f87 [715499,27742] 0 2024-03-23T15:44:56.496 INFO:tasks.workunit.client.0.smithi076.stdout:6/426: write d1/d2/f9 [1940047,60652] 0 2024-03-23T15:44:56.497 INFO:tasks.workunit.client.0.smithi076.stdout:6/427: readlink d1/dd/df/d5b/l1d 0 2024-03-23T15:44:56.498 INFO:tasks.workunit.client.0.smithi076.stdout:4/627: symlink d8/d9/lba 0 2024-03-23T15:44:56.499 INFO:tasks.workunit.client.0.smithi076.stdout:6/428: truncate d1/d4/f35 138735 0 2024-03-23T15:44:56.500 INFO:tasks.workunit.client.0.smithi076.stdout:4/628: creat d8/d3a/d18/d75/fbb x:0 0 0 2024-03-23T15:44:56.500 INFO:tasks.workunit.client.0.smithi076.stdout:4/629: readlink l7 0 2024-03-23T15:44:56.502 INFO:tasks.workunit.client.0.smithi076.stdout:2/663: dread d1/d4/db/f3d [0,4194304] 0 2024-03-23T15:44:56.502 INFO:tasks.workunit.client.0.smithi076.stdout:6/429: mknod d1/dd/d2a/d23/c94 0 2024-03-23T15:44:56.502 INFO:tasks.workunit.client.0.smithi076.stdout:1/351: dwrite d0/d35/d40/d47/d17/f38 [0,4194304] 0 2024-03-23T15:44:56.503 INFO:tasks.workunit.client.0.smithi076.stdout:3/486: dwrite d5/d17/d1c/d31/fa1 [0,4194304] 0 2024-03-23T15:44:56.504 INFO:tasks.workunit.client.0.smithi076.stdout:6/430: creat d1/d81/f95 x:0 0 0 2024-03-23T15:44:56.506 INFO:tasks.workunit.client.0.smithi076.stdout:0/665: dread d2/de/d10/d43/d56/f60 [0,4194304] 0 2024-03-23T15:44:56.506 INFO:tasks.workunit.client.0.smithi076.stdout:1/352: creat d0/d35/f6f x:0 0 0 2024-03-23T15:44:56.510 INFO:tasks.workunit.client.0.smithi076.stdout:3/487: mknod d5/d17/d1c/d31/d47/ca2 0 2024-03-23T15:44:56.511 INFO:tasks.workunit.client.0.smithi076.stdout:1/353: mkdir d0/d2/d70 0 2024-03-23T15:44:56.511 INFO:tasks.workunit.client.0.smithi076.stdout:1/354: fsync d0/d2/d1f/f43 0 2024-03-23T15:44:56.514 INFO:tasks.workunit.client.0.smithi076.stdout:3/488: symlink d5/d35/la3 0 2024-03-23T15:44:56.518 INFO:tasks.workunit.client.0.smithi076.stdout:0/666: dread d2/d8/d2d/f3f [0,4194304] 0 2024-03-23T15:44:56.518 INFO:tasks.workunit.client.0.smithi076.stdout:0/667: chown d2/de/d10/d3a/fca 3204070 1 2024-03-23T15:44:56.518 INFO:tasks.workunit.client.0.smithi076.stdout:1/355: dread d0/d35/d52/f5b [4194304,4194304] 0 2024-03-23T15:44:56.522 INFO:tasks.workunit.client.0.smithi076.stdout:2/664: dwrite d1/d4/d64/d1d/fca [0,4194304] 0 2024-03-23T15:44:56.523 INFO:tasks.workunit.client.0.smithi076.stdout:3/489: rmdir d5/d17 39 2024-03-23T15:44:56.523 INFO:tasks.workunit.client.0.smithi076.stdout:3/490: chown d5/d35/d4c 26 1 2024-03-23T15:44:56.525 INFO:tasks.workunit.client.0.smithi076.stdout:0/668: unlink d2/d8/c17 0 2024-03-23T15:44:56.525 INFO:tasks.workunit.client.0.smithi076.stdout:6/431: dwrite d1/f85 [0,4194304] 0 2024-03-23T15:44:56.525 INFO:tasks.workunit.client.0.smithi076.stdout:2/665: symlink d1/d4/d52/le4 0 2024-03-23T15:44:56.528 INFO:tasks.workunit.client.0.smithi076.stdout:6/432: getdents d1/d4/d10/d6b/d72 0 2024-03-23T15:44:56.528 INFO:tasks.workunit.client.0.smithi076.stdout:0/669: mknod d2/de/d10/d43/d56/ccc 0 2024-03-23T15:44:56.532 INFO:tasks.workunit.client.0.smithi076.stdout:2/666: creat d1/d4/d64/d1d/fe5 x:0 0 0 2024-03-23T15:44:56.532 INFO:tasks.workunit.client.0.smithi076.stdout:2/667: stat d1/d4/db/d15/d29/f9e 0 2024-03-23T15:44:56.532 INFO:tasks.workunit.client.0.smithi076.stdout:2/668: fdatasync d1/d4/f98 0 2024-03-23T15:44:56.533 INFO:tasks.workunit.client.0.smithi076.stdout:0/670: mknod d2/de/d10/d43/d56/ccd 0 2024-03-23T15:44:56.533 INFO:tasks.workunit.client.0.smithi076.stdout:0/671: write d2/de/d10/d43/fb9 [701104,97904] 0 2024-03-23T15:44:56.535 INFO:tasks.workunit.client.0.smithi076.stdout:0/672: creat d2/de/d10/d3a/fce x:0 0 0 2024-03-23T15:44:56.539 INFO:tasks.workunit.client.0.smithi076.stdout:1/356: dwrite d0/d3c/f57 [0,4194304] 0 2024-03-23T15:44:56.539 INFO:tasks.workunit.client.0.smithi076.stdout:1/357: dread - d0/d35/d40/d47/d17/d20/f58 zero size 2024-03-23T15:44:56.539 INFO:tasks.workunit.client.0.smithi076.stdout:0/673: symlink d2/de/d10/d43/d56/lcf 0 2024-03-23T15:44:56.543 INFO:tasks.workunit.client.0.smithi076.stdout:7/774: rename d8/d4e/d98/df6/d104/d40/d88/d5a/dd9/l109 to d8/l10a 0 2024-03-23T15:44:56.543 INFO:tasks.workunit.client.0.smithi076.stdout:7/775: chown d8/d4e/d98/df6/d104/d3a/d85/lc5 423 1 2024-03-23T15:44:56.543 INFO:tasks.workunit.client.0.smithi076.stdout:7/776: fsync d8/d4e/d98/df6/d104/d40/f90 0 2024-03-23T15:44:56.543 INFO:tasks.workunit.client.0.smithi076.stdout:7/777: stat d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/db9 0 2024-03-23T15:44:56.545 INFO:tasks.workunit.client.0.smithi076.stdout:3/491: dwrite d5/d35/f36 [4194304,4194304] 0 2024-03-23T15:44:56.546 INFO:tasks.workunit.client.0.smithi076.stdout:3/492: getdents d5/d35/d3c/d67 0 2024-03-23T15:44:56.546 INFO:tasks.workunit.client.0.smithi076.stdout:7/778: creat d8/d4e/d98/f10b x:0 0 0 2024-03-23T15:44:56.546 INFO:tasks.workunit.client.0.smithi076.stdout:7/779: write d8/de/d48/fc3 [1083349,21276] 0 2024-03-23T15:44:56.549 INFO:tasks.workunit.client.0.smithi076.stdout:3/493: rmdir d5/d17 39 2024-03-23T15:44:56.549 INFO:tasks.workunit.client.0.smithi076.stdout:3/494: dread - d5/d35/d3c/f94 zero size 2024-03-23T15:44:56.552 INFO:tasks.workunit.client.0.smithi076.stdout:6/433: dwrite d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:56.552 INFO:tasks.workunit.client.0.smithi076.stdout:2/669: dread d1/f2f [0,4194304] 0 2024-03-23T15:44:56.555 INFO:tasks.workunit.client.0.smithi076.stdout:2/670: mkdir d1/d4/d64/d1d/dbc/de6 0 2024-03-23T15:44:56.557 INFO:tasks.workunit.client.0.smithi076.stdout:2/671: symlink d1/d4/d64/d82/d7f/le7 0 2024-03-23T15:44:56.557 INFO:tasks.workunit.client.0.smithi076.stdout:2/672: read d1/f5 [3975277,11759] 0 2024-03-23T15:44:56.558 INFO:tasks.workunit.client.0.smithi076.stdout:2/673: creat d1/d4/d52/fe8 x:0 0 0 2024-03-23T15:44:56.567 INFO:tasks.workunit.client.0.smithi076.stdout:1/358: dwrite d0/d35/f64 [0,4194304] 0 2024-03-23T15:44:56.568 INFO:tasks.workunit.client.0.smithi076.stdout:0/674: dwrite d2/d8/d2d/f45 [0,4194304] 0 2024-03-23T15:44:56.571 INFO:tasks.workunit.client.0.smithi076.stdout:7/780: dwrite d8/d4e/d98/df6/d104/d40/d88/fb2 [0,4194304] 0 2024-03-23T15:44:56.573 INFO:tasks.workunit.client.0.smithi076.stdout:0/675: creat d2/d9a/fd0 x:0 0 0 2024-03-23T15:44:56.573 INFO:tasks.workunit.client.0.smithi076.stdout:7/781: truncate d8/de/f8a 2938758 0 2024-03-23T15:44:56.576 INFO:tasks.workunit.client.0.smithi076.stdout:3/495: dwrite d5/d17/d1c/d32/d39/d59/d79/f8d [0,4194304] 0 2024-03-23T15:44:56.576 INFO:tasks.workunit.client.0.smithi076.stdout:7/782: mknod d8/d4e/d98/df6/d104/c10c 0 2024-03-23T15:44:56.582 INFO:tasks.workunit.client.0.smithi076.stdout:6/434: dwrite d1/d4/d10/f90 [4194304,4194304] 0 2024-03-23T15:44:56.582 INFO:tasks.workunit.client.0.smithi076.stdout:0/676: creat d2/de/d10/d3a/d47/d69/d84/dc5/fd1 x:0 0 0 2024-03-23T15:44:56.582 INFO:tasks.workunit.client.0.smithi076.stdout:6/435: dread d1/d2/f87 [0,4194304] 0 2024-03-23T15:44:56.583 INFO:tasks.workunit.client.0.smithi076.stdout:7/783: dread d8/d4e/d98/df6/d104/f2f [0,4194304] 0 2024-03-23T15:44:56.583 INFO:tasks.workunit.client.0.smithi076.stdout:7/784: write d8/d4e/d98/df6/d104/d18/fc9 [857735,123342] 0 2024-03-23T15:44:56.584 INFO:tasks.workunit.client.0.smithi076.stdout:7/785: read d8/de/d48/f84 [5462,45559] 0 2024-03-23T15:44:56.584 INFO:tasks.workunit.client.0.smithi076.stdout:7/786: fsync d8/d4e/d98/df6/d104/d40/d5d/f3e 0 2024-03-23T15:44:56.585 INFO:tasks.workunit.client.0.smithi076.stdout:1/359: dwrite d0/d35/d40/d47/d17/d20/f33 [0,4194304] 0 2024-03-23T15:44:56.585 INFO:tasks.workunit.client.0.smithi076.stdout:1/360: write d0/d3c/f6c [480785,22219] 0 2024-03-23T15:44:56.585 INFO:tasks.workunit.client.0.smithi076.stdout:3/496: unlink d5/d17/d1c/f7b 0 2024-03-23T15:44:56.588 INFO:tasks.workunit.client.0.smithi076.stdout:3/497: write d5/d17/d1c/d32/d39/f6c [641344,122221] 0 2024-03-23T15:44:56.588 INFO:tasks.workunit.client.0.smithi076.stdout:3/498: write d5/d17/d1c/d32/d39/f65 [204638,21346] 0 2024-03-23T15:44:56.588 INFO:tasks.workunit.client.0.smithi076.stdout:3/499: chown d5/d35/f6e 2 1 2024-03-23T15:44:56.590 INFO:tasks.workunit.client.0.smithi076.stdout:0/677: creat d2/de/d10/d12/da6/fd2 x:0 0 0 2024-03-23T15:44:56.590 INFO:tasks.workunit.client.0.smithi076.stdout:0/678: readlink d2/lc 0 2024-03-23T15:44:56.590 INFO:tasks.workunit.client.0.smithi076.stdout:7/787: rmdir d8/d4e/d98/df6 39 2024-03-23T15:44:56.591 INFO:tasks.workunit.client.0.smithi076.stdout:0/679: mknod d2/de/d10/d12/da6/cd3 0 2024-03-23T15:44:56.592 INFO:tasks.workunit.client.0.smithi076.stdout:7/788: mkdir d8/d4e/d98/df6/d10d 0 2024-03-23T15:44:56.592 INFO:tasks.workunit.client.0.smithi076.stdout:7/789: truncate d8/f62 4453862 0 2024-03-23T15:44:56.594 INFO:tasks.workunit.client.0.smithi076.stdout:7/790: dread d8/de/fad [0,4194304] 0 2024-03-23T15:44:56.595 INFO:tasks.workunit.client.0.smithi076.stdout:7/791: dread d8/d4e/d98/df6/d104/d40/d5d/d3c/d8e/fb7 [0,4194304] 0 2024-03-23T15:44:56.597 INFO:tasks.workunit.client.0.smithi076.stdout:7/792: creat d8/d4e/d98/df6/d104/d24/d91/d96/f10e x:0 0 0 2024-03-23T15:44:56.597 INFO:tasks.workunit.client.0.smithi076.stdout:7/793: write d8/d4e/d98/f9f [642688,73906] 0 2024-03-23T15:44:56.598 INFO:tasks.workunit.client.0.smithi076.stdout:6/436: dwrite d1/d2/f38 [0,4194304] 0 2024-03-23T15:44:56.598 INFO:tasks.workunit.client.0.smithi076.stdout:7/794: mkdir d8/d4e/d98/df6/d104/d40/d5d/d10f 0 2024-03-23T15:44:56.600 INFO:tasks.workunit.client.0.smithi076.stdout:6/437: creat d1/dd/d2a/d23/f96 x:0 0 0 2024-03-23T15:44:56.600 INFO:tasks.workunit.client.0.smithi076.stdout:6/438: fsync d1/f2b 0 2024-03-23T15:44:56.600 INFO:tasks.workunit.client.0.smithi076.stdout:6/439: truncate d1/d4/d10/f4a 4754316 0 2024-03-23T15:44:56.603 INFO:tasks.workunit.client.0.smithi076.stdout:6/440: symlink d1/d4/d10/d6b/d8d/l97 0 2024-03-23T15:44:56.604 INFO:tasks.workunit.client.0.smithi076.stdout:6/441: symlink d1/d4/d10/l98 0 2024-03-23T15:44:56.608 INFO:tasks.workunit.client.0.smithi076.stdout:6/442: rmdir d1 39 2024-03-23T15:44:56.608 INFO:tasks.workunit.client.0.smithi076.stdout:1/361: dwrite d0/d35/d40/d47/f6b [0,4194304] 0 2024-03-23T15:44:56.608 INFO:tasks.workunit.client.0.smithi076.stdout:1/362: write d0/d35/f60 [36293,19712] 0 2024-03-23T15:44:56.609 INFO:tasks.workunit.client.0.smithi076.stdout:1/363: mknod d0/d35/d40/d47/d55/c71 0 2024-03-23T15:44:56.609 INFO:tasks.workunit.client.0.smithi076.stdout:6/443: creat d1/dd/df/d5b/f99 x:0 0 0 2024-03-23T15:44:56.610 INFO:tasks.workunit.client.0.smithi076.stdout:0/680: dwrite d2/de/d10/f8c [0,4194304] 0 2024-03-23T15:44:56.611 INFO:tasks.workunit.client.0.smithi076.stdout:1/364: symlink d0/d2/d1f/l72 0 2024-03-23T15:44:56.618 INFO:tasks.workunit.client.0.smithi076.stdout:5/656: sync 2024-03-23T15:44:56.619 INFO:tasks.workunit.client.0.smithi076.stdout:9/646: sync 2024-03-23T15:44:56.619 INFO:tasks.workunit.client.0.smithi076.stdout:8/535: rename d8/d4f/c65 to d8/d2e/d30/d69/caf 0 2024-03-23T15:44:56.619 INFO:tasks.workunit.client.0.smithi076.stdout:5/657: write d1a/d33/d3a/f41 [5239554,67212] 0 2024-03-23T15:44:56.619 INFO:tasks.workunit.client.0.smithi076.stdout:7/795: dwrite d8/fc [0,4194304] 0 2024-03-23T15:44:56.621 INFO:tasks.workunit.client.0.smithi076.stdout:9/647: symlink d1/d1f/d46/d4a/d5a/lcd 0 2024-03-23T15:44:56.621 INFO:tasks.workunit.client.0.smithi076.stdout:9/648: chown d1/f12 580022654 1 2024-03-23T15:44:56.621 INFO:tasks.workunit.client.0.smithi076.stdout:5/658: symlink d1a/d33/le3 0 2024-03-23T15:44:56.622 INFO:tasks.workunit.client.0.smithi076.stdout:4/630: rename d8/d3a/d18/d52/f61 to d8/d80/fbc 0 2024-03-23T15:44:56.622 INFO:tasks.workunit.client.0.smithi076.stdout:9/649: mknod d1/d1f/d29/d2d/d37/cce 0 2024-03-23T15:44:56.623 INFO:tasks.workunit.client.0.smithi076.stdout:9/650: chown d1/d1f/d29/d2d/d37/d47/f52 652 1 2024-03-23T15:44:56.623 INFO:tasks.workunit.client.0.smithi076.stdout:9/651: write d1/d1f/f22 [420898,69692] 0 2024-03-23T15:44:56.623 INFO:tasks.workunit.client.0.smithi076.stdout:8/536: dread d8/fa [0,4194304] 0 2024-03-23T15:44:56.623 INFO:tasks.workunit.client.0.smithi076.stdout:0/681: dwrite d2/de/d10/d43/f88 [0,4194304] 0 2024-03-23T15:44:56.624 INFO:tasks.workunit.client.0.smithi076.stdout:0/682: readlink d2/d1b/d77/la4 0 2024-03-23T15:44:56.624 INFO:tasks.workunit.client.0.smithi076.stdout:0/683: readlink d2/lc 0 2024-03-23T15:44:56.624 INFO:tasks.workunit.client.0.smithi076.stdout:7/796: dread d8/d4e/fe9 [0,4194304] 0 2024-03-23T15:44:56.626 INFO:tasks.workunit.client.0.smithi076.stdout:4/631: creat d8/d3a/d18/d75/d8d/fbd x:0 0 0 2024-03-23T15:44:56.627 INFO:tasks.workunit.client.0.smithi076.stdout:1/365: dwrite d0/f26 [0,4194304] 0 2024-03-23T15:44:56.627 INFO:tasks.workunit.client.0.smithi076.stdout:9/652: mknod d1/d1f/d46/ccf 0 2024-03-23T15:44:56.628 INFO:tasks.workunit.client.0.smithi076.stdout:5/659: symlink d1a/le4 0 2024-03-23T15:44:56.628 INFO:tasks.workunit.client.0.smithi076.stdout:8/537: symlink d8/d2e/d3e/lb0 0 2024-03-23T15:44:56.629 INFO:tasks.workunit.client.0.smithi076.stdout:7/797: dread d8/d4e/d98/f9f [0,4194304] 0 2024-03-23T15:44:56.629 INFO:tasks.workunit.client.0.smithi076.stdout:7/798: stat d8/d4e/d98/df6/d104/d24/de4/la2 0 2024-03-23T15:44:56.630 INFO:tasks.workunit.client.0.smithi076.stdout:4/632: symlink d8/d3a/d69/d73/d76/db4/lbe 0 2024-03-23T15:44:56.632 INFO:tasks.workunit.client.0.smithi076.stdout:8/538: mknod d8/d6b/d9a/cb1 0 2024-03-23T15:44:56.633 INFO:tasks.workunit.client.0.smithi076.stdout:8/539: write d8/d2e/d30/d69/f8f [1212984,3528] 0 2024-03-23T15:44:56.635 INFO:tasks.workunit.client.0.smithi076.stdout:0/684: dread d2/d1b/fc4 [0,4194304] 0 2024-03-23T15:44:56.636 INFO:tasks.workunit.client.0.smithi076.stdout:4/633: creat d8/d3a/d69/fbf x:0 0 0 2024-03-23T15:44:56.636 INFO:tasks.workunit.client.0.smithi076.stdout:4/634: write d8/d3a/d18/d75/faa [266747,126311] 0 2024-03-23T15:44:56.637 INFO:tasks.workunit.client.0.smithi076.stdout:8/540: mkdir d8/d89/db2 0 2024-03-23T15:44:56.638 INFO:tasks.workunit.client.0.smithi076.stdout:8/541: stat d8/c9 0 2024-03-23T15:44:56.639 INFO:tasks.workunit.client.0.smithi076.stdout:0/685: dread d2/d1b/f42 [0,4194304] 0 2024-03-23T15:44:56.639 INFO:tasks.workunit.client.0.smithi076.stdout:8/542: symlink d8/d6b/d8e/lb3 0 2024-03-23T15:44:56.641 INFO:tasks.workunit.client.0.smithi076.stdout:0/686: creat d2/de/d71/fd4 x:0 0 0 2024-03-23T15:44:56.642 INFO:tasks.workunit.client.0.smithi076.stdout:8/543: truncate d8/f18 6597621 0 2024-03-23T15:44:56.647 INFO:tasks.workunit.client.0.smithi076.stdout:4/635: read d8/f42 [8176290,128865] 0 2024-03-23T15:44:56.647 INFO:tasks.workunit.client.0.smithi076.stdout:4/636: read - d8/d9/d32/f90 zero size 2024-03-23T15:44:56.647 INFO:tasks.workunit.client.0.smithi076.stdout:1/366: dwrite d0/d35/d40/d47/d17/f30 [0,4194304] 0 2024-03-23T15:44:56.648 INFO:tasks.workunit.client.0.smithi076.stdout:5/660: dwrite d1a/d4b/f78 [0,4194304] 0 2024-03-23T15:44:56.649 INFO:tasks.workunit.client.0.smithi076.stdout:4/637: dread d8/d9/f5a [0,4194304] 0 2024-03-23T15:44:56.650 INFO:tasks.workunit.client.0.smithi076.stdout:5/661: getdents d1a/d33/d36 0 2024-03-23T15:44:56.651 INFO:tasks.workunit.client.0.smithi076.stdout:5/662: write d1a/d33/f3b [1232647,124168] 0 2024-03-23T15:44:56.652 INFO:tasks.workunit.client.0.smithi076.stdout:5/663: symlink d1a/d20/le5 0 2024-03-23T15:44:56.652 INFO:tasks.workunit.client.0.smithi076.stdout:1/367: write d0/d35/d40/d47/d17/f30 [4660766,75060] 0 2024-03-23T15:44:56.657 INFO:tasks.workunit.client.0.smithi076.stdout:8/544: write d8/f18 [2851468,15501] 0 2024-03-23T15:44:56.657 INFO:tasks.workunit.client.0.smithi076.stdout:4/638: getdents d8/d3a 0 2024-03-23T15:44:56.657 INFO:tasks.workunit.client.0.smithi076.stdout:8/545: write d8/d6b/f7e [1372968,10104] 0 2024-03-23T15:44:56.659 INFO:tasks.workunit.client.0.smithi076.stdout:1/368: symlink d0/d35/d40/l73 0 2024-03-23T15:44:56.659 INFO:tasks.workunit.client.0.smithi076.stdout:3/500: rename d5/d17/c19 to d5/d17/d1c/d32/d39/d59/d79/ca4 0 2024-03-23T15:44:56.660 INFO:tasks.workunit.client.0.smithi076.stdout:5/664: dread d1a/d33/d36/fa2 [0,4194304] 0 2024-03-23T15:44:56.660 INFO:tasks.workunit.client.0.smithi076.stdout:5/665: read - d1a/d2c/fbf zero size 2024-03-23T15:44:56.660 INFO:tasks.workunit.client.0.smithi076.stdout:8/546: truncate d8/d26/f4d 2347099 0 2024-03-23T15:44:56.660 INFO:tasks.workunit.client.0.smithi076.stdout:8/547: chown d8/f2a 111 1 2024-03-23T15:44:56.660 INFO:tasks.workunit.client.0.smithi076.stdout:8/548: readlink d8/d26/d27/l8a 0 2024-03-23T15:44:56.661 INFO:tasks.workunit.client.0.smithi076.stdout:2/674: sync 2024-03-23T15:44:56.662 INFO:tasks.workunit.client.0.smithi076.stdout:8/549: creat d8/d4f/d80/da3/fb4 x:0 0 0 2024-03-23T15:44:56.663 INFO:tasks.workunit.client.0.smithi076.stdout:8/550: creat d8/d26/d33/d8c/fb5 x:0 0 0 2024-03-23T15:44:56.664 INFO:tasks.workunit.client.0.smithi076.stdout:2/675: unlink d1/d4/f72 0 2024-03-23T15:44:56.665 INFO:tasks.workunit.client.0.smithi076.stdout:3/501: mkdir d5/d17/d1c/d32/d39/da5 0 2024-03-23T15:44:56.666 INFO:tasks.workunit.client.0.smithi076.stdout:2/676: getdents d1/d4/d64 0 2024-03-23T15:44:56.666 INFO:tasks.workunit.client.0.smithi076.stdout:1/369: dread d0/d35/d52/f61 [4194304,4194304] 0 2024-03-23T15:44:56.667 INFO:tasks.workunit.client.0.smithi076.stdout:3/502: rename d5/d35/d4c/f51 to d5/d35/d4c/fa6 0 2024-03-23T15:44:56.667 INFO:tasks.workunit.client.0.smithi076.stdout:3/503: truncate d5/d35/f6e 36237 0 2024-03-23T15:44:56.668 INFO:tasks.workunit.client.0.smithi076.stdout:2/677: unlink d1/d2e/f3f 0 2024-03-23T15:44:56.670 INFO:tasks.workunit.client.0.smithi076.stdout:3/504: rmdir d5/d17/d1c/d32 39 2024-03-23T15:44:56.672 INFO:tasks.workunit.client.0.smithi076.stdout:8/551: dread d8/f19 [4194304,4194304] 0 2024-03-23T15:44:56.673 INFO:tasks.workunit.client.0.smithi076.stdout:3/505: read d5/d17/d1c/d32/d39/d59/f88 [3632042,110161] 0 2024-03-23T15:44:56.673 INFO:tasks.workunit.client.0.smithi076.stdout:2/678: link d1/d2e/d8a/cab d1/ce9 0 2024-03-23T15:44:56.675 INFO:tasks.workunit.client.0.smithi076.stdout:1/370: mkdir d0/d35/d40/d47/d74 0 2024-03-23T15:44:56.675 INFO:tasks.workunit.client.0.smithi076.stdout:3/506: creat d5/d35/d6d/fa7 x:0 0 0 2024-03-23T15:44:56.675 INFO:tasks.workunit.client.0.smithi076.stdout:4/639: dwrite d8/d3a/d69/fbf [0,4194304] 0 2024-03-23T15:44:56.676 INFO:tasks.workunit.client.0.smithi076.stdout:2/679: mkdir d1/d4/d64/d1b/d32/d79/dac/dd8/dea 0 2024-03-23T15:44:56.678 INFO:tasks.workunit.client.0.smithi076.stdout:1/371: creat d0/d2/f75 x:0 0 0 2024-03-23T15:44:56.680 INFO:tasks.workunit.client.0.smithi076.stdout:3/507: symlink d5/d17/d1c/d32/d8e/la8 0 2024-03-23T15:44:56.680 INFO:tasks.workunit.client.0.smithi076.stdout:3/508: dread - d5/d17/d1c/d31/d47/f7d zero size 2024-03-23T15:44:56.680 INFO:tasks.workunit.client.0.smithi076.stdout:3/509: chown d5/d35/d4c/d5a 2 1 2024-03-23T15:44:56.681 INFO:tasks.workunit.client.0.smithi076.stdout:8/552: write d8/d2e/d3e/f9b [2167491,10017] 0 2024-03-23T15:44:56.682 INFO:tasks.workunit.client.0.smithi076.stdout:1/372: creat d0/d3c/d6e/f76 x:0 0 0 2024-03-23T15:44:56.682 INFO:tasks.workunit.client.0.smithi076.stdout:3/510: creat d5/d17/d1c/d32/d4a/fa9 x:0 0 0 2024-03-23T15:44:56.683 INFO:tasks.workunit.client.0.smithi076.stdout:1/373: mknod d0/d35/d40/d47/d17/d20/c77 0 2024-03-23T15:44:56.687 INFO:tasks.workunit.client.0.smithi076.stdout:8/553: rmdir d8/d2e/d30/d64 39 2024-03-23T15:44:56.690 INFO:tasks.workunit.client.0.smithi076.stdout:8/554: symlink d8/lb6 0 2024-03-23T15:44:56.693 INFO:tasks.workunit.client.0.smithi076.stdout:8/555: mkdir d8/d4f/d80/db7 0 2024-03-23T15:44:56.695 INFO:tasks.workunit.client.0.smithi076.stdout:8/556: creat d8/d2e/d30/d69/fb8 x:0 0 0 2024-03-23T15:44:56.695 INFO:tasks.workunit.client.0.smithi076.stdout:8/557: chown d8/f16 5801548 1 2024-03-23T15:44:56.696 INFO:tasks.workunit.client.0.smithi076.stdout:6/444: rename d1/dd/df/d5b/f3e to d1/d81/f9a 0 2024-03-23T15:44:56.696 INFO:tasks.workunit.client.0.smithi076.stdout:9/653: rename d1/d1f/d29/d7f to d1/d1f/d29/d7f/dd0 22 2024-03-23T15:44:56.696 INFO:tasks.workunit.client.0.smithi076.stdout:6/445: readlink d1/dd/df/d5b/l40 0 2024-03-23T15:44:56.696 INFO:tasks.workunit.client.0.smithi076.stdout:3/511: dread d5/d17/d1c/d32/f97 [0,4194304] 0 2024-03-23T15:44:56.697 INFO:tasks.workunit.client.0.smithi076.stdout:1/374: write d0/d35/d40/d47/f6b [4020714,80899] 0 2024-03-23T15:44:56.698 INFO:tasks.workunit.client.0.smithi076.stdout:9/654: creat d1/d1f/fd1 x:0 0 0 2024-03-23T15:44:56.698 INFO:tasks.workunit.client.0.smithi076.stdout:9/655: write d1/d1f/d29/d2d/f33 [428196,81346] 0 2024-03-23T15:44:56.698 INFO:tasks.workunit.client.0.smithi076.stdout:7/799: rename d8/d4e/d98/df6/d104/d40/d5d/f29 to d8/d4e/d98/df6/f110 0 2024-03-23T15:44:56.698 INFO:tasks.workunit.client.0.smithi076.stdout:7/800: stat d8/d4e/c53 0 2024-03-23T15:44:56.698 INFO:tasks.workunit.client.0.smithi076.stdout:6/446: dread d1/f2b [0,4194304] 0 2024-03-23T15:44:56.698 INFO:tasks.workunit.client.0.smithi076.stdout:2/680: dwrite d1/d2e/d8a/fd6 [0,4194304] 0 2024-03-23T15:44:56.699 INFO:tasks.workunit.client.0.smithi076.stdout:3/512: creat d5/d17/d1c/d31/d61/faa x:0 0 0 2024-03-23T15:44:56.699 INFO:tasks.workunit.client.0.smithi076.stdout:6/447: mkdir d1/dd/df/d9b 0 2024-03-23T15:44:56.700 INFO:tasks.workunit.client.0.smithi076.stdout:1/375: symlink d0/d2/l78 0 2024-03-23T15:44:56.700 INFO:tasks.workunit.client.0.smithi076.stdout:1/376: fsync d0/d35/d40/d47/d17/d20/f33 0 2024-03-23T15:44:56.701 INFO:tasks.workunit.client.0.smithi076.stdout:1/377: fsync d0/d35/d40/d47/d17/f30 0 2024-03-23T15:44:56.701 INFO:tasks.workunit.client.0.smithi076.stdout:1/378: chown d0/d3c/d6e/f76 35 1 2024-03-23T15:44:56.701 INFO:tasks.workunit.client.0.smithi076.stdout:6/448: link d1/dd/df/c8f d1/dd/d2a/d23/c9c 0 2024-03-23T15:44:56.701 INFO:tasks.workunit.client.0.smithi076.stdout:8/558: mkdir d8/d2e/d30/d69/db9 0 2024-03-23T15:44:56.701 INFO:tasks.workunit.client.0.smithi076.stdout:0/687: rename d2/fad to d2/de/d10/fd5 0 2024-03-23T15:44:56.702 INFO:tasks.workunit.client.0.smithi076.stdout:2/681: creat d1/d4/d64/d7c/feb x:0 0 0 2024-03-23T15:44:56.702 INFO:tasks.workunit.client.0.smithi076.stdout:2/682: chown d1/l4a 3057 1 2024-03-23T15:44:56.703 INFO:tasks.workunit.client.0.smithi076.stdout:2/683: write d1/d4/d64/d7c/feb [712583,49145] 0 2024-03-23T15:44:56.703 INFO:tasks.workunit.client.0.smithi076.stdout:2/684: dread - d1/d4/d64/d1d/dbc/fdf zero size 2024-03-23T15:44:56.703 INFO:tasks.workunit.client.0.smithi076.stdout:5/666: rename d1a/d33/d3a/d67/da8/dc4/f5b to d1a/d33/fe6 0 2024-03-23T15:44:56.703 INFO:tasks.workunit.client.0.smithi076.stdout:5/667: fsync d1a/d33/dc3/da6/fbc 0 2024-03-23T15:44:56.703 INFO:tasks.workunit.client.0.smithi076.stdout:5/668: readlink d1a/d20/lb0 0 2024-03-23T15:44:56.704 INFO:tasks.workunit.client.0.smithi076.stdout:3/513: creat d5/d17/d1c/d32/d70/d8c/fab x:0 0 0 2024-03-23T15:44:56.704 INFO:tasks.workunit.client.0.smithi076.stdout:3/514: readlink d5/d17/d1c/d31/d61/la0 0 2024-03-23T15:44:56.704 INFO:tasks.workunit.client.0.smithi076.stdout:2/685: mkdir d1/d4/db/d15/d29/d7d/dbd/dec 0 2024-03-23T15:44:56.704 INFO:tasks.workunit.client.0.smithi076.stdout:2/686: readlink d1/d4/db/d15/d29/l87 0 2024-03-23T15:44:56.705 INFO:tasks.workunit.client.0.smithi076.stdout:8/559: creat d8/d26/d27/d56/fba x:0 0 0 2024-03-23T15:44:56.705 INFO:tasks.workunit.client.0.smithi076.stdout:8/560: fsync d8/d26/d33/f3d 0 2024-03-23T15:44:56.705 INFO:tasks.workunit.client.0.smithi076.stdout:5/669: creat d1a/d7e/fe7 x:0 0 0 2024-03-23T15:44:56.706 INFO:tasks.workunit.client.0.smithi076.stdout:2/687: unlink d1/d4/d64/d1b/ca7 0 2024-03-23T15:44:56.706 INFO:tasks.workunit.client.0.smithi076.stdout:2/688: write d1/d4/db/f68 [461168,16843] 0 2024-03-23T15:44:56.708 INFO:tasks.workunit.client.0.smithi076.stdout:8/561: symlink d8/d2e/d30/d69/db9/lbb 0 2024-03-23T15:44:56.710 INFO:tasks.workunit.client.0.smithi076.stdout:8/562: mknod d8/d26/d33/dae/cbc 0 2024-03-23T15:44:56.710 INFO:tasks.workunit.client.0.smithi076.stdout:5/670: dread d1a/f2a [0,4194304] 0 2024-03-23T15:44:56.710 INFO:tasks.workunit.client.0.smithi076.stdout:5/671: truncate d1a/d4b/fb8 776028 0 2024-03-23T15:44:56.710 INFO:tasks.workunit.client.0.smithi076.stdout:5/672: chown d1a/d33/d3a/d67/da8/dc4/d65 3 1 2024-03-23T15:44:56.710 INFO:tasks.workunit.client.0.smithi076.stdout:6/449: unlink d1/d2/f9 0 2024-03-23T15:44:56.710 INFO:tasks.workunit.client.0.smithi076.stdout:6/450: dread - d1/dd/d2a/d23/f96 zero size 2024-03-23T15:44:56.711 INFO:tasks.workunit.client.0.smithi076.stdout:0/688: write d2/d8/f65 [271868,3177] 0 2024-03-23T15:44:56.712 INFO:tasks.workunit.client.0.smithi076.stdout:2/689: rmdir d1/d4/d64/d1b/d32/d79/dac/dd8 39 2024-03-23T15:44:56.712 INFO:tasks.workunit.client.0.smithi076.stdout:5/673: creat d1a/d33/d3a/d80/fe8 x:0 0 0 2024-03-23T15:44:56.713 INFO:tasks.workunit.client.0.smithi076.stdout:0/689: link d2/de/d10/d3a/fce d2/d1b/d77/fd6 0 2024-03-23T15:44:56.713 INFO:tasks.workunit.client.0.smithi076.stdout:0/690: stat d2/d1b/l8b 0 2024-03-23T15:44:56.713 INFO:tasks.workunit.client.0.smithi076.stdout:5/674: getdents d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96 0 2024-03-23T15:44:56.715 INFO:tasks.workunit.client.0.smithi076.stdout:5/675: link d1a/d33/d3a/d67/da8/dc4/d65/l95 d1a/d33/d3a/d67/da8/dc4/le9 0 2024-03-23T15:44:56.715 INFO:tasks.workunit.client.0.smithi076.stdout:5/676: chown d1a/f2a 853 1 2024-03-23T15:44:56.715 INFO:tasks.workunit.client.0.smithi076.stdout:0/691: dread d2/de/d10/d43/f50 [0,4194304] 0 2024-03-23T15:44:56.715 INFO:tasks.workunit.client.0.smithi076.stdout:7/801: getdents d8/d4e/d98/df6/d104/d40/d5d 0 2024-03-23T15:44:56.715 INFO:tasks.workunit.client.0.smithi076.stdout:7/802: read - d8/d4e/d98/df6/d104/d40/d88/d5a/dde/fe7 zero size 2024-03-23T15:44:56.716 INFO:tasks.workunit.client.0.smithi076.stdout:8/563: read d8/f50 [2287950,128942] 0 2024-03-23T15:44:56.716 INFO:tasks.workunit.client.0.smithi076.stdout:8/564: fsync d8/f57 0 2024-03-23T15:44:56.717 INFO:tasks.workunit.client.0.smithi076.stdout:9/656: dwrite d1/f15 [0,4194304] 0 2024-03-23T15:44:56.719 INFO:tasks.workunit.client.0.smithi076.stdout:6/451: truncate d1/d4/d10/d6b/d8d/d60/f61 3079633 0 2024-03-23T15:44:56.720 INFO:tasks.workunit.client.0.smithi076.stdout:0/692: truncate d2/d1b/f86 6537639 0 2024-03-23T15:44:56.720 INFO:tasks.workunit.client.0.smithi076.stdout:0/693: chown d2/d9a/fc8 764 1 2024-03-23T15:44:56.720 INFO:tasks.workunit.client.0.smithi076.stdout:0/694: fdatasync d2/d8/d32/f4a 0 2024-03-23T15:44:56.720 INFO:tasks.workunit.client.0.smithi076.stdout:0/695: dread - d2/de/d10/d3a/fca zero size 2024-03-23T15:44:56.723 INFO:tasks.workunit.client.0.smithi076.stdout:8/565: symlink d8/d4f/lbd 0 2024-03-23T15:44:56.726 INFO:tasks.workunit.client.0.smithi076.stdout:9/657: symlink d1/d1f/d82/ld2 0 2024-03-23T15:44:56.728 INFO:tasks.workunit.client.0.smithi076.stdout:6/452: write d1/f11 [3330671,44553] 0 2024-03-23T15:44:56.729 INFO:tasks.workunit.client.0.smithi076.stdout:7/803: read d8/fd [3898412,107547] 0 2024-03-23T15:44:56.729 INFO:tasks.workunit.client.0.smithi076.stdout:7/804: truncate d8/de/d48/f84 894855 0 2024-03-23T15:44:56.734 INFO:tasks.workunit.client.0.smithi076.stdout:5/677: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/f9a [0,4194304] 0 2024-03-23T15:44:56.734 INFO:tasks.workunit.client.0.smithi076.stdout:6/453: dread d1/d4/d10/d6b/d8d/f46 [0,4194304] 0 2024-03-23T15:44:56.744 INFO:tasks.workunit.client.0.smithi076.stdout:1/379: rename d0/d35/d40/d47/d17/l3b to d0/d2/d70/l79 0 2024-03-23T15:44:56.745 INFO:tasks.workunit.client.0.smithi076.stdout:6/454: dread d1/dd/f26 [0,4194304] 0 2024-03-23T15:44:56.746 INFO:tasks.workunit.client.0.smithi076.stdout:0/696: dwrite d2/de/d10/d3a/d47/d69/d84/dc5/fd1 [0,4194304] 0 2024-03-23T15:44:56.746 INFO:tasks.workunit.client.0.smithi076.stdout:0/697: fdatasync d2/de/f95 0 2024-03-23T15:44:56.746 INFO:tasks.workunit.client.0.smithi076.stdout:2/690: write d1/d2e/d6a/f9b [3052727,38368] 0 2024-03-23T15:44:56.747 INFO:tasks.workunit.client.0.smithi076.stdout:9/658: dwrite d1/f48 [0,4194304] 0 2024-03-23T15:44:56.748 INFO:tasks.workunit.client.0.smithi076.stdout:8/566: rename d8/f52 to d8/d4f/d80/db7/fbe 0 2024-03-23T15:44:56.748 INFO:tasks.workunit.client.0.smithi076.stdout:1/380: mkdir d0/d2/d70/d7a 0 2024-03-23T15:44:56.749 INFO:tasks.workunit.client.0.smithi076.stdout:7/805: rename d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/fba to d8/d4e/d98/df6/f111 0 2024-03-23T15:44:56.750 INFO:tasks.workunit.client.0.smithi076.stdout:1/381: mkdir d0/d35/d40/d47/d17/d20/d7b 0 2024-03-23T15:44:56.752 INFO:tasks.workunit.client.0.smithi076.stdout:5/678: write d1a/d2c/fbe [4031163,13247] 0 2024-03-23T15:44:56.752 INFO:tasks.workunit.client.0.smithi076.stdout:7/806: write d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/f51 [1169735,67263] 0 2024-03-23T15:44:56.753 INFO:tasks.workunit.client.0.smithi076.stdout:9/659: creat d1/d1f/d46/d83/dad/fd3 x:0 0 0 2024-03-23T15:44:56.753 INFO:tasks.workunit.client.0.smithi076.stdout:5/679: creat d1a/d33/dc3/da6/fea x:0 0 0 2024-03-23T15:44:56.753 INFO:tasks.workunit.client.0.smithi076.stdout:5/680: write d1a/d2c/fce [546459,44929] 0 2024-03-23T15:44:56.754 INFO:tasks.workunit.client.0.smithi076.stdout:5/681: creat d1a/d2c/feb x:0 0 0 2024-03-23T15:44:56.755 INFO:tasks.workunit.client.0.smithi076.stdout:1/382: mknod d0/d35/d52/c7c 0 2024-03-23T15:44:56.756 INFO:tasks.workunit.client.0.smithi076.stdout:7/807: creat d8/d4e/d98/df6/d104/d24/d91/df7/f112 x:0 0 0 2024-03-23T15:44:56.756 INFO:tasks.workunit.client.0.smithi076.stdout:5/682: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fec x:0 0 0 2024-03-23T15:44:56.757 INFO:tasks.workunit.client.0.smithi076.stdout:5/683: truncate d1a/d33/fe6 152155 0 2024-03-23T15:44:56.757 INFO:tasks.workunit.client.0.smithi076.stdout:1/383: mknod d0/d2/d70/c7d 0 2024-03-23T15:44:56.758 INFO:tasks.workunit.client.0.smithi076.stdout:5/684: mknod d1a/d33/d36/d8b/ddf/ced 0 2024-03-23T15:44:56.758 INFO:tasks.workunit.client.0.smithi076.stdout:1/384: mknod d0/d2/d62/c7e 0 2024-03-23T15:44:56.760 INFO:tasks.workunit.client.0.smithi076.stdout:5/685: symlink d1a/d33/d3a/d67/da8/dc4/d5f/d8f/lee 0 2024-03-23T15:44:56.762 INFO:tasks.workunit.client.0.smithi076.stdout:5/686: getdents d1a/d33/d3a/d67 0 2024-03-23T15:44:56.764 INFO:tasks.workunit.client.0.smithi076.stdout:5/687: creat d1a/d33/dc3/da6/fef x:0 0 0 2024-03-23T15:44:56.765 INFO:tasks.workunit.client.0.smithi076.stdout:5/688: fdatasync d1a/d33/f73 0 2024-03-23T15:44:56.766 INFO:tasks.workunit.client.0.smithi076.stdout:6/455: dwrite d1/dd/df/f66 [0,4194304] 0 2024-03-23T15:44:56.766 INFO:tasks.workunit.client.0.smithi076.stdout:0/698: dread d2/de/d10/d43/f63 [0,4194304] 0 2024-03-23T15:44:56.767 INFO:tasks.workunit.client.0.smithi076.stdout:5/689: creat d1a/d33/ff0 x:0 0 0 2024-03-23T15:44:56.771 INFO:tasks.workunit.client.0.smithi076.stdout:1/385: dread d0/d35/d40/d47/d17/d20/f33 [0,4194304] 0 2024-03-23T15:44:56.774 INFO:tasks.workunit.client.0.smithi076.stdout:1/386: creat d0/d35/d40/d47/d74/f7f x:0 0 0 2024-03-23T15:44:56.776 INFO:tasks.workunit.client.0.smithi076.stdout:9/660: dwrite d1/d20/f43 [0,4194304] 0 2024-03-23T15:44:56.776 INFO:tasks.workunit.client.0.smithi076.stdout:9/661: dread - d1/d2f/fbc zero size 2024-03-23T15:44:56.778 INFO:tasks.workunit.client.0.smithi076.stdout:1/387: creat d0/d3c/d6e/f80 x:0 0 0 2024-03-23T15:44:56.779 INFO:tasks.workunit.client.0.smithi076.stdout:9/662: creat d1/d2f/d3a/d74/fd4 x:0 0 0 2024-03-23T15:44:56.780 INFO:tasks.workunit.client.0.smithi076.stdout:1/388: creat d0/d35/d52/f81 x:0 0 0 2024-03-23T15:44:56.781 INFO:tasks.workunit.client.0.smithi076.stdout:0/699: read d2/d8/d2d/f45 [2691431,82405] 0 2024-03-23T15:44:56.781 INFO:tasks.workunit.client.0.smithi076.stdout:0/700: write d2/de/d10/d3a/d47/d69/d84/f93 [3598816,47764] 0 2024-03-23T15:44:56.786 INFO:tasks.workunit.client.0.smithi076.stdout:6/456: write d1/d4/d10/d6b/d8d/d60/d7c/f83 [3370436,78362] 0 2024-03-23T15:44:56.786 INFO:tasks.workunit.client.0.smithi076.stdout:1/389: getdents d0/d35/d40/d47/d17/d20 0 2024-03-23T15:44:56.787 INFO:tasks.workunit.client.0.smithi076.stdout:1/390: write d0/f4b [108723,48555] 0 2024-03-23T15:44:56.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/663: write d1/d2f/d3a/d64/f85 [2346409,107259] 0 2024-03-23T15:44:56.789 INFO:tasks.workunit.client.0.smithi076.stdout:0/701: creat d2/de/d10/d43/d56/fd7 x:0 0 0 2024-03-23T15:44:56.790 INFO:tasks.workunit.client.0.smithi076.stdout:7/808: dwrite d8/de/fad [0,4194304] 0 2024-03-23T15:44:56.790 INFO:tasks.workunit.client.0.smithi076.stdout:9/664: mkdir d1/d2f/d3f/da0/dd5 0 2024-03-23T15:44:56.790 INFO:tasks.workunit.client.0.smithi076.stdout:5/690: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/f9b [0,4194304] 0 2024-03-23T15:44:56.790 INFO:tasks.workunit.client.0.smithi076.stdout:9/665: write d1/d1f/d29/d2d/d37/f59 [4719791,85430] 0 2024-03-23T15:44:56.790 INFO:tasks.workunit.client.0.smithi076.stdout:9/666: write d1/d1f/d46/d83/f73 [1492967,116181] 0 2024-03-23T15:44:56.790 INFO:tasks.workunit.client.0.smithi076.stdout:5/691: write d1a/d33/d36/d8b/fc2 [579574,81002] 0 2024-03-23T15:44:56.790 INFO:tasks.workunit.client.0.smithi076.stdout:5/692: stat d1a/d33/d36/d8b/fc2 0 2024-03-23T15:44:56.791 INFO:tasks.workunit.client.0.smithi076.stdout:0/702: symlink d2/de/d10/d3a/d47/ld8 0 2024-03-23T15:44:56.794 INFO:tasks.workunit.client.0.smithi076.stdout:7/809: mknod d8/d4e/d98/c113 0 2024-03-23T15:44:56.800 INFO:tasks.workunit.client.0.smithi076.stdout:0/703: unlink d2/daf/cbf 0 2024-03-23T15:44:56.800 INFO:tasks.workunit.client.0.smithi076.stdout:0/704: chown d2/de/d10/d3a/d47/d69/fbe 12 1 2024-03-23T15:44:56.800 INFO:tasks.workunit.client.0.smithi076.stdout:0/705: stat d2/d8/d2d/c83 0 2024-03-23T15:44:56.804 INFO:tasks.workunit.client.0.smithi076.stdout:2/691: rename d1/d4/db/d15/d29/d7d/dbd/dec to d1/d4/d52/ded 0 2024-03-23T15:44:56.804 INFO:tasks.workunit.client.0.smithi076.stdout:2/692: chown d1/d4/db/d3e/c6d 376 1 2024-03-23T15:44:56.804 INFO:tasks.workunit.client.0.smithi076.stdout:2/693: fdatasync d1/d4/db/d15/d29/d7d/fad 0 2024-03-23T15:44:56.807 INFO:tasks.workunit.client.0.smithi076.stdout:2/694: write d1/d4/d64/f48 [769626,13846] 0 2024-03-23T15:44:56.807 INFO:tasks.workunit.client.0.smithi076.stdout:6/457: dwrite d1/d2/f1c [0,4194304] 0 2024-03-23T15:44:56.807 INFO:tasks.workunit.client.0.smithi076.stdout:1/391: dwrite d0/d35/d40/d47/f9 [8388608,4194304] 0 2024-03-23T15:44:56.807 INFO:tasks.workunit.client.0.smithi076.stdout:0/706: dread d2/de/d10/d12/f23 [0,4194304] 0 2024-03-23T15:44:56.807 INFO:tasks.workunit.client.0.smithi076.stdout:9/667: dwrite d1/d1f/d46/d4a/d5a/daa/fb8 [0,4194304] 0 2024-03-23T15:44:56.808 INFO:tasks.workunit.client.0.smithi076.stdout:2/695: mknod d1/dbf/cee 0 2024-03-23T15:44:56.808 INFO:tasks.workunit.client.0.smithi076.stdout:2/696: fsync d1/d4/db/f68 0 2024-03-23T15:44:56.811 INFO:tasks.workunit.client.0.smithi076.stdout:6/458: link d1/d4/d10/c52 d1/d4/c9d 0 2024-03-23T15:44:56.815 INFO:tasks.workunit.client.0.smithi076.stdout:4/640: sync 2024-03-23T15:44:56.817 INFO:tasks.workunit.client.0.smithi076.stdout:4/641: read d8/d3a/d18/d52/f6d [185595,119843] 0 2024-03-23T15:44:56.818 INFO:tasks.workunit.client.0.smithi076.stdout:1/392: mkdir d0/d35/d82 0 2024-03-23T15:44:56.823 INFO:tasks.workunit.client.0.smithi076.stdout:2/697: dwrite d1/d4/d64/f26 [0,4194304] 0 2024-03-23T15:44:56.824 INFO:tasks.workunit.client.0.smithi076.stdout:4/642: link d8/ff d8/d80/fc0 0 2024-03-23T15:44:56.825 INFO:tasks.workunit.client.0.smithi076.stdout:9/668: rmdir d1/d1f 39 2024-03-23T15:44:56.825 INFO:tasks.workunit.client.0.smithi076.stdout:1/393: chown d0/d35/d40/d47/cf 12 1 2024-03-23T15:44:56.826 INFO:tasks.workunit.client.0.smithi076.stdout:4/643: link d8/d3a/d18/d1b/l2c d8/d9/lc1 0 2024-03-23T15:44:56.827 INFO:tasks.workunit.client.0.smithi076.stdout:1/394: mkdir d0/d35/d40/d47/d74/d83 0 2024-03-23T15:44:56.828 INFO:tasks.workunit.client.0.smithi076.stdout:4/644: rmdir d8/d3a/dab 39 2024-03-23T15:44:56.834 INFO:tasks.workunit.client.0.smithi076.stdout:0/707: dwrite d2/de/d10/d12/f9e [0,4194304] 0 2024-03-23T15:44:56.835 INFO:tasks.workunit.client.0.smithi076.stdout:0/708: dread d2/de/d10/d3a/d47/fab [0,4194304] 0 2024-03-23T15:44:56.836 INFO:tasks.workunit.client.0.smithi076.stdout:1/395: mknod d0/d3c/d6e/c84 0 2024-03-23T15:44:56.838 INFO:tasks.workunit.client.0.smithi076.stdout:0/709: getdents d2/d8 0 2024-03-23T15:44:56.842 INFO:tasks.workunit.client.0.smithi076.stdout:2/698: dwrite d1/d4/db/d15/f21 [0,4194304] 0 2024-03-23T15:44:56.842 INFO:tasks.workunit.client.0.smithi076.stdout:2/699: write d1/fd9 [130053,61139] 0 2024-03-23T15:44:56.844 INFO:tasks.workunit.client.0.smithi076.stdout:2/700: unlink d1/dbf/fd3 0 2024-03-23T15:44:56.846 INFO:tasks.workunit.client.0.smithi076.stdout:2/701: truncate d1/d4/d64/d1b/f44 1291630 0 2024-03-23T15:44:56.846 INFO:tasks.workunit.client.0.smithi076.stdout:2/702: read d1/d2e/d8a/f94 [721419,80357] 0 2024-03-23T15:44:56.846 INFO:tasks.workunit.client.0.smithi076.stdout:4/645: dwrite d8/d3a/d69/f86 [0,4194304] 0 2024-03-23T15:44:56.848 INFO:tasks.workunit.client.0.smithi076.stdout:2/703: creat d1/d4/d64/d1b/d32/d79/dac/dd8/dea/fef x:0 0 0 2024-03-23T15:44:56.850 INFO:tasks.workunit.client.0.smithi076.stdout:2/704: creat d1/d4/d64/d1b/dd4/ff0 x:0 0 0 2024-03-23T15:44:56.852 INFO:tasks.workunit.client.0.smithi076.stdout:2/705: creat d1/d4/d64/d1b/dd4/ff1 x:0 0 0 2024-03-23T15:44:56.853 INFO:tasks.workunit.client.0.smithi076.stdout:2/706: readlink d1/d4/db/d15/l74 0 2024-03-23T15:44:56.854 INFO:tasks.workunit.client.0.smithi076.stdout:0/710: rmdir d2/de/d10/d43/d56 39 2024-03-23T15:44:56.855 INFO:tasks.workunit.client.0.smithi076.stdout:2/707: symlink d1/d4/d64/d82/d7f/lf2 0 2024-03-23T15:44:56.856 INFO:tasks.workunit.client.0.smithi076.stdout:2/708: rmdir d1/d4/d64/d1d/dbc 39 2024-03-23T15:44:56.856 INFO:tasks.workunit.client.0.smithi076.stdout:9/669: dwrite d1/d1f/d46/d4a/f50 [0,4194304] 0 2024-03-23T15:44:56.857 INFO:tasks.workunit.client.0.smithi076.stdout:3/515: sync 2024-03-23T15:44:56.857 INFO:tasks.workunit.client.0.smithi076.stdout:8/567: rename d8/d2e/l38 to d8/d2e/d30/d69/db9/lbf 0 2024-03-23T15:44:56.857 INFO:tasks.workunit.client.0.smithi076.stdout:9/670: dread d1/d1f/d29/d2d/d37/d47/fbe [0,4194304] 0 2024-03-23T15:44:56.858 INFO:tasks.workunit.client.0.smithi076.stdout:8/568: rmdir d8/d6b 39 2024-03-23T15:44:56.858 INFO:tasks.workunit.client.0.smithi076.stdout:2/709: symlink d1/d2e/lf3 0 2024-03-23T15:44:56.859 INFO:tasks.workunit.client.0.smithi076.stdout:2/710: fdatasync d1/d4/d64/d1d/fca 0 2024-03-23T15:44:56.859 INFO:tasks.workunit.client.0.smithi076.stdout:2/711: chown d1/d4/d64/d1b/f63 2660 1 2024-03-23T15:44:56.859 INFO:tasks.workunit.client.0.smithi076.stdout:2/712: write d1/d4/d64/d1b/d32/d79/fa9 [580853,32962] 0 2024-03-23T15:44:56.859 INFO:tasks.workunit.client.0.smithi076.stdout:2/713: fsync d1/d4/d64/fc7 0 2024-03-23T15:44:56.859 INFO:tasks.workunit.client.0.smithi076.stdout:2/714: getdents d1/d4/db/d15/d29 0 2024-03-23T15:44:56.860 INFO:tasks.workunit.client.0.smithi076.stdout:5/693: rename d1a/d33/d36/d8b to d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1 0 2024-03-23T15:44:56.860 INFO:tasks.workunit.client.0.smithi076.stdout:5/694: chown d1a/d2c/fbe 87770 1 2024-03-23T15:44:56.860 INFO:tasks.workunit.client.0.smithi076.stdout:5/695: write f18 [4686468,100276] 0 2024-03-23T15:44:56.860 INFO:tasks.workunit.client.0.smithi076.stdout:3/516: truncate d5/d17/d1c/d32/d39/f53 475203 0 2024-03-23T15:44:56.860 INFO:tasks.workunit.client.0.smithi076.stdout:3/517: stat d5/d17/d1c/d32/f5f 0 2024-03-23T15:44:56.861 INFO:tasks.workunit.client.0.smithi076.stdout:2/715: symlink d1/d2e/d6a/lf4 0 2024-03-23T15:44:56.861 INFO:tasks.workunit.client.0.smithi076.stdout:2/716: chown d1/d4/db/c66 5647 1 2024-03-23T15:44:56.861 INFO:tasks.workunit.client.0.smithi076.stdout:9/671: symlink d1/d1f/d29/d7f/ld6 0 2024-03-23T15:44:56.861 INFO:tasks.workunit.client.0.smithi076.stdout:2/717: write d1/fd9 [374165,24197] 0 2024-03-23T15:44:56.861 INFO:tasks.workunit.client.0.smithi076.stdout:5/696: stat le 0 2024-03-23T15:44:56.863 INFO:tasks.workunit.client.0.smithi076.stdout:8/569: creat d8/d2e/d30/d64/fc0 x:0 0 0 2024-03-23T15:44:56.863 INFO:tasks.workunit.client.0.smithi076.stdout:3/518: creat d5/d35/d6d/d7f/fac x:0 0 0 2024-03-23T15:44:56.864 INFO:tasks.workunit.client.0.smithi076.stdout:9/672: mkdir d1/d2f/d3a/d74/dd7 0 2024-03-23T15:44:56.864 INFO:tasks.workunit.client.0.smithi076.stdout:9/673: write d1/d1f/fbb [913103,80026] 0 2024-03-23T15:44:56.869 INFO:tasks.workunit.client.0.smithi076.stdout:8/570: write d8/d26/f4d [1112718,49431] 0 2024-03-23T15:44:56.869 INFO:tasks.workunit.client.0.smithi076.stdout:8/571: truncate d8/d26/d33/f3a 1819825 0 2024-03-23T15:44:56.871 INFO:tasks.workunit.client.0.smithi076.stdout:9/674: rmdir d1/d1f/d46/d83/dad 39 2024-03-23T15:44:56.871 INFO:tasks.workunit.client.0.smithi076.stdout:9/675: chown d1/d1f/d46/d4a/l4f 1551488207 1 2024-03-23T15:44:56.873 INFO:tasks.workunit.client.0.smithi076.stdout:9/676: creat d1/d1f/d29/d2d/fd8 x:0 0 0 2024-03-23T15:44:56.873 INFO:tasks.workunit.client.0.smithi076.stdout:9/677: truncate d1/d2f/d3f/da0/fc9 585395 0 2024-03-23T15:44:56.874 INFO:tasks.workunit.client.0.smithi076.stdout:7/810: rename d8/d4e/d98/df6/d104/d3a/fae to d8/d4e/d98/df6/d104/d24/de4/f114 0 2024-03-23T15:44:56.874 INFO:tasks.workunit.client.0.smithi076.stdout:7/811: dread - d8/d4e/d98/df6/d104/d3a/dbf/dc7/f66 zero size 2024-03-23T15:44:56.875 INFO:tasks.workunit.client.0.smithi076.stdout:7/812: chown d8/d4e/d98/df6 36991 1 2024-03-23T15:44:56.875 INFO:tasks.workunit.client.0.smithi076.stdout:2/718: dwrite d1/d4/d64/d82/d7f/f8c [0,4194304] 0 2024-03-23T15:44:56.875 INFO:tasks.workunit.client.0.smithi076.stdout:8/572: dread d8/f57 [0,4194304] 0 2024-03-23T15:44:56.876 INFO:tasks.workunit.client.0.smithi076.stdout:2/719: mknod d1/d4/cf5 0 2024-03-23T15:44:56.876 INFO:tasks.workunit.client.0.smithi076.stdout:2/720: fsync d1/d2e/d8a/fd6 0 2024-03-23T15:44:56.876 INFO:tasks.workunit.client.0.smithi076.stdout:9/678: getdents d1/d2f/d3f 0 2024-03-23T15:44:56.877 INFO:tasks.workunit.client.0.smithi076.stdout:5/697: rename d1a/d33/d3a/d67/da8/dc4/c9e to d1a/d2c/cf2 0 2024-03-23T15:44:56.877 INFO:tasks.workunit.client.0.smithi076.stdout:2/721: mkdir d1/d2e/dd1/df6 0 2024-03-23T15:44:56.877 INFO:tasks.workunit.client.0.smithi076.stdout:2/722: chown d1/l4a 413039 1 2024-03-23T15:44:56.878 INFO:tasks.workunit.client.0.smithi076.stdout:9/679: creat d1/d2f/d3f/da0/fd9 x:0 0 0 2024-03-23T15:44:56.878 INFO:tasks.workunit.client.0.smithi076.stdout:9/680: fdatasync d1/d1f/d46/d4a/f50 0 2024-03-23T15:44:56.878 INFO:tasks.workunit.client.0.smithi076.stdout:2/723: creat d1/d2e/ff7 x:0 0 0 2024-03-23T15:44:56.878 INFO:tasks.workunit.client.0.smithi076.stdout:2/724: dread - d1/d4/d64/d1b/f71 zero size 2024-03-23T15:44:56.879 INFO:tasks.workunit.client.0.smithi076.stdout:9/681: symlink d1/d1f/d29/d7f/lda 0 2024-03-23T15:44:56.880 INFO:tasks.workunit.client.0.smithi076.stdout:2/725: creat d1/d2e/dd1/ddc/ff8 x:0 0 0 2024-03-23T15:44:56.881 INFO:tasks.workunit.client.0.smithi076.stdout:2/726: mknod d1/d2e/dd1/cf9 0 2024-03-23T15:44:56.881 INFO:tasks.workunit.client.0.smithi076.stdout:5/698: mknod d1a/d33/d3a/d67/d84/cf3 0 2024-03-23T15:44:56.883 INFO:tasks.workunit.client.0.smithi076.stdout:5/699: creat d1a/d4b/ff4 x:0 0 0 2024-03-23T15:44:56.883 INFO:tasks.workunit.client.0.smithi076.stdout:9/682: mknod d1/d1f/d46/d83/cdb 0 2024-03-23T15:44:56.884 INFO:tasks.workunit.client.0.smithi076.stdout:5/700: dread d1a/f68 [0,4194304] 0 2024-03-23T15:44:56.884 INFO:tasks.workunit.client.0.smithi076.stdout:8/573: rename d8/d26/d33/c40 to d8/d89/cc1 0 2024-03-23T15:44:56.885 INFO:tasks.workunit.client.0.smithi076.stdout:9/683: creat d1/d1f/d82/d9c/fdc x:0 0 0 2024-03-23T15:44:56.885 INFO:tasks.workunit.client.0.smithi076.stdout:5/701: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/ff5 x:0 0 0 2024-03-23T15:44:56.887 INFO:tasks.workunit.client.0.smithi076.stdout:8/574: creat d8/d89/db2/fc2 x:0 0 0 2024-03-23T15:44:56.887 INFO:tasks.workunit.client.0.smithi076.stdout:8/575: chown d8/d6b/d91 208833 1 2024-03-23T15:44:56.887 INFO:tasks.workunit.client.0.smithi076.stdout:7/813: dwrite d8/d4e/d98/df6/f110 [0,4194304] 0 2024-03-23T15:44:56.888 INFO:tasks.workunit.client.0.smithi076.stdout:9/684: rename d1/f15 to d1/d2f/d3f/da0/dd5/fdd 0 2024-03-23T15:44:56.888 INFO:tasks.workunit.client.0.smithi076.stdout:5/702: rmdir d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf 39 2024-03-23T15:44:56.889 INFO:tasks.workunit.client.0.smithi076.stdout:7/814: unlink d8/de/de8/lfb 0 2024-03-23T15:44:56.889 INFO:tasks.workunit.client.0.smithi076.stdout:7/815: dread - d8/d4e/d98/df6/d104/d3a/dbf/dc7/f66 zero size 2024-03-23T15:44:56.891 INFO:tasks.workunit.client.0.smithi076.stdout:9/685: link d1/ca d1/d1f/d46/d83/da7/cde 0 2024-03-23T15:44:56.892 INFO:tasks.workunit.client.0.smithi076.stdout:3/519: link d5/d17/d1c/d32/d39/f3e d5/d35/d6d/d7f/fad 0 2024-03-23T15:44:56.892 INFO:tasks.workunit.client.0.smithi076.stdout:3/520: write d5/d35/d4c/d5a/f49 [1667028,97748] 0 2024-03-23T15:44:56.893 INFO:tasks.workunit.client.0.smithi076.stdout:7/816: unlink d8/d4e/d98/df6/d104/d40/d88/fb5 0 2024-03-23T15:44:56.896 INFO:tasks.workunit.client.0.smithi076.stdout:9/686: unlink d1/d1f/d29/d2d/d37/d47/l8d 0 2024-03-23T15:44:56.905 INFO:tasks.workunit.client.0.smithi076.stdout:8/576: dwrite d8/d26/d27/f9e [0,4194304] 0 2024-03-23T15:44:56.905 INFO:tasks.workunit.client.0.smithi076.stdout:3/521: dread d5/d17/f29 [0,4194304] 0 2024-03-23T15:44:56.905 INFO:tasks.workunit.client.0.smithi076.stdout:3/522: chown d5/l8 3 1 2024-03-23T15:44:56.906 INFO:tasks.workunit.client.0.smithi076.stdout:5/703: dwrite d1a/d33/d3a/d67/da8/dc4/f52 [0,4194304] 0 2024-03-23T15:44:56.906 INFO:tasks.workunit.client.0.smithi076.stdout:5/704: chown d1a/d2c 757 1 2024-03-23T15:44:56.909 INFO:tasks.workunit.client.0.smithi076.stdout:5/705: creat d1a/d33/d3a/d67/da8/dc4/ff6 x:0 0 0 2024-03-23T15:44:56.909 INFO:tasks.workunit.client.0.smithi076.stdout:5/706: chown d1a/d33/d3a/fda 664896156 1 2024-03-23T15:44:56.910 INFO:tasks.workunit.client.0.smithi076.stdout:5/707: creat d1a/d2c/ff7 x:0 0 0 2024-03-23T15:44:56.910 INFO:tasks.workunit.client.0.smithi076.stdout:5/708: write d1a/d33/f3b [1362810,100799] 0 2024-03-23T15:44:56.913 INFO:tasks.workunit.client.0.smithi076.stdout:5/709: unlink d1a/d33/c60 0 2024-03-23T15:44:56.913 INFO:tasks.workunit.client.0.smithi076.stdout:5/710: fsync d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/fe1 0 2024-03-23T15:44:56.913 INFO:tasks.workunit.client.0.smithi076.stdout:5/711: write d1a/d33/d3a/d80/fca [554618,19163] 0 2024-03-23T15:44:56.914 INFO:tasks.workunit.client.0.smithi076.stdout:7/817: dwrite d8/d4e/d98/df6/d104/d40/d5d/d3c/f73 [0,4194304] 0 2024-03-23T15:44:56.914 INFO:tasks.workunit.client.0.smithi076.stdout:7/818: readlink d8/l10a 0 2024-03-23T15:44:56.914 INFO:tasks.workunit.client.0.smithi076.stdout:5/712: creat d1a/d33/d3a/d80/ff8 x:0 0 0 2024-03-23T15:44:56.916 INFO:tasks.workunit.client.0.smithi076.stdout:5/713: write d1a/d33/f73 [689309,75628] 0 2024-03-23T15:44:56.916 INFO:tasks.workunit.client.0.smithi076.stdout:5/714: write d1a/d33/d3a/d67/da8/dc4/d5f/f62 [1730679,67927] 0 2024-03-23T15:44:56.927 INFO:tasks.workunit.client.0.smithi076.stdout:2/727: dread d1/d4/f54 [0,4194304] 0 2024-03-23T15:44:56.927 INFO:tasks.workunit.client.0.smithi076.stdout:7/819: write d8/de/f26 [3319062,2064] 0 2024-03-23T15:44:56.928 INFO:tasks.workunit.client.0.smithi076.stdout:5/715: read d1a/f29 [2160681,38420] 0 2024-03-23T15:44:56.928 INFO:tasks.workunit.client.0.smithi076.stdout:7/820: getdents d8/d4e/d98/df6/d104/d18 0 2024-03-23T15:44:56.928 INFO:tasks.workunit.client.0.smithi076.stdout:7/821: chown d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/fdc 21773049 1 2024-03-23T15:44:56.929 INFO:tasks.workunit.client.0.smithi076.stdout:5/716: write d1a/d7e/fab [195839,122073] 0 2024-03-23T15:44:56.929 INFO:tasks.workunit.client.0.smithi076.stdout:7/822: mknod d8/de/d48/c115 0 2024-03-23T15:44:56.931 INFO:tasks.workunit.client.0.smithi076.stdout:5/717: truncate d1a/d33/f35 2224591 0 2024-03-23T15:44:56.931 INFO:tasks.workunit.client.0.smithi076.stdout:5/718: dread - d1a/d33/dc3/da6/fea zero size 2024-03-23T15:44:56.932 INFO:tasks.workunit.client.0.smithi076.stdout:9/687: dwrite d1/d2f/d3f/f92 [0,4194304] 0 2024-03-23T15:44:56.935 INFO:tasks.workunit.client.0.smithi076.stdout:7/823: dread f1 [0,4194304] 0 2024-03-23T15:44:56.936 INFO:tasks.workunit.client.0.smithi076.stdout:5/719: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/ff9 x:0 0 0 2024-03-23T15:44:56.937 INFO:tasks.workunit.client.0.smithi076.stdout:7/824: creat d8/d4e/d98/df6/d104/f116 x:0 0 0 2024-03-23T15:44:56.942 INFO:tasks.workunit.client.0.smithi076.stdout:7/825: mknod d8/d4e/d98/df6/d104/d24/de4/c117 0 2024-03-23T15:44:56.944 INFO:tasks.workunit.client.0.smithi076.stdout:2/728: dwrite d1/d4/d64/d1d/fe5 [0,4194304] 0 2024-03-23T15:44:56.945 INFO:tasks.workunit.client.0.smithi076.stdout:2/729: read d1/d4/db/f5b [1851402,91281] 0 2024-03-23T15:44:56.945 INFO:tasks.workunit.client.0.smithi076.stdout:2/730: chown d1/d4/d64/f20 489 1 2024-03-23T15:44:56.946 INFO:tasks.workunit.client.0.smithi076.stdout:8/577: rename d8/d26/d27/fa8 to d8/d2e/d30/fc3 0 2024-03-23T15:44:56.953 INFO:tasks.workunit.client.0.smithi076.stdout:8/578: read d8/f16 [1089812,19961] 0 2024-03-23T15:44:56.953 INFO:tasks.workunit.client.0.smithi076.stdout:5/720: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/ff9 [0,4194304] 0 2024-03-23T15:44:56.953 INFO:tasks.workunit.client.0.smithi076.stdout:9/688: dwrite d1/d1f/fd1 [0,4194304] 0 2024-03-23T15:44:56.954 INFO:tasks.workunit.client.0.smithi076.stdout:2/731: write d1/d4/f5f [5318204,45819] 0 2024-03-23T15:44:56.958 INFO:tasks.workunit.client.0.smithi076.stdout:9/689: creat d1/d1f/d82/d9c/fdf x:0 0 0 2024-03-23T15:44:56.958 INFO:tasks.workunit.client.0.smithi076.stdout:7/826: write d8/d4e/d98/df6/d104/d24/de4/f114 [855161,77483] 0 2024-03-23T15:44:56.959 INFO:tasks.workunit.client.0.smithi076.stdout:7/827: readlink d8/d4e/d98/df6/d104/d3a/dbf/dc7/lda 0 2024-03-23T15:44:56.959 INFO:tasks.workunit.client.0.smithi076.stdout:7/828: readlink d8/d4e/d98/df6/d104/d18/l86 0 2024-03-23T15:44:56.959 INFO:tasks.workunit.client.0.smithi076.stdout:8/579: mknod d8/d26/d33/cc4 0 2024-03-23T15:44:56.959 INFO:tasks.workunit.client.0.smithi076.stdout:9/690: symlink d1/d1f/d46/d83/da7/le0 0 2024-03-23T15:44:56.960 INFO:tasks.workunit.client.0.smithi076.stdout:2/732: rename d1/f2f to d1/d4/d64/d82/ffa 0 2024-03-23T15:44:56.962 INFO:tasks.workunit.client.0.smithi076.stdout:7/829: creat d8/d4e/d98/df6/f118 x:0 0 0 2024-03-23T15:44:56.962 INFO:tasks.workunit.client.0.smithi076.stdout:7/830: read - d8/de/f105 zero size 2024-03-23T15:44:56.963 INFO:tasks.workunit.client.0.smithi076.stdout:9/691: creat d1/d1f/d46/d4a/fe1 x:0 0 0 2024-03-23T15:44:56.963 INFO:tasks.workunit.client.0.smithi076.stdout:9/692: fsync d1/d1f/d29/d2d/f86 0 2024-03-23T15:44:56.963 INFO:tasks.workunit.client.0.smithi076.stdout:2/733: stat d1/d4/d64/d82/c7b 0 2024-03-23T15:44:56.964 INFO:tasks.workunit.client.0.smithi076.stdout:2/734: stat d1/d4/f54 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:8/580: rename d8/f10 to d8/d26/fc5 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:2/735: mkdir d1/d4/dfb 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:2/736: dread - d1/d4/d64/d7c/fc3 zero size 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:2/737: stat d1/d4/d52/f70 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:8/581: fdatasync d8/d2e/d30/d69/f71 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:7/831: symlink d8/d4e/d98/df6/d104/d3a/d85/daf/dca/l119 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:2/738: symlink d1/d4/d64/d1b/d32/d79/dac/dd8/lfc 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:9/693: creat d1/d2f/d3a/fe2 x:0 0 0 2024-03-23T15:44:56.966 INFO:tasks.workunit.client.0.smithi076.stdout:9/694: fsync d1/d1f/fbb 0 2024-03-23T15:44:56.967 INFO:tasks.workunit.client.0.smithi076.stdout:7/832: mkdir d8/d4e/d98/df6/d104/d3a/d85/daf/dca/d11a 0 2024-03-23T15:44:56.969 INFO:tasks.workunit.client.0.smithi076.stdout:8/582: symlink d8/d2e/d30/lc6 0 2024-03-23T15:44:56.969 INFO:tasks.workunit.client.0.smithi076.stdout:7/833: creat d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/f11b x:0 0 0 2024-03-23T15:44:56.969 INFO:tasks.workunit.client.0.smithi076.stdout:7/834: write d8/d4e/d8d/dbe/fdb [4476811,91399] 0 2024-03-23T15:44:56.970 INFO:tasks.workunit.client.0.smithi076.stdout:8/583: mknod d8/d6b/d9a/cc7 0 2024-03-23T15:44:56.970 INFO:tasks.workunit.client.0.smithi076.stdout:8/584: write d8/d26/d33/d8c/fb5 [488519,22492] 0 2024-03-23T15:44:56.970 INFO:tasks.workunit.client.0.smithi076.stdout:8/585: dread - d8/d2e/d3e/f44 zero size 2024-03-23T15:44:56.970 INFO:tasks.workunit.client.0.smithi076.stdout:8/586: dread - d8/d2e/d30/fa2 zero size 2024-03-23T15:44:56.970 INFO:tasks.workunit.client.0.smithi076.stdout:8/587: rmdir d8/d6b/d9a 39 2024-03-23T15:44:56.972 INFO:tasks.workunit.client.0.smithi076.stdout:5/721: dwrite d1a/d2c/f87 [0,4194304] 0 2024-03-23T15:44:56.972 INFO:tasks.workunit.client.0.smithi076.stdout:5/722: chown d1a/d33/d3a/d67/d84/fd3 338839 1 2024-03-23T15:44:56.972 INFO:tasks.workunit.client.0.smithi076.stdout:5/723: write f5 [176079,44157] 0 2024-03-23T15:44:56.992 INFO:tasks.workunit.client.0.smithi076.stdout:9/695: dwrite d1/f18 [0,4194304] 0 2024-03-23T15:44:56.992 INFO:tasks.workunit.client.0.smithi076.stdout:6/459: sync 2024-03-23T15:44:56.992 INFO:tasks.workunit.client.0.smithi076.stdout:6/460: unlink d1/d4/d10/d6b/d8d/d60/f61 0 2024-03-23T15:44:56.992 INFO:tasks.workunit.client.0.smithi076.stdout:9/696: symlink d1/d2f/d3a/le3 0 2024-03-23T15:44:56.992 INFO:tasks.workunit.client.0.smithi076.stdout:7/835: dwrite d8/d4e/d98/df6/d104/d3a/dbf/dc7/ff9 [0,4194304] 0 2024-03-23T15:44:56.993 INFO:tasks.workunit.client.0.smithi076.stdout:9/697: symlink d1/d1f/d29/d2d/d37/d47/le4 0 2024-03-23T15:44:56.993 INFO:tasks.workunit.client.0.smithi076.stdout:5/724: dwrite d1a/d33/d3a/d80/fd8 [0,4194304] 0 2024-03-23T15:44:56.993 INFO:tasks.workunit.client.0.smithi076.stdout:7/836: creat d8/d4e/d8d/f11c x:0 0 0 2024-03-23T15:44:56.994 INFO:tasks.workunit.client.0.smithi076.stdout:7/837: write d8/d4e/d98/df6/d104/f21 [929722,121143] 0 2024-03-23T15:44:56.994 INFO:tasks.workunit.client.0.smithi076.stdout:6/461: unlink d1/d4/d10/d6b/c80 0 2024-03-23T15:44:56.996 INFO:tasks.workunit.client.0.smithi076.stdout:9/698: rename d1/d2f/d3a/f96 to d1/d1f/d46/d83/da7/fe5 0 2024-03-23T15:44:56.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/838: creat d8/d4e/d98/df6/d104/d3a/dd2/f11d x:0 0 0 2024-03-23T15:44:56.997 INFO:tasks.workunit.client.0.smithi076.stdout:7/839: stat d8 0 2024-03-23T15:44:56.998 INFO:tasks.workunit.client.0.smithi076.stdout:7/840: chown d8/d4e/d98/df6/d104/d18/c23 0 1 2024-03-23T15:44:56.999 INFO:tasks.workunit.client.0.smithi076.stdout:7/841: creat d8/f11e x:0 0 0 2024-03-23T15:44:57.000 INFO:tasks.workunit.client.0.smithi076.stdout:6/462: read d1/f85 [2632079,130930] 0 2024-03-23T15:44:57.002 INFO:tasks.workunit.client.0.smithi076.stdout:5/725: read d1a/f92 [2006308,119053] 0 2024-03-23T15:44:57.003 INFO:tasks.workunit.client.0.smithi076.stdout:5/726: symlink d1a/d33/d3a/d67/da8/dc4/d5f/ddb/lfa 0 2024-03-23T15:44:57.004 INFO:tasks.workunit.client.0.smithi076.stdout:5/727: symlink d1a/d33/d3a/d67/da8/dc4/lfb 0 2024-03-23T15:44:57.004 INFO:tasks.workunit.client.0.smithi076.stdout:5/728: write d1a/d4b/fc6 [714566,124628] 0 2024-03-23T15:44:57.013 INFO:tasks.workunit.client.0.smithi076.stdout:7/842: write d8/d4e/d8d/dbe/fdb [3460962,11100] 0 2024-03-23T15:44:57.014 INFO:tasks.workunit.client.0.smithi076.stdout:7/843: rmdir d8/d4e/d8d 39 2024-03-23T15:44:57.015 INFO:tasks.workunit.client.0.smithi076.stdout:9/699: dwrite d1/d1f/d29/d2d/f33 [0,4194304] 0 2024-03-23T15:44:57.016 INFO:tasks.workunit.client.0.smithi076.stdout:7/844: mkdir d8/d4e/d98/df6/d104/d3a/d85/daf/dca/d11f 0 2024-03-23T15:44:57.017 INFO:tasks.workunit.client.0.smithi076.stdout:7/845: chown d8/d4e/d98/df6/d104/d24/de4/f114 117235816 1 2024-03-23T15:44:57.017 INFO:tasks.workunit.client.0.smithi076.stdout:7/846: truncate d8/d4e/d98/df6/d104/d40/f90 583336 0 2024-03-23T15:44:57.017 INFO:tasks.workunit.client.0.smithi076.stdout:7/847: readlink d8/d4e/d98/df6/d104/d40/d5d/la8 0 2024-03-23T15:44:57.019 INFO:tasks.workunit.client.0.smithi076.stdout:6/463: dwrite d1/d2/f6d [0,4194304] 0 2024-03-23T15:44:57.020 INFO:tasks.workunit.client.0.smithi076.stdout:5/729: dwrite d1a/d4b/f50 [4194304,4194304] 0 2024-03-23T15:44:57.020 INFO:tasks.workunit.client.0.smithi076.stdout:5/730: truncate d1a/d33/d36/f3d 768950 0 2024-03-23T15:44:57.022 INFO:tasks.workunit.client.0.smithi076.stdout:6/464: creat d1/d4/d10/d6b/d8d/f9e x:0 0 0 2024-03-23T15:44:57.022 INFO:tasks.workunit.client.0.smithi076.stdout:6/465: fsync d1/d4/d10/f4a 0 2024-03-23T15:44:57.023 INFO:tasks.workunit.client.0.smithi076.stdout:5/731: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/ffc x:0 0 0 2024-03-23T15:44:57.024 INFO:tasks.workunit.client.0.smithi076.stdout:6/466: creat d1/d4/d10/d6b/d8d/f9f x:0 0 0 2024-03-23T15:44:57.024 INFO:tasks.workunit.client.0.smithi076.stdout:7/848: dread d8/fd [0,4194304] 0 2024-03-23T15:44:57.025 INFO:tasks.workunit.client.0.smithi076.stdout:7/849: chown d8/d4e/fd0 4 1 2024-03-23T15:44:57.025 INFO:tasks.workunit.client.0.smithi076.stdout:7/850: fsync d8/d4e/d98/df6/d104/f2f 0 2024-03-23T15:44:57.025 INFO:tasks.workunit.client.0.smithi076.stdout:7/851: dread - d8/f11e zero size 2024-03-23T15:44:57.025 INFO:tasks.workunit.client.0.smithi076.stdout:5/732: mkdir d1a/d33/dfd 0 2024-03-23T15:44:57.025 INFO:tasks.workunit.client.0.smithi076.stdout:6/467: mknod d1/d4/d10/d6b/ca0 0 2024-03-23T15:44:57.027 INFO:tasks.workunit.client.0.smithi076.stdout:5/733: link d1a/d2c/f87 d1a/d33/d3a/d67/da8/dc4/ffe 0 2024-03-23T15:44:57.027 INFO:tasks.workunit.client.0.smithi076.stdout:5/734: write d1a/d33/f51 [1572307,3387] 0 2024-03-23T15:44:57.028 INFO:tasks.workunit.client.0.smithi076.stdout:5/735: creat d1a/d20/fff x:0 0 0 2024-03-23T15:44:57.028 INFO:tasks.workunit.client.0.smithi076.stdout:5/736: write d1a/d33/d3a/d67/da8/dc4/fc8 [611960,124755] 0 2024-03-23T15:44:57.028 INFO:tasks.workunit.client.0.smithi076.stdout:5/737: dread - d1a/d33/d3a/d67/da8/dc4/d65/f88 zero size 2024-03-23T15:44:57.030 INFO:tasks.workunit.client.0.smithi076.stdout:5/738: mkdir d1a/d33/d3a/d80/d100 0 2024-03-23T15:44:57.032 INFO:tasks.workunit.client.0.smithi076.stdout:9/700: dwrite d1/f35 [0,4194304] 0 2024-03-23T15:44:57.034 INFO:tasks.workunit.client.0.smithi076.stdout:9/701: symlink d1/d1f/d46/d4a/d5a/le6 0 2024-03-23T15:44:57.034 INFO:tasks.workunit.client.0.smithi076.stdout:9/702: readlink d1/d1f/d29/d2d/l5f 0 2024-03-23T15:44:57.036 INFO:tasks.workunit.client.0.smithi076.stdout:9/703: rmdir d1/d1f/d46/d4a/d5a/daa 39 2024-03-23T15:44:57.036 INFO:tasks.workunit.client.0.smithi076.stdout:9/704: dread - d1/d2f/d3f/f8b zero size 2024-03-23T15:44:57.037 INFO:tasks.workunit.client.0.smithi076.stdout:6/468: read d1/dd/df/f66 [1037071,19928] 0 2024-03-23T15:44:57.037 INFO:tasks.workunit.client.0.smithi076.stdout:9/705: symlink d1/d2f/d3a/d64/le7 0 2024-03-23T15:44:57.038 INFO:tasks.workunit.client.0.smithi076.stdout:9/706: dread - d1/d1f/d46/d4a/fe1 zero size 2024-03-23T15:44:57.038 INFO:tasks.workunit.client.0.smithi076.stdout:5/739: dread d1a/d33/d3a/d67/da8/dc4/f52 [0,4194304] 0 2024-03-23T15:44:57.039 INFO:tasks.workunit.client.0.smithi076.stdout:9/707: symlink d1/d1f/d29/d2d/d37/d47/le8 0 2024-03-23T15:44:57.040 INFO:tasks.workunit.client.0.smithi076.stdout:5/740: getdents d1a/d33/d3a/d67/da8/dc4/d5f/ddb 0 2024-03-23T15:44:57.040 INFO:tasks.workunit.client.0.smithi076.stdout:5/741: chown d1a/d2c/le0 23 1 2024-03-23T15:44:57.041 INFO:tasks.workunit.client.0.smithi076.stdout:9/708: truncate d1/d1f/d29/d2d/d37/f3c 1371095 0 2024-03-23T15:44:57.041 INFO:tasks.workunit.client.0.smithi076.stdout:9/709: fsync d1/d1f/d82/d9c/fa9 0 2024-03-23T15:44:57.041 INFO:tasks.workunit.client.0.smithi076.stdout:9/710: fsync d1/f35 0 2024-03-23T15:44:57.043 INFO:tasks.workunit.client.0.smithi076.stdout:5/742: link d1a/d20/c71 d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/c101 0 2024-03-23T15:44:57.044 INFO:tasks.workunit.client.0.smithi076.stdout:7/852: dwrite d8/d4e/fe9 [0,4194304] 0 2024-03-23T15:44:57.044 INFO:tasks.workunit.client.0.smithi076.stdout:5/743: rmdir d1a/d33/dc3/da6 39 2024-03-23T15:44:57.050 INFO:tasks.workunit.client.0.smithi076.stdout:7/853: dread d8/d4e/fd0 [0,4194304] 0 2024-03-23T15:44:57.051 INFO:tasks.workunit.client.0.smithi076.stdout:9/711: dread d1/d2f/d3a/d74/fb3 [0,4194304] 0 2024-03-23T15:44:57.052 INFO:tasks.workunit.client.0.smithi076.stdout:7/854: mkdir d8/de/de8/d120 0 2024-03-23T15:44:57.053 INFO:tasks.workunit.client.0.smithi076.stdout:9/712: symlink d1/d2f/le9 0 2024-03-23T15:44:57.053 INFO:tasks.workunit.client.0.smithi076.stdout:9/713: chown d1/d1f/d29/c94 432 1 2024-03-23T15:44:57.058 INFO:tasks.workunit.client.0.smithi076.stdout:7/855: dread f1 [0,4194304] 0 2024-03-23T15:44:57.059 INFO:tasks.workunit.client.0.smithi076.stdout:7/856: symlink d8/d4e/d98/df6/d104/d40/d5d/l121 0 2024-03-23T15:44:57.060 INFO:tasks.workunit.client.0.smithi076.stdout:7/857: read d8/d4e/d98/df6/d104/d3a/dbf/dc7/f77 [977059,51010] 0 2024-03-23T15:44:57.060 INFO:tasks.workunit.client.0.smithi076.stdout:7/858: fsync d8/de/fad 0 2024-03-23T15:44:57.060 INFO:tasks.workunit.client.0.smithi076.stdout:7/859: stat d8/d4e/d98/df6/d104/d24/de4/f114 0 2024-03-23T15:44:57.060 INFO:tasks.workunit.client.0.smithi076.stdout:7/860: chown d8/d4e/d98/df6/d104/d40/d5d/la8 3767092 1 2024-03-23T15:44:57.061 INFO:tasks.workunit.client.0.smithi076.stdout:7/861: creat d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/f122 x:0 0 0 2024-03-23T15:44:57.062 INFO:tasks.workunit.client.0.smithi076.stdout:3/523: sync 2024-03-23T15:44:57.062 INFO:tasks.workunit.client.0.smithi076.stdout:4/646: sync 2024-03-23T15:44:57.062 INFO:tasks.workunit.client.0.smithi076.stdout:0/711: sync 2024-03-23T15:44:57.062 INFO:tasks.workunit.client.0.smithi076.stdout:2/739: sync 2024-03-23T15:44:57.062 INFO:tasks.workunit.client.0.smithi076.stdout:1/396: sync 2024-03-23T15:44:57.063 INFO:tasks.workunit.client.0.smithi076.stdout:8/588: sync 2024-03-23T15:44:57.063 INFO:tasks.workunit.client.0.smithi076.stdout:6/469: link d1/d4/d10/d6b/d8d/f6a d1/d4/d10/d6b/fa1 0 2024-03-23T15:44:57.063 INFO:tasks.workunit.client.0.smithi076.stdout:2/740: getdents d1/d4/db/d15/d29/d7d/dbd 0 2024-03-23T15:44:57.064 INFO:tasks.workunit.client.0.smithi076.stdout:9/714: dread d1/d2f/d3f/f92 [0,4194304] 0 2024-03-23T15:44:57.064 INFO:tasks.workunit.client.0.smithi076.stdout:8/589: write d8/d2e/d30/fc3 [251068,7920] 0 2024-03-23T15:44:57.065 INFO:tasks.workunit.client.0.smithi076.stdout:7/862: creat d8/d4e/d98/df6/d104/d40/d5d/f123 x:0 0 0 2024-03-23T15:44:57.065 INFO:tasks.workunit.client.0.smithi076.stdout:7/863: dread - d8/de/d99/fbd zero size 2024-03-23T15:44:57.065 INFO:tasks.workunit.client.0.smithi076.stdout:7/864: chown d8/d4e/d98/cd5 117730 1 2024-03-23T15:44:57.065 INFO:tasks.workunit.client.0.smithi076.stdout:7/865: symlink d8/d4e/d98/df6/d104/d3a/d85/daf/dca/l124 0 2024-03-23T15:44:57.066 INFO:tasks.workunit.client.0.smithi076.stdout:8/590: write d8/d2e/d30/d69/f82 [391099,76609] 0 2024-03-23T15:44:57.066 INFO:tasks.workunit.client.0.smithi076.stdout:8/591: truncate d8/d4f/f53 744343 0 2024-03-23T15:44:57.067 INFO:tasks.workunit.client.0.smithi076.stdout:7/866: mkdir d8/d4e/d98/df6/d104/d18/dfd/d125 0 2024-03-23T15:44:57.067 INFO:tasks.workunit.client.0.smithi076.stdout:6/470: dread d1/dd/d2a/d23/f3a [0,4194304] 0 2024-03-23T15:44:57.068 INFO:tasks.workunit.client.0.smithi076.stdout:8/592: symlink d8/d83/lc8 0 2024-03-23T15:44:57.068 INFO:tasks.workunit.client.0.smithi076.stdout:7/867: creat d8/d4e/f126 x:0 0 0 2024-03-23T15:44:57.069 INFO:tasks.workunit.client.0.smithi076.stdout:6/471: rmdir d1/d4/d10 39 2024-03-23T15:44:57.071 INFO:tasks.workunit.client.0.smithi076.stdout:4/647: dread d8/d3a/d69/d73/d76/f77 [0,4194304] 0 2024-03-23T15:44:57.071 INFO:tasks.workunit.client.0.smithi076.stdout:4/648: write d8/d3a/d18/d75/fbb [987493,27540] 0 2024-03-23T15:44:57.079 INFO:tasks.workunit.client.0.smithi076.stdout:7/868: write f1 [6135992,120856] 0 2024-03-23T15:44:57.081 INFO:tasks.workunit.client.0.smithi076.stdout:7/869: creat d8/de/d99/dc4/df1/f127 x:0 0 0 2024-03-23T15:44:57.082 INFO:tasks.workunit.client.0.smithi076.stdout:7/870: write d8/d4e/d98/df6/d104/d3a/dbf/dc7/ff9 [4451044,53709] 0 2024-03-23T15:44:57.085 INFO:tasks.workunit.client.0.smithi076.stdout:1/397: dwrite d0/f2e [4194304,4194304] 0 2024-03-23T15:44:57.085 INFO:tasks.workunit.client.0.smithi076.stdout:1/398: fsync d0/d35/d40/d47/d17/f1d 0 2024-03-23T15:44:57.085 INFO:tasks.workunit.client.0.smithi076.stdout:8/593: dread d8/d26/d27/d56/f5b [0,4194304] 0 2024-03-23T15:44:57.087 INFO:tasks.workunit.client.0.smithi076.stdout:8/594: creat d8/d26/d27/da9/fc9 x:0 0 0 2024-03-23T15:44:57.088 INFO:tasks.workunit.client.0.smithi076.stdout:8/595: readlink d8/d26/d27/l8a 0 2024-03-23T15:44:57.089 INFO:tasks.workunit.client.0.smithi076.stdout:8/596: stat d8/d89/c3f 0 2024-03-23T15:44:57.090 INFO:tasks.workunit.client.0.smithi076.stdout:3/524: dwrite d5/d17/d1c/d32/d39/d59/d79/f8f [0,4194304] 0 2024-03-23T15:44:57.090 INFO:tasks.workunit.client.0.smithi076.stdout:3/525: stat d5/d17/d1c/d32/d8e/la8 0 2024-03-23T15:44:57.090 INFO:tasks.workunit.client.0.smithi076.stdout:3/526: chown d5 314073 1 2024-03-23T15:44:57.091 INFO:tasks.workunit.client.0.smithi076.stdout:3/527: write d5/d35/f36 [2092183,97332] 0 2024-03-23T15:44:57.092 INFO:tasks.workunit.client.0.smithi076.stdout:9/715: dwrite d1/f26 [0,4194304] 0 2024-03-23T15:44:57.092 INFO:tasks.workunit.client.0.smithi076.stdout:9/716: chown d1/d20/l2e 65 1 2024-03-23T15:44:57.092 INFO:tasks.workunit.client.0.smithi076.stdout:8/597: mknod d8/d6b/d9d/cca 0 2024-03-23T15:44:57.092 INFO:tasks.workunit.client.0.smithi076.stdout:7/871: read d8/d4e/d98/df6/d104/d18/f2b [5366927,59666] 0 2024-03-23T15:44:57.093 INFO:tasks.workunit.client.0.smithi076.stdout:8/598: creat d8/d89/db2/fcb x:0 0 0 2024-03-23T15:44:57.097 INFO:tasks.workunit.client.0.smithi076.stdout:0/712: dwrite d2/f7 [0,4194304] 0 2024-03-23T15:44:57.098 INFO:tasks.workunit.client.0.smithi076.stdout:4/649: dwrite d8/d3a/d18/d52/f6d [0,4194304] 0 2024-03-23T15:44:57.098 INFO:tasks.workunit.client.0.smithi076.stdout:0/713: creat d2/d8/d2d/fd9 x:0 0 0 2024-03-23T15:44:57.101 INFO:tasks.workunit.client.0.smithi076.stdout:0/714: symlink d2/d8/d52/lda 0 2024-03-23T15:44:57.101 INFO:tasks.workunit.client.0.smithi076.stdout:0/715: write d2/d8/f28 [878783,56968] 0 2024-03-23T15:44:57.103 INFO:tasks.workunit.client.0.smithi076.stdout:0/716: link d2/l2a d2/d1b/ldb 0 2024-03-23T15:44:57.103 INFO:tasks.workunit.client.0.smithi076.stdout:0/717: chown d2/d1b/c3c 376895 1 2024-03-23T15:44:57.112 INFO:tasks.workunit.client.0.smithi076.stdout:3/528: dwrite d5/d17/d1c/d31/d47/f7d [0,4194304] 0 2024-03-23T15:44:57.114 INFO:tasks.workunit.client.0.smithi076.stdout:9/717: dwrite d1/f26 [4194304,4194304] 0 2024-03-23T15:44:57.116 INFO:tasks.workunit.client.0.smithi076.stdout:9/718: dread d1/d1f/d29/d2d/d37/d47/fbe [0,4194304] 0 2024-03-23T15:44:57.117 INFO:tasks.workunit.client.0.smithi076.stdout:3/529: creat d5/d35/d6d/d7f/fae x:0 0 0 2024-03-23T15:44:57.119 INFO:tasks.workunit.client.0.smithi076.stdout:4/650: dwrite d8/d3a/d18/f98 [4194304,4194304] 0 2024-03-23T15:44:57.119 INFO:tasks.workunit.client.0.smithi076.stdout:9/719: creat d1/d2f/d3a/d64/fea x:0 0 0 2024-03-23T15:44:57.119 INFO:tasks.workunit.client.0.smithi076.stdout:9/720: fsync d1/d1f/d29/d2d/d37/f59 0 2024-03-23T15:44:57.119 INFO:tasks.workunit.client.0.smithi076.stdout:9/721: chown d1/d2f/d3a/d53/f61 292176 1 2024-03-23T15:44:57.119 INFO:tasks.workunit.client.0.smithi076.stdout:7/872: dwrite d8/de/f8a [0,4194304] 0 2024-03-23T15:44:57.119 INFO:tasks.workunit.client.0.smithi076.stdout:9/722: read d1/d1f/d29/d2d/d37/d47/fbe [1511,38668] 0 2024-03-23T15:44:57.121 INFO:tasks.workunit.client.0.smithi076.stdout:3/530: link d5/d17/d1c/f1e d5/d17/d1c/d32/d8e/faf 0 2024-03-23T15:44:57.121 INFO:tasks.workunit.client.0.smithi076.stdout:3/531: fdatasync d5/f7 0 2024-03-23T15:44:57.124 INFO:tasks.workunit.client.0.smithi076.stdout:0/718: dwrite d2/de/d10/d3a/d47/d69/d84/fa2 [0,4194304] 0 2024-03-23T15:44:57.124 INFO:tasks.workunit.client.0.smithi076.stdout:4/651: creat d8/d3a/d69/d73/d85/da9/fc2 x:0 0 0 2024-03-23T15:44:57.125 INFO:tasks.workunit.client.0.smithi076.stdout:1/399: link d0/d35/d40/d47/d17/c2a d0/d2/d70/c85 0 2024-03-23T15:44:57.126 INFO:tasks.workunit.client.0.smithi076.stdout:1/400: readlink d0/d35/d52/l4c 0 2024-03-23T15:44:57.126 INFO:tasks.workunit.client.0.smithi076.stdout:1/401: truncate d0/d2/f75 948024 0 2024-03-23T15:44:57.127 INFO:tasks.workunit.client.0.smithi076.stdout:9/723: mkdir d1/d2f/d3a/d74/deb 0 2024-03-23T15:44:57.131 INFO:tasks.workunit.client.0.smithi076.stdout:0/719: link d2/d8/l20 d2/de/d10/d3a/d47/d69/d84/dc5/ldc 0 2024-03-23T15:44:57.132 INFO:tasks.workunit.client.0.smithi076.stdout:7/873: dread d8/de/f81 [0,4194304] 0 2024-03-23T15:44:57.133 INFO:tasks.workunit.client.0.smithi076.stdout:7/874: getdents d8/d4e/d8d/dea 0 2024-03-23T15:44:57.133 INFO:tasks.workunit.client.0.smithi076.stdout:4/652: mkdir d8/d3a/dab/dc3 0 2024-03-23T15:44:57.133 INFO:tasks.workunit.client.0.smithi076.stdout:8/599: rename d8/c20 to d8/d26/d33/ccc 0 2024-03-23T15:44:57.134 INFO:tasks.workunit.client.0.smithi076.stdout:7/875: mknod d8/d4e/d98/df6/d104/d40/d88/d5a/dde/c128 0 2024-03-23T15:44:57.136 INFO:tasks.workunit.client.0.smithi076.stdout:8/600: dread d8/f16 [0,4194304] 0 2024-03-23T15:44:57.137 INFO:tasks.workunit.client.0.smithi076.stdout:9/724: link d1/d20/f25 d1/d2f/d9f/fec 0 2024-03-23T15:44:57.141 INFO:tasks.workunit.client.0.smithi076.stdout:3/532: dwrite d5/d17/d1c/d32/d39/d59/d79/f8f [0,4194304] 0 2024-03-23T15:44:57.142 INFO:tasks.workunit.client.0.smithi076.stdout:4/653: write d8/d9/f12 [3496807,9468] 0 2024-03-23T15:44:57.147 INFO:tasks.workunit.client.0.smithi076.stdout:1/402: mkdir d0/d35/d52/d86 0 2024-03-23T15:44:57.147 INFO:tasks.workunit.client.0.smithi076.stdout:1/403: fsync d0/d35/d40/d47/f6b 0 2024-03-23T15:44:57.147 INFO:tasks.workunit.client.0.smithi076.stdout:9/725: mknod d1/d2f/d3a/d64/ced 0 2024-03-23T15:44:57.148 INFO:tasks.workunit.client.0.smithi076.stdout:8/601: dwrite d8/d2e/f4a [0,4194304] 0 2024-03-23T15:44:57.149 INFO:tasks.workunit.client.0.smithi076.stdout:4/654: symlink d8/d3a/d18/d75/d8d/lc4 0 2024-03-23T15:44:57.149 INFO:tasks.workunit.client.0.smithi076.stdout:4/655: chown d8/d3a/d69/d73/c7d 4 1 2024-03-23T15:44:57.151 INFO:tasks.workunit.client.0.smithi076.stdout:7/876: unlink f1 0 2024-03-23T15:44:57.151 INFO:tasks.workunit.client.0.smithi076.stdout:0/720: rename d2/d1b/cbd to d2/de/d10/d12/cdd 0 2024-03-23T15:44:57.151 INFO:tasks.workunit.client.0.smithi076.stdout:7/877: write d8/d4e/d8d/f11c [772515,100999] 0 2024-03-23T15:44:57.151 INFO:tasks.workunit.client.0.smithi076.stdout:7/878: chown d8/d4e/d98/df6/d104/d3a/dbf 16623154 1 2024-03-23T15:44:57.152 INFO:tasks.workunit.client.0.smithi076.stdout:9/726: creat d1/d2f/d3a/fee x:0 0 0 2024-03-23T15:44:57.152 INFO:tasks.workunit.client.0.smithi076.stdout:9/727: chown d1/d2f/d3a/d74/fd4 98 1 2024-03-23T15:44:57.152 INFO:tasks.workunit.client.0.smithi076.stdout:9/728: stat d1/d2f/d3a/d64/f85 0 2024-03-23T15:44:57.155 INFO:tasks.workunit.client.0.smithi076.stdout:3/533: rename d5/ff to d5/d35/d4c/d5a/fb0 0 2024-03-23T15:44:57.156 INFO:tasks.workunit.client.0.smithi076.stdout:0/721: getdents d2/de/d10/d3a/d47/d69/d84/dc5 0 2024-03-23T15:44:57.160 INFO:tasks.workunit.client.0.smithi076.stdout:1/404: dwrite d0/d35/d40/d47/d17/d20/f3e [0,4194304] 0 2024-03-23T15:44:57.160 INFO:tasks.workunit.client.0.smithi076.stdout:8/602: rename d8/f50 to d8/d4f/fcd 0 2024-03-23T15:44:57.161 INFO:tasks.workunit.client.0.smithi076.stdout:0/722: mknod d2/d8/d32/cde 0 2024-03-23T15:44:57.162 INFO:tasks.workunit.client.0.smithi076.stdout:1/405: creat d0/d35/f87 x:0 0 0 2024-03-23T15:44:57.163 INFO:tasks.workunit.client.0.smithi076.stdout:8/603: truncate d8/d89/f5f 2127638 0 2024-03-23T15:44:57.164 INFO:tasks.workunit.client.0.smithi076.stdout:4/656: rename d8/d3a/d18/d75/faa to d8/d3a/d69/d73/d76/db4/fc5 0 2024-03-23T15:44:57.165 INFO:tasks.workunit.client.0.smithi076.stdout:1/406: symlink d0/d35/d40/l88 0 2024-03-23T15:44:57.167 INFO:tasks.workunit.client.0.smithi076.stdout:8/604: creat d8/d83/fce x:0 0 0 2024-03-23T15:44:57.169 INFO:tasks.workunit.client.0.smithi076.stdout:7/879: dwrite d8/d4e/d98/df6/d104/f15 [0,4194304] 0 2024-03-23T15:44:57.169 INFO:tasks.workunit.client.0.smithi076.stdout:7/880: write d8/d4e/d98/df6/d104/d40/f59 [522754,62120] 0 2024-03-23T15:44:57.169 INFO:tasks.workunit.client.0.smithi076.stdout:0/723: write d2/de/d10/d3a/d47/f59 [7573281,41215] 0 2024-03-23T15:44:57.170 INFO:tasks.workunit.client.0.smithi076.stdout:4/657: symlink d8/d3a/d18/d52/lc6 0 2024-03-23T15:44:57.170 INFO:tasks.workunit.client.0.smithi076.stdout:8/605: creat d8/d26/fcf x:0 0 0 2024-03-23T15:44:57.173 INFO:tasks.workunit.client.0.smithi076.stdout:3/534: dwrite d5/f73 [0,4194304] 0 2024-03-23T15:44:57.173 INFO:tasks.workunit.client.0.smithi076.stdout:3/535: readlink d5/d17/d1c/d32/l42 0 2024-03-23T15:44:57.173 INFO:tasks.workunit.client.0.smithi076.stdout:7/881: write d8/d4e/fd0 [3025713,126205] 0 2024-03-23T15:44:57.174 INFO:tasks.workunit.client.0.smithi076.stdout:0/724: rename d2/f7 to d2/de/d10/d12/da6/fdf 0 2024-03-23T15:44:57.174 INFO:tasks.workunit.client.0.smithi076.stdout:0/725: readlink d2/d8/d52/l5e 0 2024-03-23T15:44:57.175 INFO:tasks.workunit.client.0.smithi076.stdout:7/882: read d8/d4e/f87 [118501,96544] 0 2024-03-23T15:44:57.176 INFO:tasks.workunit.client.0.smithi076.stdout:4/658: link d8/d3a/d18/d75/d8d/fbd d8/d3a/d69/d73/d85/d88/fc7 0 2024-03-23T15:44:57.178 INFO:tasks.workunit.client.0.smithi076.stdout:8/606: mkdir d8/dd0 0 2024-03-23T15:44:57.180 INFO:tasks.workunit.client.0.smithi076.stdout:4/659: read d8/d3a/d18/d1b/f67 [2884927,73776] 0 2024-03-23T15:44:57.182 INFO:tasks.workunit.client.0.smithi076.stdout:1/407: dwrite d0/d35/d40/d47/d17/f54 [0,4194304] 0 2024-03-23T15:44:57.185 INFO:tasks.workunit.client.0.smithi076.stdout:3/536: creat d5/fb1 x:0 0 0 2024-03-23T15:44:57.190 INFO:tasks.workunit.client.0.smithi076.stdout:1/408: dread d0/d35/d40/d47/d17/f30 [0,4194304] 0 2024-03-23T15:44:57.194 INFO:tasks.workunit.client.0.smithi076.stdout:4/660: link d8/d3a/d18/d75/d8d/lc4 d8/d3a/d69/d73/d9a/lc8 0 2024-03-23T15:44:57.194 INFO:tasks.workunit.client.0.smithi076.stdout:4/661: write d8/fae [795149,2796] 0 2024-03-23T15:44:57.194 INFO:tasks.workunit.client.0.smithi076.stdout:4/662: write d8/d3a/d18/f2b [1689426,74589] 0 2024-03-23T15:44:57.196 INFO:tasks.workunit.client.0.smithi076.stdout:7/883: rmdir d8/d4e/d98/df6/d104/d3a/dbf 39 2024-03-23T15:44:57.196 INFO:tasks.workunit.client.0.smithi076.stdout:7/884: chown d8/d4e 30708 1 2024-03-23T15:44:57.198 INFO:tasks.workunit.client.0.smithi076.stdout:0/726: mkdir d2/de/de0 0 2024-03-23T15:44:57.199 INFO:tasks.workunit.client.0.smithi076.stdout:3/537: dwrite d5/d17/f29 [0,4194304] 0 2024-03-23T15:44:57.199 INFO:tasks.workunit.client.0.smithi076.stdout:3/538: dread - d5/d17/d1c/d32/d39/d59/f60 zero size 2024-03-23T15:44:57.200 INFO:tasks.workunit.client.0.smithi076.stdout:8/607: symlink d8/d83/ld1 0 2024-03-23T15:44:57.200 INFO:tasks.workunit.client.0.smithi076.stdout:8/608: read - d8/d89/db2/fc2 zero size 2024-03-23T15:44:57.200 INFO:tasks.workunit.client.0.smithi076.stdout:1/409: getdents d0/d3c/d6e 0 2024-03-23T15:44:57.205 INFO:tasks.workunit.client.0.smithi076.stdout:4/663: rmdir d8/d3a/dab 39 2024-03-23T15:44:57.206 INFO:tasks.workunit.client.0.smithi076.stdout:9/729: link d1/d2f/d3f/dc0/lc2 d1/d1f/d29/d7f/lef 0 2024-03-23T15:44:57.207 INFO:tasks.workunit.client.0.smithi076.stdout:5/744: sync 2024-03-23T15:44:57.207 INFO:tasks.workunit.client.0.smithi076.stdout:6/472: sync 2024-03-23T15:44:57.207 INFO:tasks.workunit.client.0.smithi076.stdout:2/741: sync 2024-03-23T15:44:57.208 INFO:tasks.workunit.client.0.smithi076.stdout:5/745: dread d1a/d4b/fb8 [0,4194304] 0 2024-03-23T15:44:57.208 INFO:tasks.workunit.client.0.smithi076.stdout:5/746: fdatasync d1a/f5a 0 2024-03-23T15:44:57.208 INFO:tasks.workunit.client.0.smithi076.stdout:8/609: symlink d8/d4f/ld2 0 2024-03-23T15:44:57.208 INFO:tasks.workunit.client.0.smithi076.stdout:8/610: write d8/d2e/d30/f96 [944979,14502] 0 2024-03-23T15:44:57.208 INFO:tasks.workunit.client.0.smithi076.stdout:6/473: creat d1/dd/df/d5b/fa2 x:0 0 0 2024-03-23T15:44:57.209 INFO:tasks.workunit.client.0.smithi076.stdout:2/742: dread d1/fd9 [0,4194304] 0 2024-03-23T15:44:57.209 INFO:tasks.workunit.client.0.smithi076.stdout:1/410: creat d0/d35/d82/f89 x:0 0 0 2024-03-23T15:44:57.211 INFO:tasks.workunit.client.0.smithi076.stdout:4/664: mkdir d8/d9/d1d/dc9 0 2024-03-23T15:44:57.211 INFO:tasks.workunit.client.0.smithi076.stdout:8/611: mkdir d8/d2e/d30/d69/dd3 0 2024-03-23T15:44:57.211 INFO:tasks.workunit.client.0.smithi076.stdout:4/665: truncate d8/d3a/d69/d73/d85/da9/fb6 523966 0 2024-03-23T15:44:57.214 INFO:tasks.workunit.client.0.smithi076.stdout:6/474: mknod d1/d4/d10/d39/ca3 0 2024-03-23T15:44:57.214 INFO:tasks.workunit.client.0.smithi076.stdout:9/730: dread d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:57.215 INFO:tasks.workunit.client.0.smithi076.stdout:7/885: symlink d8/d4e/d98/df6/d104/d24/de4/l129 0 2024-03-23T15:44:57.215 INFO:tasks.workunit.client.0.smithi076.stdout:7/886: readlink d8/lb0 0 2024-03-23T15:44:57.215 INFO:tasks.workunit.client.0.smithi076.stdout:0/727: dwrite d2/de/d10/d3a/d47/d69/d84/dc5/fd1 [4194304,4194304] 0 2024-03-23T15:44:57.216 INFO:tasks.workunit.client.0.smithi076.stdout:6/475: creat d1/dd/df/fa4 x:0 0 0 2024-03-23T15:44:57.220 INFO:tasks.workunit.client.0.smithi076.stdout:1/411: dread d0/d35/d40/d47/f4e [0,4194304] 0 2024-03-23T15:44:57.220 INFO:tasks.workunit.client.0.smithi076.stdout:4/666: link d8/d3a/d3b/f39 d8/d9/d1d/fca 0 2024-03-23T15:44:57.221 INFO:tasks.workunit.client.0.smithi076.stdout:7/887: rename d8/de/fcd to d8/d4e/d98/df6/d104/d18/dcc/f12a 0 2024-03-23T15:44:57.221 INFO:tasks.workunit.client.0.smithi076.stdout:7/888: write d8/d4e/d98/df6/d104/f21 [250116,78965] 0 2024-03-23T15:44:57.221 INFO:tasks.workunit.client.0.smithi076.stdout:9/731: symlink d1/d1f/d46/d83/lf0 0 2024-03-23T15:44:57.221 INFO:tasks.workunit.client.0.smithi076.stdout:6/476: unlink d1/dd/df/l33 0 2024-03-23T15:44:57.222 INFO:tasks.workunit.client.0.smithi076.stdout:8/612: write d8/d83/f95 [4703500,117546] 0 2024-03-23T15:44:57.222 INFO:tasks.workunit.client.0.smithi076.stdout:8/613: chown d8/d26/d27/d56/f5b 178942914 1 2024-03-23T15:44:57.222 INFO:tasks.workunit.client.0.smithi076.stdout:1/412: creat d0/d35/d40/d47/d17/f8a x:0 0 0 2024-03-23T15:44:57.223 INFO:tasks.workunit.client.0.smithi076.stdout:7/889: mknod d8/d4e/d98/df6/d104/d18/dcc/c12b 0 2024-03-23T15:44:57.224 INFO:tasks.workunit.client.0.smithi076.stdout:4/667: rename d8/d3a/d3b/f91 to d8/d3a/d18/d75/d8d/fcb 0 2024-03-23T15:44:57.224 INFO:tasks.workunit.client.0.smithi076.stdout:4/668: dread - d8/d9/d32/f90 zero size 2024-03-23T15:44:57.226 INFO:tasks.workunit.client.0.smithi076.stdout:9/732: creat d1/d2f/ff1 x:0 0 0 2024-03-23T15:44:57.226 INFO:tasks.workunit.client.0.smithi076.stdout:5/747: dwrite d1a/d4b/faa [0,4194304] 0 2024-03-23T15:44:57.227 INFO:tasks.workunit.client.0.smithi076.stdout:9/733: write d1/d1f/fbb [48919,36882] 0 2024-03-23T15:44:57.227 INFO:tasks.workunit.client.0.smithi076.stdout:9/734: dread - d1/d1f/d29/d2d/d37/d47/f52 zero size 2024-03-23T15:44:57.228 INFO:tasks.workunit.client.0.smithi076.stdout:6/477: link d1/d81/l89 d1/d81/la5 0 2024-03-23T15:44:57.231 INFO:tasks.workunit.client.0.smithi076.stdout:1/413: mkdir d0/d2/d70/d7a/d8b 0 2024-03-23T15:44:57.231 INFO:tasks.workunit.client.0.smithi076.stdout:1/414: chown d0/d3c/d6e/c84 282369 1 2024-03-23T15:44:57.231 INFO:tasks.workunit.client.0.smithi076.stdout:1/415: fdatasync d0/d3c/f6c 0 2024-03-23T15:44:57.234 INFO:tasks.workunit.client.0.smithi076.stdout:4/669: mknod d8/d3a/d18/d75/d8d/db0/ccc 0 2024-03-23T15:44:57.234 INFO:tasks.workunit.client.0.smithi076.stdout:4/670: write d8/d41/d97/fb9 [1553544,104965] 0 2024-03-23T15:44:57.234 INFO:tasks.workunit.client.0.smithi076.stdout:4/671: write d8/d3a/d18/d1b/fa5 [1684870,114337] 0 2024-03-23T15:44:57.234 INFO:tasks.workunit.client.0.smithi076.stdout:5/748: creat d1a/f102 x:0 0 0 2024-03-23T15:44:57.235 INFO:tasks.workunit.client.0.smithi076.stdout:0/728: dwrite d2/d8/d32/f48 [0,4194304] 0 2024-03-23T15:44:57.235 INFO:tasks.workunit.client.0.smithi076.stdout:2/743: dwrite d1/d4/db/d3e/f77 [4194304,4194304] 0 2024-03-23T15:44:57.235 INFO:tasks.workunit.client.0.smithi076.stdout:2/744: write d1/d2e/ff7 [213624,104654] 0 2024-03-23T15:44:57.235 INFO:tasks.workunit.client.0.smithi076.stdout:8/614: rename d8/d26/d27/l85 to d8/d4f/ld4 0 2024-03-23T15:44:57.236 INFO:tasks.workunit.client.0.smithi076.stdout:5/749: truncate d1a/d33/f35 967406 0 2024-03-23T15:44:57.238 INFO:tasks.workunit.client.0.smithi076.stdout:8/615: mkdir d8/d89/db2/dd5 0 2024-03-23T15:44:57.239 INFO:tasks.workunit.client.0.smithi076.stdout:7/890: rmdir d8/d4e/d98/df6/d104/d3a/d85 39 2024-03-23T15:44:57.240 INFO:tasks.workunit.client.0.smithi076.stdout:5/750: link d1a/d33/d3a/l9c d1a/d33/l103 0 2024-03-23T15:44:57.240 INFO:tasks.workunit.client.0.smithi076.stdout:2/745: write d1/d4/db/d15/d29/f61 [424881,71903] 0 2024-03-23T15:44:57.241 INFO:tasks.workunit.client.0.smithi076.stdout:1/416: rename d0/d35/d40/d47/d17/d20/d7b to d0/d35/d40/d47/d8c 0 2024-03-23T15:44:57.241 INFO:tasks.workunit.client.0.smithi076.stdout:1/417: dread - d0/d3c/d6e/f80 zero size 2024-03-23T15:44:57.242 INFO:tasks.workunit.client.0.smithi076.stdout:2/746: mkdir d1/d2e/dd1/df6/dfd 0 2024-03-23T15:44:57.242 INFO:tasks.workunit.client.0.smithi076.stdout:5/751: symlink d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/l104 0 2024-03-23T15:44:57.242 INFO:tasks.workunit.client.0.smithi076.stdout:7/891: dread d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/f51 [0,4194304] 0 2024-03-23T15:44:57.242 INFO:tasks.workunit.client.0.smithi076.stdout:5/752: write d1a/d33/d3a/d67/da8/dc4/d5f/d8f/fc5 [682929,20772] 0 2024-03-23T15:44:57.242 INFO:tasks.workunit.client.0.smithi076.stdout:7/892: write d8/fec [64644,32643] 0 2024-03-23T15:44:57.242 INFO:tasks.workunit.client.0.smithi076.stdout:5/753: stat d1a/d33/d3a/c4f 0 2024-03-23T15:44:57.243 INFO:tasks.workunit.client.0.smithi076.stdout:7/893: write d8/d4e/d98/df6/d104/d3a/f8b [93209,36505] 0 2024-03-23T15:44:57.243 INFO:tasks.workunit.client.0.smithi076.stdout:6/478: creat d1/d4/d10/d6b/d72/fa6 x:0 0 0 2024-03-23T15:44:57.245 INFO:tasks.workunit.client.0.smithi076.stdout:7/894: dread d8/d4e/d98/df6/d104/d40/f59 [0,4194304] 0 2024-03-23T15:44:57.245 INFO:tasks.workunit.client.0.smithi076.stdout:7/895: read d8/d4e/d98/df6/d104/d40/d5d/f3e [542349,7078] 0 2024-03-23T15:44:57.245 INFO:tasks.workunit.client.0.smithi076.stdout:5/754: mkdir d1a/d33/dc3/d105 0 2024-03-23T15:44:57.245 INFO:tasks.workunit.client.0.smithi076.stdout:7/896: write d8/d4e/d98/df6/d104/d3a/dd2/f11d [548077,123340] 0 2024-03-23T15:44:57.245 INFO:tasks.workunit.client.0.smithi076.stdout:5/755: write d1a/d4b/fb8 [1301397,106617] 0 2024-03-23T15:44:57.245 INFO:tasks.workunit.client.0.smithi076.stdout:5/756: truncate d1a/d33/dc3/fc7 841073 0 2024-03-23T15:44:57.245 INFO:tasks.workunit.client.0.smithi076.stdout:7/897: rmdir d8/d4e/d98/df6/d104/d40/d88/d5a/dde 39 2024-03-23T15:44:57.246 INFO:tasks.workunit.client.0.smithi076.stdout:7/898: write d8/d4e/d98/df6/d104/d40/d5d/d3c/f73 [4510670,45715] 0 2024-03-23T15:44:57.246 INFO:tasks.workunit.client.0.smithi076.stdout:6/479: dread d1/f85 [0,4194304] 0 2024-03-23T15:44:57.248 INFO:tasks.workunit.client.0.smithi076.stdout:4/672: symlink d8/d9/lcd 0 2024-03-23T15:44:57.250 INFO:tasks.workunit.client.0.smithi076.stdout:8/616: creat d8/d26/d27/d56/fd6 x:0 0 0 2024-03-23T15:44:57.250 INFO:tasks.workunit.client.0.smithi076.stdout:1/418: dread d0/d35/d40/d47/d17/f1d [0,4194304] 0 2024-03-23T15:44:57.251 INFO:tasks.workunit.client.0.smithi076.stdout:1/419: chown d0/d35/f6f 3 1 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:1/420: fsync d0/f34 0 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:1/421: chown d0/d35/d40/d47/d17/d20/f33 6651655 1 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:9/735: mknod d1/d1f/d46/d83/dad/cf2 0 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:9/736: chown d1/d2f/d3a/d64/f85 2454317 1 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:9/737: truncate d1/d2f/fbc 933717 0 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:9/738: chown d1/d1f/d46/d83/cdb 792129396 1 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:9/739: fdatasync d1/f35 0 2024-03-23T15:44:57.252 INFO:tasks.workunit.client.0.smithi076.stdout:9/740: readlink d1/d1f/d46/d83/dad/l6d 0 2024-03-23T15:44:57.253 INFO:tasks.workunit.client.0.smithi076.stdout:1/422: getdents d0/d35/d40/d47/d8c 0 2024-03-23T15:44:57.253 INFO:tasks.workunit.client.0.smithi076.stdout:1/423: readlink d0/d2/l78 0 2024-03-23T15:44:57.253 INFO:tasks.workunit.client.0.smithi076.stdout:0/729: rename d2/c5 to d2/de/d10/ce1 0 2024-03-23T15:44:57.257 INFO:tasks.workunit.client.0.smithi076.stdout:2/747: rename d1/d2e/l6b to d1/d4/db/d15/lfe 0 2024-03-23T15:44:57.257 INFO:tasks.workunit.client.0.smithi076.stdout:0/730: link d2/de/d10/d3a/fce d2/d8/d52/fe2 0 2024-03-23T15:44:57.257 INFO:tasks.workunit.client.0.smithi076.stdout:1/424: dread d0/d35/d40/d47/d17/d20/f46 [0,4194304] 0 2024-03-23T15:44:57.258 INFO:tasks.workunit.client.0.smithi076.stdout:4/673: dread d8/d3a/d18/d1b/fa5 [0,4194304] 0 2024-03-23T15:44:57.258 INFO:tasks.workunit.client.0.smithi076.stdout:4/674: chown d8/d41/c9b 6 1 2024-03-23T15:44:57.259 INFO:tasks.workunit.client.0.smithi076.stdout:4/675: stat d8/d3a/l7c 0 2024-03-23T15:44:57.259 INFO:tasks.workunit.client.0.smithi076.stdout:4/676: write d8/f56 [966027,35131] 0 2024-03-23T15:44:57.260 INFO:tasks.workunit.client.0.smithi076.stdout:9/741: write d1/d1f/d46/d83/da7/fe5 [1841932,59288] 0 2024-03-23T15:44:57.260 INFO:tasks.workunit.client.0.smithi076.stdout:0/731: mkdir d2/de/d10/d3a/d47/de3 0 2024-03-23T15:44:57.262 INFO:tasks.workunit.client.0.smithi076.stdout:1/425: readlink d0/d35/d52/l3d 0 2024-03-23T15:44:57.262 INFO:tasks.workunit.client.0.smithi076.stdout:1/426: write d0/d35/f60 [406032,81078] 0 2024-03-23T15:44:57.262 INFO:tasks.workunit.client.0.smithi076.stdout:1/427: fsync d0/d35/d40/d47/f6b 0 2024-03-23T15:44:57.262 INFO:tasks.workunit.client.0.smithi076.stdout:0/732: dread d2/de/d10/d3a/d47/fab [0,4194304] 0 2024-03-23T15:44:57.263 INFO:tasks.workunit.client.0.smithi076.stdout:0/733: dread d2/de/f95 [0,4194304] 0 2024-03-23T15:44:57.263 INFO:tasks.workunit.client.0.smithi076.stdout:1/428: mknod d0/d35/d40/d47/d55/c8d 0 2024-03-23T15:44:57.265 INFO:tasks.workunit.client.0.smithi076.stdout:4/677: rename d8/d3a/d69/d73/fa7 to d8/d3a/d18/d75/d8d/db0/fce 0 2024-03-23T15:44:57.266 INFO:tasks.workunit.client.0.smithi076.stdout:2/748: mkdir d1/d4/d64/d1d/dbc/dff 0 2024-03-23T15:44:57.268 INFO:tasks.workunit.client.0.smithi076.stdout:1/429: chown d0/d35/d52/c5 63 1 2024-03-23T15:44:57.269 INFO:tasks.workunit.client.0.smithi076.stdout:9/742: rmdir d1/d2f/d3a/d64 39 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:5/757: dwrite d1a/d33/dc3/da6/fd9 [0,4194304] 0 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:4/678: creat d8/d3a/d69/d73/d85/fcf x:0 0 0 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:4/679: dread - d8/d3a/d3b/f94 zero size 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:0/734: link d2/de/d10/d3a/d47/d69/d84/dc5/fd1 d2/de/d10/d3a/d47/d69/da3/fe4 0 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:0/735: truncate d2/de/d10/fd5 759469 0 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:0/736: chown d2/de/d10/fb0 26497030 1 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:0/737: dread - d2/d1b/d77/fd6 zero size 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:0/738: chown d2/de/d10/d3a/d47/la1 7323 1 2024-03-23T15:44:57.272 INFO:tasks.workunit.client.0.smithi076.stdout:0/739: write d2/de/d10/f68 [1029251,96331] 0 2024-03-23T15:44:57.273 INFO:tasks.workunit.client.0.smithi076.stdout:7/899: dwrite d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/fdc [0,4194304] 0 2024-03-23T15:44:57.277 INFO:tasks.workunit.client.0.smithi076.stdout:6/480: dwrite d1/d4/d10/d6b/d8d/f6a [4194304,4194304] 0 2024-03-23T15:44:57.280 INFO:tasks.workunit.client.0.smithi076.stdout:6/481: creat d1/dd/df/fa7 x:0 0 0 2024-03-23T15:44:57.283 INFO:tasks.workunit.client.0.smithi076.stdout:6/482: symlink d1/d4/d10/d6b/la8 0 2024-03-23T15:44:57.283 INFO:tasks.workunit.client.0.smithi076.stdout:6/483: chown d1/d2/l7f 1 1 2024-03-23T15:44:57.286 INFO:tasks.workunit.client.0.smithi076.stdout:6/484: creat d1/d4/d10/fa9 x:0 0 0 2024-03-23T15:44:57.286 INFO:tasks.workunit.client.0.smithi076.stdout:0/740: dread d2/de/d10/d3a/d47/f59 [0,4194304] 0 2024-03-23T15:44:57.286 INFO:tasks.workunit.client.0.smithi076.stdout:0/741: write d2/d9a/fd0 [447863,57956] 0 2024-03-23T15:44:57.287 INFO:tasks.workunit.client.0.smithi076.stdout:2/749: dwrite d1/d4/f1e [0,4194304] 0 2024-03-23T15:44:57.288 INFO:tasks.workunit.client.0.smithi076.stdout:0/742: rmdir d2/d8/d2d 39 2024-03-23T15:44:57.292 INFO:tasks.workunit.client.0.smithi076.stdout:0/743: symlink d2/de/d10/d3a/d47/d69/le5 0 2024-03-23T15:44:57.293 INFO:tasks.workunit.client.0.smithi076.stdout:1/430: dwrite d0/f5e [0,4194304] 0 2024-03-23T15:44:57.293 INFO:tasks.workunit.client.0.smithi076.stdout:0/744: mknod d2/de/d71/ce6 0 2024-03-23T15:44:57.294 INFO:tasks.workunit.client.0.smithi076.stdout:9/743: dwrite d1/d20/f25 [0,4194304] 0 2024-03-23T15:44:57.294 INFO:tasks.workunit.client.0.smithi076.stdout:9/744: write d1/d1f/d29/d2d/f86 [374224,116379] 0 2024-03-23T15:44:57.296 INFO:tasks.workunit.client.0.smithi076.stdout:7/900: dwrite d8/d4e/d98/df6/d104/d40/d5d/d3c/fee [0,4194304] 0 2024-03-23T15:44:57.297 INFO:tasks.workunit.client.0.smithi076.stdout:7/901: read d8/f3f [2705881,79291] 0 2024-03-23T15:44:57.300 INFO:tasks.workunit.client.0.smithi076.stdout:7/902: mkdir d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/d12c 0 2024-03-23T15:44:57.300 INFO:tasks.workunit.client.0.smithi076.stdout:7/903: dread - d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/f122 zero size 2024-03-23T15:44:57.301 INFO:tasks.workunit.client.0.smithi076.stdout:2/750: dread d1/d4/d64/d82/d7f/f8c [0,4194304] 0 2024-03-23T15:44:57.301 INFO:tasks.workunit.client.0.smithi076.stdout:7/904: getdents d8/d4e/d98/df6/d104/d40/d88/d5a 0 2024-03-23T15:44:57.302 INFO:tasks.workunit.client.0.smithi076.stdout:4/680: dwrite d8/d3a/d18/f2b [0,4194304] 0 2024-03-23T15:44:57.303 INFO:tasks.workunit.client.0.smithi076.stdout:2/751: mknod d1/d4/db/dc8/c100 0 2024-03-23T15:44:57.303 INFO:tasks.workunit.client.0.smithi076.stdout:8/617: dwrite d8/f19 [4194304,4194304] 0 2024-03-23T15:44:57.303 INFO:tasks.workunit.client.0.smithi076.stdout:8/618: write d8/f7a [331544,111036] 0 2024-03-23T15:44:57.303 INFO:tasks.workunit.client.0.smithi076.stdout:5/758: dwrite f18 [4194304,4194304] 0 2024-03-23T15:44:57.307 INFO:tasks.workunit.client.0.smithi076.stdout:0/745: fsync d2/d8/d52/fe2 0 2024-03-23T15:44:57.307 INFO:tasks.workunit.client.0.smithi076.stdout:0/746: chown d2/d9a/fbc 2 1 2024-03-23T15:44:57.307 INFO:tasks.workunit.client.0.smithi076.stdout:7/905: link d8/de/c9d d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/c12d 0 2024-03-23T15:44:57.307 INFO:tasks.workunit.client.0.smithi076.stdout:5/759: truncate d1a/f5a 267166 0 2024-03-23T15:44:57.308 INFO:tasks.workunit.client.0.smithi076.stdout:7/906: write d8/d4e/d98/df6/d104/d40/d5d/f3e [2248881,42435] 0 2024-03-23T15:44:57.310 INFO:tasks.workunit.client.0.smithi076.stdout:5/760: mkdir d1a/d2c/d106 0 2024-03-23T15:44:57.313 INFO:tasks.workunit.client.0.smithi076.stdout:5/761: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/f107 x:0 0 0 2024-03-23T15:44:57.313 INFO:tasks.workunit.client.0.smithi076.stdout:5/762: dread - d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/ff5 zero size 2024-03-23T15:44:57.313 INFO:tasks.workunit.client.0.smithi076.stdout:5/763: chown d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fd5 2 1 2024-03-23T15:44:57.313 INFO:tasks.workunit.client.0.smithi076.stdout:5/764: write d1a/d33/f35 [703851,121138] 0 2024-03-23T15:44:57.313 INFO:tasks.workunit.client.0.smithi076.stdout:5/765: truncate d1a/f102 759327 0 2024-03-23T15:44:57.313 INFO:tasks.workunit.client.0.smithi076.stdout:8/619: link d8/f57 d8/d26/d27/da9/fd7 0 2024-03-23T15:44:57.314 INFO:tasks.workunit.client.0.smithi076.stdout:1/431: dwrite d0/d35/d40/d47/d17/f49 [4194304,4194304] 0 2024-03-23T15:44:57.315 INFO:tasks.workunit.client.0.smithi076.stdout:5/766: dread d1a/d33/f73 [0,4194304] 0 2024-03-23T15:44:57.316 INFO:tasks.workunit.client.0.smithi076.stdout:4/681: dread d8/d3a/d18/d52/f64 [0,4194304] 0 2024-03-23T15:44:57.316 INFO:tasks.workunit.client.0.smithi076.stdout:4/682: read d8/f6a [778482,59552] 0 2024-03-23T15:44:57.316 INFO:tasks.workunit.client.0.smithi076.stdout:0/747: creat d2/d9a/db8/fe7 x:0 0 0 2024-03-23T15:44:57.320 INFO:tasks.workunit.client.0.smithi076.stdout:1/432: mknod d0/d2/d1f/c8e 0 2024-03-23T15:44:57.320 INFO:tasks.workunit.client.0.smithi076.stdout:2/752: dwrite d1/d2e/d8a/f94 [0,4194304] 0 2024-03-23T15:44:57.324 INFO:tasks.workunit.client.0.smithi076.stdout:0/748: dread d2/d8/d52/f5f [0,4194304] 0 2024-03-23T15:44:57.324 INFO:tasks.workunit.client.0.smithi076.stdout:0/749: chown d2/de/d10/d3a/d47/d69/d84/f93 1613626 1 2024-03-23T15:44:57.324 INFO:tasks.workunit.client.0.smithi076.stdout:4/683: write d8/d3a/d18/f2b [2092034,109342] 0 2024-03-23T15:44:57.324 INFO:tasks.workunit.client.0.smithi076.stdout:4/684: chown d8/d3a/d18/d52/lc6 63951 1 2024-03-23T15:44:57.331 INFO:tasks.workunit.client.0.smithi076.stdout:1/433: mkdir d0/d35/d40/d47/d74/d83/d8f 0 2024-03-23T15:44:57.333 INFO:tasks.workunit.client.0.smithi076.stdout:5/767: dwrite d1a/d33/dc3/fd6 [0,4194304] 0 2024-03-23T15:44:57.335 INFO:tasks.workunit.client.0.smithi076.stdout:7/907: dwrite d8/d4e/d98/df6/d104/d40/d88/f2c [4194304,4194304] 0 2024-03-23T15:44:57.335 INFO:tasks.workunit.client.0.smithi076.stdout:5/768: getdents d1a/d33/d36 0 2024-03-23T15:44:57.339 INFO:tasks.workunit.client.0.smithi076.stdout:6/485: rename d1/d4/d10/f34 to d1/dd/d2a/d23/faa 0 2024-03-23T15:44:57.342 INFO:tasks.workunit.client.0.smithi076.stdout:1/434: dread d0/d35/d40/d47/d17/f49 [4194304,4194304] 0 2024-03-23T15:44:57.343 INFO:tasks.workunit.client.0.smithi076.stdout:6/486: mknod d1/d2/cab 0 2024-03-23T15:44:57.345 INFO:tasks.workunit.client.0.smithi076.stdout:6/487: mkdir d1/dd/df/dac 0 2024-03-23T15:44:57.345 INFO:tasks.workunit.client.0.smithi076.stdout:2/753: creat d1/d4/d64/d1d/dbc/de6/f101 x:0 0 0 2024-03-23T15:44:57.345 INFO:tasks.workunit.client.0.smithi076.stdout:2/754: write d1/d4/d64/d82/fa6 [994233,31389] 0 2024-03-23T15:44:57.347 INFO:tasks.workunit.client.0.smithi076.stdout:5/769: dwrite d1a/d4b/fb4 [0,4194304] 0 2024-03-23T15:44:57.348 INFO:tasks.workunit.client.0.smithi076.stdout:0/750: symlink d2/d1b/le8 0 2024-03-23T15:44:57.348 INFO:tasks.workunit.client.0.smithi076.stdout:4/685: symlink d8/d3a/d69/d73/d85/da9/ld0 0 2024-03-23T15:44:57.348 INFO:tasks.workunit.client.0.smithi076.stdout:4/686: write d8/d3a/f34 [604921,72347] 0 2024-03-23T15:44:57.351 INFO:tasks.workunit.client.0.smithi076.stdout:2/755: read d1/d4/db/d15/f21 [2920161,34521] 0 2024-03-23T15:44:57.352 INFO:tasks.workunit.client.0.smithi076.stdout:9/745: rename d1/ca to d1/d2f/d9f/cf3 0 2024-03-23T15:44:57.352 INFO:tasks.workunit.client.0.smithi076.stdout:2/756: rmdir d1/d4/db 39 2024-03-23T15:44:57.352 INFO:tasks.workunit.client.0.smithi076.stdout:2/757: getdents d1/d4/db/d15/d29 0 2024-03-23T15:44:57.353 INFO:tasks.workunit.client.0.smithi076.stdout:2/758: write d1/d4/d64/d1d/dbc/fdf [628322,60017] 0 2024-03-23T15:44:57.353 INFO:tasks.workunit.client.0.smithi076.stdout:6/488: dread d1/d2/f22 [0,4194304] 0 2024-03-23T15:44:57.353 INFO:tasks.workunit.client.0.smithi076.stdout:2/759: fsync d1/d4/f96 0 2024-03-23T15:44:57.354 INFO:tasks.workunit.client.0.smithi076.stdout:9/746: creat d1/d1f/d29/d2d/d37/d47/ff4 x:0 0 0 2024-03-23T15:44:57.354 INFO:tasks.workunit.client.0.smithi076.stdout:0/751: dread d2/d1b/f42 [0,4194304] 0 2024-03-23T15:44:57.354 INFO:tasks.workunit.client.0.smithi076.stdout:1/435: link d0/d3c/f6c d0/d35/d40/f90 0 2024-03-23T15:44:57.354 INFO:tasks.workunit.client.0.smithi076.stdout:9/747: symlink d1/d1f/d46/d4a/d5a/lf5 0 2024-03-23T15:44:57.355 INFO:tasks.workunit.client.0.smithi076.stdout:9/748: write d1/d1f/d29/d2d/d37/fb0 [700204,24674] 0 2024-03-23T15:44:57.356 INFO:tasks.workunit.client.0.smithi076.stdout:9/749: creat d1/d1f/d29/d7f/ff6 x:0 0 0 2024-03-23T15:44:57.358 INFO:tasks.workunit.client.0.smithi076.stdout:8/620: rename d8/d4f/f53 to d8/d4f/d80/db7/fd8 0 2024-03-23T15:44:57.358 INFO:tasks.workunit.client.0.smithi076.stdout:7/908: mknod d8/d4e/d98/df6/d104/d3a/c12e 0 2024-03-23T15:44:57.361 INFO:tasks.workunit.client.0.smithi076.stdout:8/621: mkdir d8/d89/db2/dd9 0 2024-03-23T15:44:57.362 INFO:tasks.workunit.client.0.smithi076.stdout:1/436: dread d0/d35/d40/d47/d17/f38 [0,4194304] 0 2024-03-23T15:44:57.362 INFO:tasks.workunit.client.0.smithi076.stdout:7/909: mkdir d8/de/de8/d12f 0 2024-03-23T15:44:57.362 INFO:tasks.workunit.client.0.smithi076.stdout:3/539: sync 2024-03-23T15:44:57.362 INFO:tasks.workunit.client.0.smithi076.stdout:3/540: readlink d5/d35/d4c/d5a/l4b 0 2024-03-23T15:44:57.362 INFO:tasks.workunit.client.0.smithi076.stdout:6/489: symlink d1/d81/lad 0 2024-03-23T15:44:57.363 INFO:tasks.workunit.client.0.smithi076.stdout:0/752: unlink d2/de/d10/d43/d56/lcf 0 2024-03-23T15:44:57.363 INFO:tasks.workunit.client.0.smithi076.stdout:4/687: mknod d8/d3a/d3b/cd1 0 2024-03-23T15:44:57.363 INFO:tasks.workunit.client.0.smithi076.stdout:6/490: creat d1/d4/d10/d6b/d8d/d60/fae x:0 0 0 2024-03-23T15:44:57.364 INFO:tasks.workunit.client.0.smithi076.stdout:5/770: rename d1a/c59 to d1a/d33/c108 0 2024-03-23T15:44:57.364 INFO:tasks.workunit.client.0.smithi076.stdout:3/541: symlink d5/d17/d1c/d31/d61/d92/lb2 0 2024-03-23T15:44:57.364 INFO:tasks.workunit.client.0.smithi076.stdout:2/760: rename d1/d4/d64/f26 to d1/d2e/dd1/f102 0 2024-03-23T15:44:57.366 INFO:tasks.workunit.client.0.smithi076.stdout:2/761: rmdir d1/d2e/dd1/df6 39 2024-03-23T15:44:57.366 INFO:tasks.workunit.client.0.smithi076.stdout:2/762: truncate d1/d2e/d8a/fd6 4635596 0 2024-03-23T15:44:57.366 INFO:tasks.workunit.client.0.smithi076.stdout:2/763: getdents d1/d4/db/d15/d29/d7d/dbd/de3 0 2024-03-23T15:44:57.366 INFO:tasks.workunit.client.0.smithi076.stdout:6/491: unlink d1/f2b 0 2024-03-23T15:44:57.367 INFO:tasks.workunit.client.0.smithi076.stdout:5/771: mknod d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/c109 0 2024-03-23T15:44:57.368 INFO:tasks.workunit.client.0.smithi076.stdout:8/622: dread d8/d2e/d3e/f8d [0,4194304] 0 2024-03-23T15:44:57.368 INFO:tasks.workunit.client.0.smithi076.stdout:6/492: symlink d1/d2/laf 0 2024-03-23T15:44:57.370 INFO:tasks.workunit.client.0.smithi076.stdout:9/750: write d1/f19 [5488142,93396] 0 2024-03-23T15:44:57.370 INFO:tasks.workunit.client.0.smithi076.stdout:9/751: chown d1/d2f/d3a/d64/lc7 100 1 2024-03-23T15:44:57.374 INFO:tasks.workunit.client.0.smithi076.stdout:1/437: rename d0/d35/d52 to d0/d2/d91 0 2024-03-23T15:44:57.375 INFO:tasks.workunit.client.0.smithi076.stdout:2/764: read d1/d4/d52/f70 [2136103,90837] 0 2024-03-23T15:44:57.376 INFO:tasks.workunit.client.0.smithi076.stdout:1/438: truncate d0/d2/d1f/f43 2136076 0 2024-03-23T15:44:57.378 INFO:tasks.workunit.client.0.smithi076.stdout:1/439: truncate d0/d35/f64 3793729 0 2024-03-23T15:44:57.378 INFO:tasks.workunit.client.0.smithi076.stdout:1/440: chown d0/d35/d40/l6a 13206535 1 2024-03-23T15:44:57.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/493: dread d1/dd/df/d5b/d62/f6e [0,4194304] 0 2024-03-23T15:44:57.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/494: dread - d1/d4/d10/d39/f76 zero size 2024-03-23T15:44:57.380 INFO:tasks.workunit.client.0.smithi076.stdout:6/495: chown d1 50949180 1 2024-03-23T15:44:57.381 INFO:tasks.workunit.client.0.smithi076.stdout:4/688: dwrite d8/f49 [8388608,4194304] 0 2024-03-23T15:44:57.382 INFO:tasks.workunit.client.0.smithi076.stdout:7/910: dread d8/fd [0,4194304] 0 2024-03-23T15:44:57.382 INFO:tasks.workunit.client.0.smithi076.stdout:5/772: link d1a/d33/dc3/lcd d1a/d33/d3a/d67/da8/l10a 0 2024-03-23T15:44:57.393 INFO:tasks.workunit.client.0.smithi076.stdout:2/765: dwrite d1/d4/db/f5b [0,4194304] 0 2024-03-23T15:44:57.394 INFO:tasks.workunit.client.0.smithi076.stdout:4/689: mkdir d8/d3a/d69/d73/d85/dd2 0 2024-03-23T15:44:57.394 INFO:tasks.workunit.client.0.smithi076.stdout:4/690: stat d8/d3a/d3b/f94 0 2024-03-23T15:44:57.394 INFO:tasks.workunit.client.0.smithi076.stdout:4/691: dread - d8/d3a/d69/d73/d85/fcf zero size 2024-03-23T15:44:57.398 INFO:tasks.workunit.client.0.smithi076.stdout:2/766: getdents d1/d4/d64/d1b/dd4 0 2024-03-23T15:44:57.404 INFO:tasks.workunit.client.0.smithi076.stdout:6/496: write d1/d4/d10/d6b/fa1 [4157938,119599] 0 2024-03-23T15:44:57.404 INFO:tasks.workunit.client.0.smithi076.stdout:4/692: creat d8/d9/fd3 x:0 0 0 2024-03-23T15:44:57.405 INFO:tasks.workunit.client.0.smithi076.stdout:8/623: dwrite d8/fe [0,4194304] 0 2024-03-23T15:44:57.405 INFO:tasks.workunit.client.0.smithi076.stdout:7/911: dread d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/f93 [0,4194304] 0 2024-03-23T15:44:57.405 INFO:tasks.workunit.client.0.smithi076.stdout:7/912: dread - d8/d4e/d98/f10b zero size 2024-03-23T15:44:57.405 INFO:tasks.workunit.client.0.smithi076.stdout:4/693: getdents d8/d3a/d69/d73/d76/db4 0 2024-03-23T15:44:57.406 INFO:tasks.workunit.client.0.smithi076.stdout:9/752: dwrite d1/d1f/d46/d83/da7/fe5 [0,4194304] 0 2024-03-23T15:44:57.406 INFO:tasks.workunit.client.0.smithi076.stdout:7/913: creat d8/d4e/d98/df6/d104/d40/d5d/d3c/d46/f130 x:0 0 0 2024-03-23T15:44:57.407 INFO:tasks.workunit.client.0.smithi076.stdout:4/694: creat d8/d3a/fd4 x:0 0 0 2024-03-23T15:44:57.414 INFO:tasks.workunit.client.0.smithi076.stdout:4/695: getdents d8/d3a/dab 0 2024-03-23T15:44:57.415 INFO:tasks.workunit.client.0.smithi076.stdout:6/497: dread d1/d2/f71 [0,4194304] 0 2024-03-23T15:44:57.417 INFO:tasks.workunit.client.0.smithi076.stdout:6/498: write d1/d2/f6d [9202462,122714] 0 2024-03-23T15:44:57.417 INFO:tasks.workunit.client.0.smithi076.stdout:4/696: symlink d8/d3a/d69/d73/d85/ld5 0 2024-03-23T15:44:57.417 INFO:tasks.workunit.client.0.smithi076.stdout:8/624: write d8/d2e/d3e/f49 [817180,100104] 0 2024-03-23T15:44:57.417 INFO:tasks.workunit.client.0.smithi076.stdout:9/753: mknod d1/d1f/d46/d83/cf7 0 2024-03-23T15:44:57.417 INFO:tasks.workunit.client.0.smithi076.stdout:8/625: mknod d8/d6b/cda 0 2024-03-23T15:44:57.417 INFO:tasks.workunit.client.0.smithi076.stdout:9/754: creat d1/d1f/d82/ff8 x:0 0 0 2024-03-23T15:44:57.417 INFO:tasks.workunit.client.0.smithi076.stdout:8/626: creat d8/d2e/d30/d69/fdb x:0 0 0 2024-03-23T15:44:57.418 INFO:tasks.workunit.client.0.smithi076.stdout:9/755: mknod d1/d1f/d46/cf9 0 2024-03-23T15:44:57.419 INFO:tasks.workunit.client.0.smithi076.stdout:7/914: dwrite d8/d4e/d98/df6/d104/d40/d5d/f34 [0,4194304] 0 2024-03-23T15:44:57.419 INFO:tasks.workunit.client.0.smithi076.stdout:8/627: write d8/d2e/d30/f6a [991410,62529] 0 2024-03-23T15:44:57.419 INFO:tasks.workunit.client.0.smithi076.stdout:8/628: dread - d8/d26/d27/d56/f8b zero size 2024-03-23T15:44:57.420 INFO:tasks.workunit.client.0.smithi076.stdout:8/629: write d8/d2e/d30/fa2 [770879,20332] 0 2024-03-23T15:44:57.425 INFO:tasks.workunit.client.0.smithi076.stdout:6/499: dwrite d1/d4/d10/f78 [0,4194304] 0 2024-03-23T15:44:57.427 INFO:tasks.workunit.client.0.smithi076.stdout:6/500: mknod d1/dd/d2a/d23/cb0 0 2024-03-23T15:44:57.430 INFO:tasks.workunit.client.0.smithi076.stdout:6/501: creat d1/d4/d10/d39/fb1 x:0 0 0 2024-03-23T15:44:57.432 INFO:tasks.workunit.client.0.smithi076.stdout:0/753: rename d2/de/d10/d3a/d47/d69/da3/cbb to d2/d8/d32/ce9 0 2024-03-23T15:44:57.432 INFO:tasks.workunit.client.0.smithi076.stdout:5/773: rename d1a/d33 to d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/d10b 22 2024-03-23T15:44:57.432 INFO:tasks.workunit.client.0.smithi076.stdout:0/754: chown d2/d8/d2d/f45 1012490187 1 2024-03-23T15:44:57.433 INFO:tasks.workunit.client.0.smithi076.stdout:0/755: rmdir d2/de/d10 39 2024-03-23T15:44:57.433 INFO:tasks.workunit.client.0.smithi076.stdout:2/767: rename d1/d4/d64/d1b/d32 to d1/d4/d64/d82/d103 0 2024-03-23T15:44:57.436 INFO:tasks.workunit.client.0.smithi076.stdout:8/630: dwrite d8/d2e/d30/f96 [0,4194304] 0 2024-03-23T15:44:57.439 INFO:tasks.workunit.client.0.smithi076.stdout:7/915: dwrite d8/d4e/d98/df6/d104/d40/d5d/d3c/d8e/fb7 [0,4194304] 0 2024-03-23T15:44:57.445 INFO:tasks.workunit.client.0.smithi076.stdout:7/916: fdatasync d8/d4e/d98/df6/d104/f116 0 2024-03-23T15:44:57.446 INFO:tasks.workunit.client.0.smithi076.stdout:4/697: rename d8/d3a/d18/f98 to d8/d9/d1d/dc9/fd6 0 2024-03-23T15:44:57.446 INFO:tasks.workunit.client.0.smithi076.stdout:4/698: write d8/d3a/d18/d75/d8d/fcb [500675,37557] 0 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:5/774: dread d1a/d33/d3a/d67/da8/dc4/f52 [0,4194304] 0 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:2/768: unlink d1/fc0 0 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:7/917: mknod d8/d4e/d98/df6/d104/d3a/d85/daf/dca/c131 0 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:5/775: symlink d1a/d7e/l10c 0 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:7/918: symlink d8/de/d48/l132 0 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:7/919: chown d8/d4e/d98/df6/d104/d24/d91/d96/l78 16 1 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:7/920: rmdir d8/d4e/d98/df6/d104/d40/d5d/d3c/d46 39 2024-03-23T15:44:57.448 INFO:tasks.workunit.client.0.smithi076.stdout:7/921: stat d8/d4e/d98/df6/d104/d40/d5d/f3e 0 2024-03-23T15:44:57.449 INFO:tasks.workunit.client.0.smithi076.stdout:7/922: rmdir d8/d4e/d98/df6 39 2024-03-23T15:44:57.449 INFO:tasks.workunit.client.0.smithi076.stdout:7/923: truncate d8/f83 295608 0 2024-03-23T15:44:57.451 INFO:tasks.workunit.client.0.smithi076.stdout:6/502: dwrite d1/f11 [0,4194304] 0 2024-03-23T15:44:57.452 INFO:tasks.workunit.client.0.smithi076.stdout:6/503: creat d1/dd/df/fb2 x:0 0 0 2024-03-23T15:44:57.460 INFO:tasks.workunit.client.0.smithi076.stdout:8/631: dwrite d8/d89/db2/fcb [0,4194304] 0 2024-03-23T15:44:57.461 INFO:tasks.workunit.client.0.smithi076.stdout:8/632: dread - d8/d26/d27/d56/fd6 zero size 2024-03-23T15:44:57.462 INFO:tasks.workunit.client.0.smithi076.stdout:8/633: rmdir d8/d6b/d9a 39 2024-03-23T15:44:57.462 INFO:tasks.workunit.client.0.smithi076.stdout:8/634: chown d8/d6b/cda 6 1 2024-03-23T15:44:57.463 INFO:tasks.workunit.client.0.smithi076.stdout:4/699: dwrite d8/f71 [0,4194304] 0 2024-03-23T15:44:57.464 INFO:tasks.workunit.client.0.smithi076.stdout:8/635: creat d8/d6b/fdc x:0 0 0 2024-03-23T15:44:57.464 INFO:tasks.workunit.client.0.smithi076.stdout:8/636: chown d8/d4f/d80/db7 408263 1 2024-03-23T15:44:57.464 INFO:tasks.workunit.client.0.smithi076.stdout:8/637: dread - d8/d26/fcf zero size 2024-03-23T15:44:57.464 INFO:tasks.workunit.client.0.smithi076.stdout:4/700: write d8/d3a/f34 [118560,2029] 0 2024-03-23T15:44:57.465 INFO:tasks.workunit.client.0.smithi076.stdout:2/769: dwrite d1/d2e/d8a/fd6 [0,4194304] 0 2024-03-23T15:44:57.467 INFO:tasks.workunit.client.0.smithi076.stdout:2/770: unlink d1/d4/d64/d7c/feb 0 2024-03-23T15:44:57.467 INFO:tasks.workunit.client.0.smithi076.stdout:2/771: write d1/d4/d64/d1d/fe0 [83728,99128] 0 2024-03-23T15:44:57.469 INFO:tasks.workunit.client.0.smithi076.stdout:9/756: rename d1/d1f/d29/d2d/l8e to d1/d1f/d46/d4a/d5a/daa/lfa 0 2024-03-23T15:44:57.470 INFO:tasks.workunit.client.0.smithi076.stdout:9/757: dread - d1/d2f/d3f/da0/fd9 zero size 2024-03-23T15:44:57.471 INFO:tasks.workunit.client.0.smithi076.stdout:7/924: rename d8/de/f8a to d8/de/de8/d120/f133 0 2024-03-23T15:44:57.471 INFO:tasks.workunit.client.0.smithi076.stdout:7/925: write d8/d4e/d98/df6/d104/f116 [556316,649] 0 2024-03-23T15:44:57.471 INFO:tasks.workunit.client.0.smithi076.stdout:7/926: mkdir d8/d4e/d98/df6/d104/d40/d5d/d10f/d134 0 2024-03-23T15:44:57.474 INFO:tasks.workunit.client.0.smithi076.stdout:8/638: write d8/d2e/d30/f61 [2397112,12870] 0 2024-03-23T15:44:57.477 INFO:tasks.workunit.client.0.smithi076.stdout:4/701: dwrite d8/d3a/d3b/f39 [4194304,4194304] 0 2024-03-23T15:44:57.477 INFO:tasks.workunit.client.0.smithi076.stdout:4/702: chown d8/d3a/d18/d1b/l2c 9 1 2024-03-23T15:44:57.482 INFO:tasks.workunit.client.0.smithi076.stdout:8/639: mkdir d8/d26/d33/ddd 0 2024-03-23T15:44:57.485 INFO:tasks.workunit.client.0.smithi076.stdout:8/640: getdents d8/d89/db2 0 2024-03-23T15:44:57.486 INFO:tasks.workunit.client.0.smithi076.stdout:8/641: mknod d8/d6b/d9d/cde 0 2024-03-23T15:44:57.495 INFO:tasks.workunit.client.0.smithi076.stdout:8/642: dread d8/f48 [0,4194304] 0 2024-03-23T15:44:57.495 INFO:tasks.workunit.client.0.smithi076.stdout:2/772: dwrite d1/d4/db/d15/d29/f61 [0,4194304] 0 2024-03-23T15:44:57.495 INFO:tasks.workunit.client.0.smithi076.stdout:8/643: creat d8/d4f/d80/da3/fdf x:0 0 0 2024-03-23T15:44:57.495 INFO:tasks.workunit.client.0.smithi076.stdout:2/773: creat d1/d2e/dd1/f104 x:0 0 0 2024-03-23T15:44:57.495 INFO:tasks.workunit.client.0.smithi076.stdout:2/774: chown d1/c2c 1321 1 2024-03-23T15:44:57.495 INFO:tasks.workunit.client.0.smithi076.stdout:2/775: chown d1/d4/d64/d82/d103/d79/fb5 84463 1 2024-03-23T15:44:57.496 INFO:tasks.workunit.client.0.smithi076.stdout:7/927: dwrite d8/fcb [0,4194304] 0 2024-03-23T15:44:57.496 INFO:tasks.workunit.client.0.smithi076.stdout:2/776: symlink d1/d4/d64/d7c/l105 0 2024-03-23T15:44:57.496 INFO:tasks.workunit.client.0.smithi076.stdout:2/777: truncate d1/d4/db/f3d 4698081 0 2024-03-23T15:44:57.497 INFO:tasks.workunit.client.0.smithi076.stdout:7/928: stat d8/cf2 0 2024-03-23T15:44:57.499 INFO:tasks.workunit.client.0.smithi076.stdout:8/644: dread d8/f18 [0,4194304] 0 2024-03-23T15:44:57.499 INFO:tasks.workunit.client.0.smithi076.stdout:2/778: link d1/d4/d52/ccd d1/d2e/c106 0 2024-03-23T15:44:57.499 INFO:tasks.workunit.client.0.smithi076.stdout:4/703: dwrite d8/d9/d1d/dc9/fd6 [4194304,4194304] 0 2024-03-23T15:44:57.499 INFO:tasks.workunit.client.0.smithi076.stdout:9/758: dwrite d1/d2f/d3a/d53/f61 [4194304,4194304] 0 2024-03-23T15:44:57.500 INFO:tasks.workunit.client.0.smithi076.stdout:9/759: fdatasync d1/d2f/d3a/d64/f91 0 2024-03-23T15:44:57.500 INFO:tasks.workunit.client.0.smithi076.stdout:9/760: dread - d1/d1f/d82/ff8 zero size 2024-03-23T15:44:57.500 INFO:tasks.workunit.client.0.smithi076.stdout:9/761: fdatasync d1/d2f/d3a/f84 0 2024-03-23T15:44:57.500 INFO:tasks.workunit.client.0.smithi076.stdout:8/645: creat d8/d4f/d80/da3/fe0 x:0 0 0 2024-03-23T15:44:57.501 INFO:tasks.workunit.client.0.smithi076.stdout:7/929: mknod d8/d4e/d98/df6/d104/d24/c135 0 2024-03-23T15:44:57.501 INFO:tasks.workunit.client.0.smithi076.stdout:7/930: fsync d8/de/fad 0 2024-03-23T15:44:57.501 INFO:tasks.workunit.client.0.smithi076.stdout:7/931: write d8/d4e/d98/df6/d104/d18/f2b [5860066,73292] 0 2024-03-23T15:44:57.501 INFO:tasks.workunit.client.0.smithi076.stdout:2/779: dread d1/d2e/ff7 [0,4194304] 0 2024-03-23T15:44:57.501 INFO:tasks.workunit.client.0.smithi076.stdout:6/504: rename d1/d4/d10/d6b to d1/dd/df/d9b/db3 0 2024-03-23T15:44:57.502 INFO:tasks.workunit.client.0.smithi076.stdout:9/762: creat d1/d1f/d29/d7f/ffb x:0 0 0 2024-03-23T15:44:57.509 INFO:tasks.workunit.client.0.smithi076.stdout:6/505: read d1/dd/df/d5b/d62/f7e [687742,6298] 0 2024-03-23T15:44:57.511 INFO:tasks.workunit.client.0.smithi076.stdout:7/932: dread d8/d4e/d98/df6/d104/d40/d5d/f3e [0,4194304] 0 2024-03-23T15:44:57.512 INFO:tasks.workunit.client.0.smithi076.stdout:6/506: creat d1/d2/fb4 x:0 0 0 2024-03-23T15:44:57.512 INFO:tasks.workunit.client.0.smithi076.stdout:7/933: chown d8/d4e/d98/df6/d104/d3a/dbf/dc7/f66 19035433 1 2024-03-23T15:44:57.512 INFO:tasks.workunit.client.0.smithi076.stdout:4/704: write d8/d3a/d18/d52/f79 [2767284,4381] 0 2024-03-23T15:44:57.513 INFO:tasks.workunit.client.0.smithi076.stdout:4/705: read d8/f6a [162552,93309] 0 2024-03-23T15:44:57.514 INFO:tasks.workunit.client.0.smithi076.stdout:6/507: mkdir d1/dd/df/db5 0 2024-03-23T15:44:57.514 INFO:tasks.workunit.client.0.smithi076.stdout:2/780: dread d1/d4/d64/d1d/f73 [0,4194304] 0 2024-03-23T15:44:57.514 INFO:tasks.workunit.client.0.smithi076.stdout:7/934: creat d8/d4e/d98/df6/d104/d3a/d85/daf/dca/d11a/f136 x:0 0 0 2024-03-23T15:44:57.514 INFO:tasks.workunit.client.0.smithi076.stdout:7/935: write d8/de/d99/fff [402589,114399] 0 2024-03-23T15:44:57.515 INFO:tasks.workunit.client.0.smithi076.stdout:2/781: creat d1/d4/d64/d7c/f107 x:0 0 0 2024-03-23T15:44:57.524 INFO:tasks.workunit.client.0.smithi076.stdout:4/706: dread d8/ff [0,4194304] 0 2024-03-23T15:44:57.524 INFO:tasks.workunit.client.0.smithi076.stdout:9/763: link d1/d1f/f72 d1/d1f/d46/d4a/d5a/daa/ffc 0 2024-03-23T15:44:57.526 INFO:tasks.workunit.client.0.smithi076.stdout:4/707: mkdir d8/dd7 0 2024-03-23T15:44:57.527 INFO:tasks.workunit.client.0.smithi076.stdout:4/708: stat d8/d3a/d18/d52/f6d 0 2024-03-23T15:44:57.527 INFO:tasks.workunit.client.0.smithi076.stdout:4/709: readlink d8/d9/lac 0 2024-03-23T15:44:57.529 INFO:tasks.workunit.client.0.smithi076.stdout:4/710: creat d8/d3a/d18/d75/fd8 x:0 0 0 2024-03-23T15:44:57.529 INFO:tasks.workunit.client.0.smithi076.stdout:4/711: write d8/ff [6958747,62645] 0 2024-03-23T15:44:57.529 INFO:tasks.workunit.client.0.smithi076.stdout:4/712: readlink d8/d9/l6f 0 2024-03-23T15:44:57.530 INFO:tasks.workunit.client.0.smithi076.stdout:4/713: getdents d8/d3a/d69/d73/d85/d88 0 2024-03-23T15:44:57.535 INFO:tasks.workunit.client.0.smithi076.stdout:2/782: write d1/d4/d64/d1b/f1c [1487338,46209] 0 2024-03-23T15:44:57.535 INFO:tasks.workunit.client.0.smithi076.stdout:4/714: dread d8/d41/d97/fb9 [0,4194304] 0 2024-03-23T15:44:57.538 INFO:tasks.workunit.client.0.smithi076.stdout:4/715: getdents d8/d3a/d3b 0 2024-03-23T15:44:57.539 INFO:tasks.workunit.client.0.smithi076.stdout:4/716: chown d8/d3a/d18/d75/d8d 4447917 1 2024-03-23T15:44:57.541 INFO:tasks.workunit.client.0.smithi076.stdout:7/936: dwrite d8/de/f81 [0,4194304] 0 2024-03-23T15:44:57.541 INFO:tasks.workunit.client.0.smithi076.stdout:6/508: rmdir d1/d2 39 2024-03-23T15:44:57.542 INFO:tasks.workunit.client.0.smithi076.stdout:6/509: dread - d1/d4/f86 zero size 2024-03-23T15:44:57.542 INFO:tasks.workunit.client.0.smithi076.stdout:6/510: chown d1/dd/df/d9b/db3/la8 97149 1 2024-03-23T15:44:57.543 INFO:tasks.workunit.client.0.smithi076.stdout:4/717: rmdir d8/d3a/d69 39 2024-03-23T15:44:57.543 INFO:tasks.workunit.client.0.smithi076.stdout:7/937: readlink d8/d4e/d98/df6/d104/d3a/l4b 0 2024-03-23T15:44:57.543 INFO:tasks.workunit.client.0.smithi076.stdout:7/938: write d8/d4e/d8d/dbe/fdb [5380583,76846] 0 2024-03-23T15:44:57.543 INFO:tasks.workunit.client.0.smithi076.stdout:6/511: mknod d1/dd/df/d9b/cb6 0 2024-03-23T15:44:57.543 INFO:tasks.workunit.client.0.smithi076.stdout:4/718: getdents d8/d80 0 2024-03-23T15:44:57.544 INFO:tasks.workunit.client.0.smithi076.stdout:4/719: chown d8/f66 16671864 1 2024-03-23T15:44:57.544 INFO:tasks.workunit.client.0.smithi076.stdout:6/512: link d1/d4/d10/c52 d1/d4/cb7 0 2024-03-23T15:44:57.547 INFO:tasks.workunit.client.0.smithi076.stdout:7/939: unlink d8/d4e/d98/df6/d104/f15 0 2024-03-23T15:44:57.547 INFO:tasks.workunit.client.0.smithi076.stdout:9/764: dwrite d1/d1f/d29/d2d/d37/fb0 [0,4194304] 0 2024-03-23T15:44:57.549 INFO:tasks.workunit.client.0.smithi076.stdout:7/940: creat d8/d4e/d98/df6/d104/d24/d91/d96/de0/f137 x:0 0 0 2024-03-23T15:44:57.550 INFO:tasks.workunit.client.0.smithi076.stdout:6/513: mkdir d1/dd/df/d9b/db3/d8d/db8 0 2024-03-23T15:44:57.553 INFO:tasks.workunit.client.0.smithi076.stdout:6/514: creat d1/d4/fb9 x:0 0 0 2024-03-23T15:44:57.557 INFO:tasks.workunit.client.0.smithi076.stdout:2/783: dwrite d1/d4/d64/d1b/f71 [0,4194304] 0 2024-03-23T15:44:57.559 INFO:tasks.workunit.client.0.smithi076.stdout:7/941: dread d8/d4e/d98/df6/d104/d40/d5d/f101 [0,4194304] 0 2024-03-23T15:44:57.559 INFO:tasks.workunit.client.0.smithi076.stdout:2/784: creat d1/d4/db/dc8/f108 x:0 0 0 2024-03-23T15:44:57.565 INFO:tasks.workunit.client.0.smithi076.stdout:6/515: write d1/d4/d10/f78 [1600903,68405] 0 2024-03-23T15:44:57.566 INFO:tasks.workunit.client.0.smithi076.stdout:6/516: dread - d1/dd/df/fb2 zero size 2024-03-23T15:44:57.566 INFO:tasks.workunit.client.0.smithi076.stdout:6/517: write d1/dd/df/d9b/db3/d8d/d60/d7c/f83 [4036030,48524] 0 2024-03-23T15:44:57.566 INFO:tasks.workunit.client.0.smithi076.stdout:2/785: symlink d1/d4/db/d15/l109 0 2024-03-23T15:44:57.571 INFO:tasks.workunit.client.0.smithi076.stdout:6/518: mknod d1/dd/cba 0 2024-03-23T15:44:57.571 INFO:tasks.workunit.client.0.smithi076.stdout:9/765: dwrite d1/f1a [8388608,4194304] 0 2024-03-23T15:44:57.572 INFO:tasks.workunit.client.0.smithi076.stdout:9/766: chown d1/d2f/d3a/le3 1051 1 2024-03-23T15:44:57.572 INFO:tasks.workunit.client.0.smithi076.stdout:6/519: symlink d1/d4/d10/d39/lbb 0 2024-03-23T15:44:57.581 INFO:tasks.workunit.client.0.smithi076.stdout:2/786: dwrite d1/d4/d64/d1d/fe0 [0,4194304] 0 2024-03-23T15:44:57.582 INFO:tasks.workunit.client.0.smithi076.stdout:2/787: rename d1/d2e/d6a/l88 to d1/d2e/d6a/l10a 0 2024-03-23T15:44:57.582 INFO:tasks.workunit.client.0.smithi076.stdout:2/788: mknod d1/d4/db/d3e/c10b 0 2024-03-23T15:44:57.583 INFO:tasks.workunit.client.0.smithi076.stdout:2/789: stat d1/d4/db/d15/fb3 0 2024-03-23T15:44:57.588 INFO:tasks.workunit.client.0.smithi076.stdout:2/790: mkdir d1/d4/db/d3e/d10c 0 2024-03-23T15:44:57.591 INFO:tasks.workunit.client.0.smithi076.stdout:6/520: mkdir d1/dd/dbc 0 2024-03-23T15:44:57.592 INFO:tasks.workunit.client.0.smithi076.stdout:6/521: readlink d1/d81/l88 0 2024-03-23T15:44:57.592 INFO:tasks.workunit.client.0.smithi076.stdout:6/522: write d1/dd/df/fa4 [389892,18236] 0 2024-03-23T15:44:57.593 INFO:tasks.workunit.client.0.smithi076.stdout:6/523: mkdir d1/d81/dbd 0 2024-03-23T15:44:57.593 INFO:tasks.workunit.client.0.smithi076.stdout:6/524: mknod d1/d4/d10/d39/cbe 0 2024-03-23T15:44:57.595 INFO:tasks.workunit.client.0.smithi076.stdout:2/791: symlink d1/l10d 0 2024-03-23T15:44:57.595 INFO:tasks.workunit.client.0.smithi076.stdout:2/792: truncate d1/d4/d64/d1b/f6c 4333988 0 2024-03-23T15:44:57.596 INFO:tasks.workunit.client.0.smithi076.stdout:2/793: write d1/d4/d52/fe8 [515233,7637] 0 2024-03-23T15:44:57.596 INFO:tasks.workunit.client.0.smithi076.stdout:9/767: dread d1/f12 [0,4194304] 0 2024-03-23T15:44:57.596 INFO:tasks.workunit.client.0.smithi076.stdout:2/794: dread d1/d4/d64/d82/d103/d79/fa9 [0,4194304] 0 2024-03-23T15:44:57.597 INFO:tasks.workunit.client.0.smithi076.stdout:9/768: mkdir d1/dfd 0 2024-03-23T15:44:57.606 INFO:tasks.workunit.client.0.smithi076.stdout:2/795: dread d1/d4/db/d15/d29/f61 [0,4194304] 0 2024-03-23T15:44:57.611 INFO:tasks.workunit.client.0.smithi076.stdout:9/769: dwrite d1/d2f/d3f/da0/fc9 [0,4194304] 0 2024-03-23T15:44:57.615 INFO:tasks.workunit.client.0.smithi076.stdout:9/770: rmdir d1/d2f/d3a/d74/deb 0 2024-03-23T15:44:57.618 INFO:tasks.workunit.client.0.smithi076.stdout:9/771: write d1/d2f/d3a/d74/fb3 [4121028,120868] 0 2024-03-23T15:44:57.619 INFO:tasks.workunit.client.0.smithi076.stdout:9/772: symlink d1/d2f/d3f/da0/lfe 0 2024-03-23T15:44:57.629 INFO:tasks.workunit.client.0.smithi076.stdout:2/796: dwrite d1/d4/d64/f20 [0,4194304] 0 2024-03-23T15:44:57.632 INFO:tasks.workunit.client.0.smithi076.stdout:2/797: creat d1/d4/db/d15/f10e x:0 0 0 2024-03-23T15:44:57.634 INFO:tasks.workunit.client.0.smithi076.stdout:2/798: symlink d1/dbf/l10f 0 2024-03-23T15:44:57.635 INFO:tasks.workunit.client.0.smithi076.stdout:2/799: symlink d1/d4/d64/d82/d7f/l110 0 2024-03-23T15:44:57.647 INFO:tasks.workunit.client.0.smithi076.stdout:1/441: sync 2024-03-23T15:44:57.648 INFO:tasks.workunit.client.0.smithi076.stdout:3/542: sync 2024-03-23T15:44:57.648 INFO:tasks.workunit.client.0.smithi076.stdout:7/942: rename d8/d4e/d98/df6/d104/d3a/dbf/dc7/fa1 to d8/de/d99/f138 0 2024-03-23T15:44:57.648 INFO:tasks.workunit.client.0.smithi076.stdout:7/943: stat d8/d4e/d98/df6/d104 0 2024-03-23T15:44:57.648 INFO:tasks.workunit.client.0.smithi076.stdout:1/442: chown d0/c15 940359 1 2024-03-23T15:44:57.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/543: creat d5/d17/d1c/d32/d70/d8c/fb3 x:0 0 0 2024-03-23T15:44:57.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/544: read - d5/d35/d4c/d5a/f82 zero size 2024-03-23T15:44:57.649 INFO:tasks.workunit.client.0.smithi076.stdout:3/545: chown d5/d17/l58 951 1 2024-03-23T15:44:57.655 INFO:tasks.workunit.client.0.smithi076.stdout:3/546: mkdir d5/d35/d4c/d5a/db4 0 2024-03-23T15:44:57.657 INFO:tasks.workunit.client.0.smithi076.stdout:2/800: dwrite d1/d4/db/d15/f55 [4194304,4194304] 0 2024-03-23T15:44:57.657 INFO:tasks.workunit.client.0.smithi076.stdout:2/801: chown d1/d2e/d6a/f9b 110735 1 2024-03-23T15:44:57.659 INFO:tasks.workunit.client.0.smithi076.stdout:3/547: creat d5/d35/d3c/fb5 x:0 0 0 2024-03-23T15:44:57.661 INFO:tasks.workunit.client.0.smithi076.stdout:6/525: mknod d1/dd/df/d9b/db3/d8d/d60/cbf 0 2024-03-23T15:44:57.661 INFO:tasks.workunit.client.0.smithi076.stdout:6/526: readlink d1/d2/l41 0 2024-03-23T15:44:57.662 INFO:tasks.workunit.client.0.smithi076.stdout:6/527: creat d1/dd/df/d9b/db3/d8d/db8/fc0 x:0 0 0 2024-03-23T15:44:57.662 INFO:tasks.workunit.client.0.smithi076.stdout:2/802: read d1/d4/db/d15/f55 [5806928,69364] 0 2024-03-23T15:44:57.663 INFO:tasks.workunit.client.0.smithi076.stdout:9/773: symlink d1/d1f/d46/d83/dad/lff 0 2024-03-23T15:44:57.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/803: dread d1/d4/f98 [4194304,4194304] 0 2024-03-23T15:44:57.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/804: fsync d1/d4/d52/fbb 0 2024-03-23T15:44:57.671 INFO:tasks.workunit.client.0.smithi076.stdout:3/548: dwrite d5/d17/f29 [4194304,4194304] 0 2024-03-23T15:44:57.672 INFO:tasks.workunit.client.0.smithi076.stdout:3/549: write d5/d17/d1c/d32/d4a/f55 [1116338,21660] 0 2024-03-23T15:44:57.673 INFO:tasks.workunit.client.0.smithi076.stdout:3/550: chown d5/d35/d3c/d67 2 1 2024-03-23T15:44:57.673 INFO:tasks.workunit.client.0.smithi076.stdout:2/805: unlink d1/d4/db/d15/d29/f61 0 2024-03-23T15:44:57.675 INFO:tasks.workunit.client.0.smithi076.stdout:2/806: creat d1/d4/db/d3e/d10c/f111 x:0 0 0 2024-03-23T15:44:57.676 INFO:tasks.workunit.client.0.smithi076.stdout:2/807: creat d1/d4/d64/d82/d103/d79/dac/f112 x:0 0 0 2024-03-23T15:44:57.678 INFO:tasks.workunit.client.0.smithi076.stdout:6/528: dwrite d1/dd/df/d5b/d62/f6e [0,4194304] 0 2024-03-23T15:44:57.679 INFO:tasks.workunit.client.0.smithi076.stdout:2/808: creat d1/d4/d52/d6e/f113 x:0 0 0 2024-03-23T15:44:57.682 INFO:tasks.workunit.client.0.smithi076.stdout:2/809: mknod d1/d4/d64/d82/d103/c114 0 2024-03-23T15:44:57.683 INFO:tasks.workunit.client.0.smithi076.stdout:2/810: dread d1/d4/db/d15/d29/f8f [0,4194304] 0 2024-03-23T15:44:57.685 INFO:tasks.workunit.client.0.smithi076.stdout:1/443: rename d0/d3c/l42 to d0/d35/d40/d47/d17/d20/l92 0 2024-03-23T15:44:57.685 INFO:tasks.workunit.client.0.smithi076.stdout:0/756: sync 2024-03-23T15:44:57.685 INFO:tasks.workunit.client.0.smithi076.stdout:9/774: dwrite d1/d1f/d29/d2d/d37/f66 [4194304,4194304] 0 2024-03-23T15:44:57.687 INFO:tasks.workunit.client.0.smithi076.stdout:9/775: symlink d1/d1f/d29/d2d/l100 0 2024-03-23T15:44:57.688 INFO:tasks.workunit.client.0.smithi076.stdout:1/444: symlink d0/d35/d40/d47/l93 0 2024-03-23T15:44:57.689 INFO:tasks.workunit.client.0.smithi076.stdout:9/776: mknod d1/d2f/d3f/dc0/c101 0 2024-03-23T15:44:57.690 INFO:tasks.workunit.client.0.smithi076.stdout:9/777: chown d1/d2f/d3a/d64 4 1 2024-03-23T15:44:57.690 INFO:tasks.workunit.client.0.smithi076.stdout:1/445: symlink d0/d35/l94 0 2024-03-23T15:44:57.691 INFO:tasks.workunit.client.0.smithi076.stdout:9/778: mkdir d1/d2f/d3a/d74/d102 0 2024-03-23T15:44:57.691 INFO:tasks.workunit.client.0.smithi076.stdout:9/779: readlink d1/d1f/d29/d2d/d37/d47/le8 0 2024-03-23T15:44:57.691 INFO:tasks.workunit.client.0.smithi076.stdout:9/780: fsync d1/d2f/d3a/d53/f61 0 2024-03-23T15:44:57.692 INFO:tasks.workunit.client.0.smithi076.stdout:1/446: symlink d0/d35/d40/d47/d74/d83/d8f/l95 0 2024-03-23T15:44:57.692 INFO:tasks.workunit.client.0.smithi076.stdout:1/447: getdents d0/d35/d40/d47/d8c 0 2024-03-23T15:44:57.695 INFO:tasks.workunit.client.0.smithi076.stdout:1/448: read d0/d35/d40/d47/d17/f38 [713841,6723] 0 2024-03-23T15:44:57.696 INFO:tasks.workunit.client.0.smithi076.stdout:1/449: truncate d0/d35/d40/d47/d17/d20/f33 2956533 0 2024-03-23T15:44:57.696 INFO:tasks.workunit.client.0.smithi076.stdout:1/450: fsync d0/f5e 0 2024-03-23T15:44:57.698 INFO:tasks.workunit.client.0.smithi076.stdout:6/529: dwrite d1/dd/df/d5b/f99 [0,4194304] 0 2024-03-23T15:44:57.700 INFO:tasks.workunit.client.0.smithi076.stdout:1/451: write d0/d35/d40/d47/d17/f30 [2882924,107727] 0 2024-03-23T15:44:57.700 INFO:tasks.workunit.client.0.smithi076.stdout:1/452: write d0/d35/d40/d47/f6b [4275529,101099] 0 2024-03-23T15:44:57.703 INFO:tasks.workunit.client.0.smithi076.stdout:6/530: mknod d1/d4/cc1 0 2024-03-23T15:44:57.704 INFO:tasks.workunit.client.0.smithi076.stdout:2/811: dwrite d1/d4/d64/d1b/dd4/ff1 [0,4194304] 0 2024-03-23T15:44:57.704 INFO:tasks.workunit.client.0.smithi076.stdout:2/812: chown d1/d4/d64/d1d/fe5 5389124 1 2024-03-23T15:44:57.707 INFO:tasks.workunit.client.0.smithi076.stdout:6/531: link d1/d2/l2f d1/dd/df/d9b/db3/lc2 0 2024-03-23T15:44:57.707 INFO:tasks.workunit.client.0.smithi076.stdout:0/757: dwrite d2/de/d10/d3a/d47/fab [0,4194304] 0 2024-03-23T15:44:57.707 INFO:tasks.workunit.client.0.smithi076.stdout:0/758: chown d2/de/d10/d12/f9e 7 1 2024-03-23T15:44:57.707 INFO:tasks.workunit.client.0.smithi076.stdout:2/813: creat d1/d4/d64/d82/d103/f115 x:0 0 0 2024-03-23T15:44:57.713 INFO:tasks.workunit.client.0.smithi076.stdout:6/532: rmdir d1/dd/df/d9b 39 2024-03-23T15:44:57.714 INFO:tasks.workunit.client.0.smithi076.stdout:6/533: chown d1/d4/d10/d39 27000987 1 2024-03-23T15:44:57.714 INFO:tasks.workunit.client.0.smithi076.stdout:6/534: chown d1/dd/d2a/d23/f68 0 1 2024-03-23T15:44:57.714 INFO:tasks.workunit.client.0.smithi076.stdout:6/535: write d1/dd/df/d9b/db3/d8d/f9f [25717,95592] 0 2024-03-23T15:44:57.714 INFO:tasks.workunit.client.0.smithi076.stdout:6/536: truncate d1/dd/df/d5b/fa2 953277 0 2024-03-23T15:44:57.716 INFO:tasks.workunit.client.0.smithi076.stdout:1/453: dwrite d0/d35/d82/f89 [0,4194304] 0 2024-03-23T15:44:57.716 INFO:tasks.workunit.client.0.smithi076.stdout:2/814: dread d1/d4/f1e [0,4194304] 0 2024-03-23T15:44:57.717 INFO:tasks.workunit.client.0.smithi076.stdout:9/781: rename d1/d1f/d29/d2d/f33 to d1/d2f/d3a/f103 0 2024-03-23T15:44:57.718 INFO:tasks.workunit.client.0.smithi076.stdout:1/454: symlink d0/d35/d40/d47/d17/d20/l96 0 2024-03-23T15:44:57.718 INFO:tasks.workunit.client.0.smithi076.stdout:1/455: write d0/d35/d40/d47/f6b [4946488,71377] 0 2024-03-23T15:44:57.720 INFO:tasks.workunit.client.0.smithi076.stdout:0/759: rename d2/d9a/lcb to d2/de/de0/lea 0 2024-03-23T15:44:57.720 INFO:tasks.workunit.client.0.smithi076.stdout:0/760: write d2/de/f1a [1642014,60441] 0 2024-03-23T15:44:57.723 INFO:tasks.workunit.client.0.smithi076.stdout:1/456: symlink d0/d2/d70/l97 0 2024-03-23T15:44:57.723 INFO:tasks.workunit.client.0.smithi076.stdout:1/457: chown d0/d2/d91/l4c 5 1 2024-03-23T15:44:57.723 INFO:tasks.workunit.client.0.smithi076.stdout:9/782: rmdir d1 39 2024-03-23T15:44:57.724 INFO:tasks.workunit.client.0.smithi076.stdout:0/761: unlink d2/d8/f53 0 2024-03-23T15:44:57.725 INFO:tasks.workunit.client.0.smithi076.stdout:1/458: dread d0/d2/f48 [4194304,4194304] 0 2024-03-23T15:44:57.726 INFO:tasks.workunit.client.0.smithi076.stdout:0/762: creat d2/de/d10/d3a/feb x:0 0 0 2024-03-23T15:44:57.726 INFO:tasks.workunit.client.0.smithi076.stdout:1/459: creat d0/d35/d40/d47/d74/d83/f98 x:0 0 0 2024-03-23T15:44:57.727 INFO:tasks.workunit.client.0.smithi076.stdout:1/460: chown d0/d35/f6f 10945087 1 2024-03-23T15:44:57.727 INFO:tasks.workunit.client.0.smithi076.stdout:0/763: mkdir d2/de/d71/dec 0 2024-03-23T15:44:57.733 INFO:tasks.workunit.client.0.smithi076.stdout:9/783: truncate d1/d1f/d46/d4a/f50 4923642 0 2024-03-23T15:44:57.734 INFO:tasks.workunit.client.0.smithi076.stdout:6/537: dwrite d1/dd/df/d9b/db3/d8d/f77 [0,4194304] 0 2024-03-23T15:44:57.735 INFO:tasks.workunit.client.0.smithi076.stdout:6/538: creat d1/dd/df/db5/fc3 x:0 0 0 2024-03-23T15:44:57.736 INFO:tasks.workunit.client.0.smithi076.stdout:2/815: dwrite d1/d4/d64/d7c/fc3 [0,4194304] 0 2024-03-23T15:44:57.736 INFO:tasks.workunit.client.0.smithi076.stdout:6/539: creat d1/dd/df/dac/fc4 x:0 0 0 2024-03-23T15:44:57.736 INFO:tasks.workunit.client.0.smithi076.stdout:6/540: chown d1/d2/f22 3600 1 2024-03-23T15:44:57.740 INFO:tasks.workunit.client.0.smithi076.stdout:2/816: mknod d1/d4/dfb/c116 0 2024-03-23T15:44:57.740 INFO:tasks.workunit.client.0.smithi076.stdout:2/817: chown d1/d2e/d6a/l10a 0 1 2024-03-23T15:44:57.740 INFO:tasks.workunit.client.0.smithi076.stdout:2/818: dread - d1/d4/d64/d82/d103/d79/dac/f112 zero size 2024-03-23T15:44:57.741 INFO:tasks.workunit.client.0.smithi076.stdout:6/541: write d1/dd/df/d9b/db3/d8d/f9f [119455,22033] 0 2024-03-23T15:44:57.741 INFO:tasks.workunit.client.0.smithi076.stdout:9/784: dread d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:57.743 INFO:tasks.workunit.client.0.smithi076.stdout:9/785: dread d1/d2f/fbf [0,4194304] 0 2024-03-23T15:44:57.744 INFO:tasks.workunit.client.0.smithi076.stdout:6/542: symlink d1/d81/dbd/lc5 0 2024-03-23T15:44:57.744 INFO:tasks.workunit.client.0.smithi076.stdout:0/764: dwrite d2/d8/f62 [4194304,4194304] 0 2024-03-23T15:44:57.744 INFO:tasks.workunit.client.0.smithi076.stdout:0/765: chown d2/de/d10/d3a/d47/d69/d84/dc5 489665220 1 2024-03-23T15:44:57.745 INFO:tasks.workunit.client.0.smithi076.stdout:6/543: truncate d1/dd/d2a/d23/f2d 4075194 0 2024-03-23T15:44:57.747 INFO:tasks.workunit.client.0.smithi076.stdout:6/544: creat d1/d4/fc6 x:0 0 0 2024-03-23T15:44:57.747 INFO:tasks.workunit.client.0.smithi076.stdout:6/545: fdatasync d1/dd/f26 0 2024-03-23T15:44:57.748 INFO:tasks.workunit.client.0.smithi076.stdout:2/819: rename d1/d2e/ff7 to d1/d4/db/d15/f117 0 2024-03-23T15:44:57.748 INFO:tasks.workunit.client.0.smithi076.stdout:2/820: readlink d1/d4/db/d3e/l58 0 2024-03-23T15:44:57.749 INFO:tasks.workunit.client.0.smithi076.stdout:1/461: dwrite d0/d35/f6f [0,4194304] 0 2024-03-23T15:44:57.762 INFO:tasks.workunit.client.0.smithi076.stdout:0/766: dwrite d2/de/d10/f8c [0,4194304] 0 2024-03-23T15:44:57.762 INFO:tasks.workunit.client.0.smithi076.stdout:6/546: mkdir d1/dd/df/d5b/d62/dc7 0 2024-03-23T15:44:57.763 INFO:tasks.workunit.client.0.smithi076.stdout:6/547: getdents d1/dd/df 0 2024-03-23T15:44:57.769 INFO:tasks.workunit.client.0.smithi076.stdout:9/786: dwrite d1/d2f/f42 [0,4194304] 0 2024-03-23T15:44:57.769 INFO:tasks.workunit.client.0.smithi076.stdout:6/548: mkdir d1/d2/dc8 0 2024-03-23T15:44:57.771 INFO:tasks.workunit.client.0.smithi076.stdout:5/776: sync 2024-03-23T15:44:57.771 INFO:tasks.workunit.client.0.smithi076.stdout:8/646: sync 2024-03-23T15:44:57.771 INFO:tasks.workunit.client.0.smithi076.stdout:4/720: sync 2024-03-23T15:44:57.771 INFO:tasks.workunit.client.0.smithi076.stdout:5/777: fdatasync d1a/d33/d3a/d67/da8/dc4/d5f/f62 0 2024-03-23T15:44:57.772 INFO:tasks.workunit.client.0.smithi076.stdout:4/721: mknod d8/d3a/d69/d73/d85/cd9 0 2024-03-23T15:44:57.773 INFO:tasks.workunit.client.0.smithi076.stdout:8/647: dread d8/f7a [0,4194304] 0 2024-03-23T15:44:57.774 INFO:tasks.workunit.client.0.smithi076.stdout:1/462: dwrite d0/d3c/d6e/f76 [0,4194304] 0 2024-03-23T15:44:57.774 INFO:tasks.workunit.client.0.smithi076.stdout:2/821: dwrite d1/d2e/f42 [0,4194304] 0 2024-03-23T15:44:57.774 INFO:tasks.workunit.client.0.smithi076.stdout:4/722: mknod d8/d9/d1d/cda 0 2024-03-23T15:44:57.775 INFO:tasks.workunit.client.0.smithi076.stdout:5/778: link d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/l104 d1a/d33/d3a/d67/da8/l10d 0 2024-03-23T15:44:57.778 INFO:tasks.workunit.client.0.smithi076.stdout:4/723: symlink d8/d3a/d69/d73/d76/ldb 0 2024-03-23T15:44:57.781 INFO:tasks.workunit.client.0.smithi076.stdout:2/822: dread d1/d4/db/d15/d29/d7d/fad [0,4194304] 0 2024-03-23T15:44:57.783 INFO:tasks.workunit.client.0.smithi076.stdout:2/823: creat d1/d2e/d6a/f118 x:0 0 0 2024-03-23T15:44:57.783 INFO:tasks.workunit.client.0.smithi076.stdout:4/724: symlink d8/d3a/d18/ldc 0 2024-03-23T15:44:57.783 INFO:tasks.workunit.client.0.smithi076.stdout:4/725: chown d8/d3a/d3b/f39 4 1 2024-03-23T15:44:57.783 INFO:tasks.workunit.client.0.smithi076.stdout:4/726: chown d8/f30 896 1 2024-03-23T15:44:57.783 INFO:tasks.workunit.client.0.smithi076.stdout:4/727: readlink d8/d3a/d69/d73/d9a/lc8 0 2024-03-23T15:44:57.785 INFO:tasks.workunit.client.0.smithi076.stdout:8/648: write d8/f1e [2715436,20694] 0 2024-03-23T15:44:57.785 INFO:tasks.workunit.client.0.smithi076.stdout:4/728: symlink d8/d3a/d18/d75/ldd 0 2024-03-23T15:44:57.786 INFO:tasks.workunit.client.0.smithi076.stdout:4/729: write d8/d3a/d18/d52/fad [819052,33144] 0 2024-03-23T15:44:57.787 INFO:tasks.workunit.client.0.smithi076.stdout:9/787: dwrite d1/d2f/d3a/d64/f91 [0,4194304] 0 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/788: chown d1/f19 10733063 1 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/789: read d1/d1f/d29/d2d/d37/f59 [908707,9756] 0 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/790: write d1/d1f/f22 [277184,116686] 0 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/791: dread - d1/d2f/d3f/da0/fa6 zero size 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/792: chown d1/d1f/d46/d4a/f50 132008 1 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/793: write d1/d1f/f22 [3896296,20395] 0 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:9/794: chown d1/d1f/d46/d83/dad/lff 1242828 1 2024-03-23T15:44:57.788 INFO:tasks.workunit.client.0.smithi076.stdout:8/649: mkdir d8/dd0/de1 0 2024-03-23T15:44:57.789 INFO:tasks.workunit.client.0.smithi076.stdout:1/463: dwrite d0/f26 [4194304,4194304] 0 2024-03-23T15:44:57.790 INFO:tasks.workunit.client.0.smithi076.stdout:4/730: creat d8/d3a/d69/fde x:0 0 0 2024-03-23T15:44:57.790 INFO:tasks.workunit.client.0.smithi076.stdout:4/731: chown d8/d41 2 1 2024-03-23T15:44:57.794 INFO:tasks.workunit.client.0.smithi076.stdout:8/650: symlink d8/le2 0 2024-03-23T15:44:57.794 INFO:tasks.workunit.client.0.smithi076.stdout:8/651: chown d8/d6b/d8e/f92 68157529 1 2024-03-23T15:44:57.796 INFO:tasks.workunit.client.0.smithi076.stdout:4/732: truncate d8/d9/f12 6272374 0 2024-03-23T15:44:57.796 INFO:tasks.workunit.client.0.smithi076.stdout:4/733: read - d8/d3a/d3b/f94 zero size 2024-03-23T15:44:57.799 INFO:tasks.workunit.client.0.smithi076.stdout:9/795: mknod d1/d1f/d46/d83/c104 0 2024-03-23T15:44:57.799 INFO:tasks.workunit.client.0.smithi076.stdout:9/796: stat d1/c70 0 2024-03-23T15:44:57.799 INFO:tasks.workunit.client.0.smithi076.stdout:9/797: stat d1/d1f/d29/d2d/d37/f59 0 2024-03-23T15:44:57.799 INFO:tasks.workunit.client.0.smithi076.stdout:9/798: fdatasync d1/d2f/d3a/d64/fea 0 2024-03-23T15:44:57.800 INFO:tasks.workunit.client.0.smithi076.stdout:8/652: link d8/d2e/d3e/f8d d8/d2e/fe3 0 2024-03-23T15:44:57.800 INFO:tasks.workunit.client.0.smithi076.stdout:8/653: getdents d8/d26/d33/ddd 0 2024-03-23T15:44:57.807 INFO:tasks.workunit.client.0.smithi076.stdout:2/824: dwrite d1/d4/fb9 [0,4194304] 0 2024-03-23T15:44:57.807 INFO:tasks.workunit.client.0.smithi076.stdout:7/944: sync 2024-03-23T15:44:57.807 INFO:tasks.workunit.client.0.smithi076.stdout:6/549: rename d1/d81/l88 to d1/dd/df/d9b/db3/d8d/d60/lc9 0 2024-03-23T15:44:57.807 INFO:tasks.workunit.client.0.smithi076.stdout:4/734: rmdir d8/d3a/d69/d73/d76 39 2024-03-23T15:44:57.808 INFO:tasks.workunit.client.0.smithi076.stdout:6/550: mknod d1/dd/dbc/cca 0 2024-03-23T15:44:57.808 INFO:tasks.workunit.client.0.smithi076.stdout:9/799: rmdir d1/d2f/d3f/da0/dd5 39 2024-03-23T15:44:57.810 INFO:tasks.workunit.client.0.smithi076.stdout:2/825: rename d1/d2e/d8a/f9a to d1/d4/db/f119 0 2024-03-23T15:44:57.810 INFO:tasks.workunit.client.0.smithi076.stdout:8/654: creat d8/d6b/d8e/fe4 x:0 0 0 2024-03-23T15:44:57.810 INFO:tasks.workunit.client.0.smithi076.stdout:7/945: mkdir d8/d4e/d98/df6/d104/d18/dfd/d139 0 2024-03-23T15:44:57.811 INFO:tasks.workunit.client.0.smithi076.stdout:2/826: getdents d1/d4/d64/d82/d103/d79/dac 0 2024-03-23T15:44:57.812 INFO:tasks.workunit.client.0.smithi076.stdout:8/655: dread d8/d2e/d30/fa2 [0,4194304] 0 2024-03-23T15:44:57.812 INFO:tasks.workunit.client.0.smithi076.stdout:8/656: write d8/f23 [1399348,114449] 0 2024-03-23T15:44:57.812 INFO:tasks.workunit.client.0.smithi076.stdout:7/946: getdents d8/de/d99/dc4/df1 0 2024-03-23T15:44:57.813 INFO:tasks.workunit.client.0.smithi076.stdout:4/735: symlink d8/ldf 0 2024-03-23T15:44:57.813 INFO:tasks.workunit.client.0.smithi076.stdout:1/464: dwrite d0/d35/d40/d47/d17/f30 [0,4194304] 0 2024-03-23T15:44:57.814 INFO:tasks.workunit.client.0.smithi076.stdout:2/827: creat d1/d4/d64/d82/d103/d79/dac/dd8/dea/f11a x:0 0 0 2024-03-23T15:44:57.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:57 smithi192 ceph-mon[36653]: pgmap v51: 97 pgs: 97 active+clean; 656 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 91 MiB/s wr, 1.04k op/s 2024-03-23T15:44:57.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:57 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:57.830 INFO:tasks.workunit.client.0.smithi076.stdout:6/551: creat d1/d81/fcb x:0 0 0 2024-03-23T15:44:57.830 INFO:tasks.workunit.client.0.smithi076.stdout:6/552: chown d1/dd/df/d5b/l40 0 1 2024-03-23T15:44:57.832 INFO:tasks.workunit.client.0.smithi076.stdout:6/553: read d1/d2/f6d [3776219,128818] 0 2024-03-23T15:44:57.832 INFO:tasks.workunit.client.0.smithi076.stdout:1/465: mkdir d0/d35/d40/d47/d8c/d99 0 2024-03-23T15:44:57.834 INFO:tasks.workunit.client.0.smithi076.stdout:6/554: creat d1/dd/df/fcc x:0 0 0 2024-03-23T15:44:57.838 INFO:tasks.workunit.client.0.smithi076.stdout:8/657: mknod d8/dd0/ce5 0 2024-03-23T15:44:57.838 INFO:tasks.workunit.client.0.smithi076.stdout:8/658: truncate d8/f11 1723589 0 2024-03-23T15:44:57.838 INFO:tasks.workunit.client.0.smithi076.stdout:8/659: truncate d8/f14 1445535 0 2024-03-23T15:44:57.838 INFO:tasks.workunit.client.0.smithi076.stdout:4/736: mkdir d8/d3a/d69/d73/d85/dd2/de0 0 2024-03-23T15:44:57.838 INFO:tasks.workunit.client.0.smithi076.stdout:2/828: creat d1/d4/d64/d1d/dbc/dff/f11b x:0 0 0 2024-03-23T15:44:57.839 INFO:tasks.workunit.client.0.smithi076.stdout:2/829: truncate d1/d4/d64/d1b/f6c 4691823 0 2024-03-23T15:44:57.839 INFO:tasks.workunit.client.0.smithi076.stdout:2/830: creat d1/d4/d52/f11c x:0 0 0 2024-03-23T15:44:57.839 INFO:tasks.workunit.client.0.smithi076.stdout:8/660: stat d8/d2e/c2f 0 2024-03-23T15:44:57.839 INFO:tasks.workunit.client.0.smithi076.stdout:4/737: getdents d8/d3a/d18/d52 0 2024-03-23T15:44:57.839 INFO:tasks.workunit.client.0.smithi076.stdout:4/738: write d8/d3a/d18/d1b/f67 [670254,51014] 0 2024-03-23T15:44:57.840 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:57 smithi076 ceph-mon[32176]: pgmap v51: 97 pgs: 97 active+clean; 656 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 91 MiB/s wr, 1.04k op/s 2024-03-23T15:44:57.840 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:57 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:57.844 INFO:tasks.workunit.client.0.smithi076.stdout:1/466: dwrite d0/d35/d82/f89 [0,4194304] 0 2024-03-23T15:44:57.844 INFO:tasks.workunit.client.0.smithi076.stdout:2/831: symlink d1/d4/d64/d1d/dbc/l11d 0 2024-03-23T15:44:57.844 INFO:tasks.workunit.client.0.smithi076.stdout:8/661: creat d8/fe6 x:0 0 0 2024-03-23T15:44:57.844 INFO:tasks.workunit.client.0.smithi076.stdout:8/662: fdatasync d8/d2e/d30/d69/f81 0 2024-03-23T15:44:57.844 INFO:tasks.workunit.client.0.smithi076.stdout:8/663: truncate d8/d2e/d30/f66 114112 0 2024-03-23T15:44:57.845 INFO:tasks.workunit.client.0.smithi076.stdout:6/555: dwrite d1/dd/df/fa7 [0,4194304] 0 2024-03-23T15:44:57.845 INFO:tasks.workunit.client.0.smithi076.stdout:6/556: truncate d1/dd/df/d9b/db3/d8d/f77 1702258 0 2024-03-23T15:44:57.845 INFO:tasks.workunit.client.0.smithi076.stdout:4/739: rename d8/d3a/d3b/cd1 to d8/d9/d1d/ce1 0 2024-03-23T15:44:57.845 INFO:tasks.workunit.client.0.smithi076.stdout:9/800: rmdir d1/d1f/d82/d9c 39 2024-03-23T15:44:57.845 INFO:tasks.workunit.client.0.smithi076.stdout:1/467: dread d0/f5e [0,4194304] 0 2024-03-23T15:44:57.845 INFO:tasks.workunit.client.0.smithi076.stdout:4/740: symlink d8/d3a/d69/d73/d76/db4/le2 0 2024-03-23T15:44:57.845 INFO:tasks.workunit.client.0.smithi076.stdout:9/801: getdents d1/d2f/d9f 0 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/802: getdents d1/d2f/d3f/da0 0 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/803: chown d1/d1f/d46/d4a 208511 1 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/804: chown d1/d2f/d3a/d64/ced 1 1 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/805: stat d1/d1f/d29/d7f/ca2 0 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/806: getdents d1/d2f/d3f/da0 0 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/807: chown d1/d2f/d3a/d53/lb1 267775267 1 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:9/808: stat d1/d1f/d46/d83/dad/cf2 0 2024-03-23T15:44:57.846 INFO:tasks.workunit.client.0.smithi076.stdout:2/832: mknod d1/d4/d64/d1d/c11e 0 2024-03-23T15:44:57.848 INFO:tasks.workunit.client.0.smithi076.stdout:8/664: dwrite d8/d2e/d30/f6a [0,4194304] 0 2024-03-23T15:44:57.851 INFO:tasks.workunit.client.0.smithi076.stdout:8/665: unlink d8/f57 0 2024-03-23T15:44:57.852 INFO:tasks.workunit.client.0.smithi076.stdout:8/666: dread - d8/d2e/d30/d64/fc0 zero size 2024-03-23T15:44:57.852 INFO:tasks.workunit.client.0.smithi076.stdout:8/667: write d8/d2e/d30/fc3 [879668,103058] 0 2024-03-23T15:44:57.854 INFO:tasks.workunit.client.0.smithi076.stdout:8/668: dread d8/d26/d33/d8c/fb5 [0,4194304] 0 2024-03-23T15:44:57.864 INFO:tasks.workunit.client.0.smithi076.stdout:9/809: dwrite d1/d1f/d46/d83/dad/fc5 [0,4194304] 0 2024-03-23T15:44:57.865 INFO:tasks.workunit.client.0.smithi076.stdout:6/557: rename d1/d81/dbd to d1/dd/df/d5b/dcd 0 2024-03-23T15:44:57.865 INFO:tasks.workunit.client.0.smithi076.stdout:6/558: fdatasync d1/d4/f86 0 2024-03-23T15:44:57.865 INFO:tasks.workunit.client.0.smithi076.stdout:9/810: creat d1/d1f/d29/d2d/d37/f105 x:0 0 0 2024-03-23T15:44:57.868 INFO:tasks.workunit.client.0.smithi076.stdout:1/468: dwrite d0/d3c/d6e/f80 [0,4194304] 0 2024-03-23T15:44:57.871 INFO:tasks.workunit.client.0.smithi076.stdout:9/811: rmdir d1/d1f/d82 39 2024-03-23T15:44:57.873 INFO:tasks.workunit.client.0.smithi076.stdout:1/469: creat d0/d2/f9a x:0 0 0 2024-03-23T15:44:57.874 INFO:tasks.workunit.client.0.smithi076.stdout:4/741: dwrite d8/d3a/d69/d73/d76/f77 [0,4194304] 0 2024-03-23T15:44:57.876 INFO:tasks.workunit.client.0.smithi076.stdout:9/812: creat d1/d2f/d3f/da0/f106 x:0 0 0 2024-03-23T15:44:57.876 INFO:tasks.workunit.client.0.smithi076.stdout:9/813: write d1/d1f/d46/d83/dad/fd3 [945636,98119] 0 2024-03-23T15:44:57.876 INFO:tasks.workunit.client.0.smithi076.stdout:1/470: link d0/d35/d40/d47/c2f d0/d35/d40/d47/d74/d83/c9b 0 2024-03-23T15:44:57.876 INFO:tasks.workunit.client.0.smithi076.stdout:4/742: truncate d8/f66 1646971 0 2024-03-23T15:44:57.878 INFO:tasks.workunit.client.0.smithi076.stdout:9/814: truncate d1/d2f/d3a/d64/f85 2712475 0 2024-03-23T15:44:57.878 INFO:tasks.workunit.client.0.smithi076.stdout:9/815: write d1/d1f/d46/d4a/fe1 [457733,109628] 0 2024-03-23T15:44:57.879 INFO:tasks.workunit.client.0.smithi076.stdout:8/669: dwrite d8/f2a [0,4194304] 0 2024-03-23T15:44:57.879 INFO:tasks.workunit.client.0.smithi076.stdout:8/670: dread d8/d2e/d30/f32 [0,4194304] 0 2024-03-23T15:44:57.881 INFO:tasks.workunit.client.0.smithi076.stdout:6/559: dwrite d1/d4/d10/f78 [0,4194304] 0 2024-03-23T15:44:57.881 INFO:tasks.workunit.client.0.smithi076.stdout:1/471: dread d0/d35/d40/d47/d17/d20/f46 [0,4194304] 0 2024-03-23T15:44:57.881 INFO:tasks.workunit.client.0.smithi076.stdout:8/671: unlink d8/d4f/c60 0 2024-03-23T15:44:57.883 INFO:tasks.workunit.client.0.smithi076.stdout:1/472: creat d0/d35/d40/d47/d74/d83/f9c x:0 0 0 2024-03-23T15:44:57.883 INFO:tasks.workunit.client.0.smithi076.stdout:8/672: mknod d8/dd0/ce7 0 2024-03-23T15:44:57.883 INFO:tasks.workunit.client.0.smithi076.stdout:8/673: truncate d8/d26/d27/d56/fba 886896 0 2024-03-23T15:44:57.883 INFO:tasks.workunit.client.0.smithi076.stdout:8/674: dread - d8/d2e/d30/d69/f81 zero size 2024-03-23T15:44:57.883 INFO:tasks.workunit.client.0.smithi076.stdout:8/675: truncate d8/d26/d27/fa5 528610 0 2024-03-23T15:44:57.884 INFO:tasks.workunit.client.0.smithi076.stdout:8/676: chown d8/d2e/d3e/l4c 30 1 2024-03-23T15:44:57.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:57 smithi183 ceph-mon[36296]: pgmap v51: 97 pgs: 97 active+clean; 656 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 91 MiB/s wr, 1.04k op/s 2024-03-23T15:44:57.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:57 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:44:57.885 INFO:tasks.workunit.client.0.smithi076.stdout:6/560: rmdir d1/d4/d10 39 2024-03-23T15:44:57.885 INFO:tasks.workunit.client.0.smithi076.stdout:8/677: creat d8/d26/d33/fe8 x:0 0 0 2024-03-23T15:44:57.887 INFO:tasks.workunit.client.0.smithi076.stdout:8/678: creat d8/d6b/d8e/fe9 x:0 0 0 2024-03-23T15:44:57.887 INFO:tasks.workunit.client.0.smithi076.stdout:8/679: readlink d8/l12 0 2024-03-23T15:44:57.888 INFO:tasks.workunit.client.0.smithi076.stdout:8/680: getdents d8/d2e/d30/d69/d93 0 2024-03-23T15:44:57.891 INFO:tasks.workunit.client.0.smithi076.stdout:9/816: dwrite d1/d2f/d3a/d74/fb3 [0,4194304] 0 2024-03-23T15:44:57.891 INFO:tasks.workunit.client.0.smithi076.stdout:9/817: readlink d1/d1f/d29/d7f/lc4 0 2024-03-23T15:44:57.891 INFO:tasks.workunit.client.0.smithi076.stdout:9/818: stat d1/d1f/d29/d2d/d37/f105 0 2024-03-23T15:44:57.895 INFO:tasks.workunit.client.0.smithi076.stdout:9/819: creat d1/d1f/f107 x:0 0 0 2024-03-23T15:44:57.895 INFO:tasks.workunit.client.0.smithi076.stdout:9/820: write d1/d2f/d3a/fe2 [467415,43686] 0 2024-03-23T15:44:57.895 INFO:tasks.workunit.client.0.smithi076.stdout:9/821: readlink d1/d1f/d46/d4a/l99 0 2024-03-23T15:44:57.895 INFO:tasks.workunit.client.0.smithi076.stdout:9/822: mknod d1/dfd/c108 0 2024-03-23T15:44:57.913 INFO:tasks.workunit.client.0.smithi076.stdout:9/823: dwrite d1/d1f/d29/d2d/d37/d47/f52 [0,4194304] 0 2024-03-23T15:44:57.920 INFO:tasks.workunit.client.0.smithi076.stdout:8/681: mkdir d8/d26/d27/da9/dea 0 2024-03-23T15:44:57.921 INFO:tasks.workunit.client.0.smithi076.stdout:8/682: mkdir d8/d2e/d3e/deb 0 2024-03-23T15:44:57.921 INFO:tasks.workunit.client.0.smithi076.stdout:8/683: chown d8/d2e/d30/fc3 2 1 2024-03-23T15:44:57.921 INFO:tasks.workunit.client.0.smithi076.stdout:8/684: fdatasync d8/d89/db2/fc2 0 2024-03-23T15:44:57.921 INFO:tasks.workunit.client.0.smithi076.stdout:8/685: write d8/d6b/fdc [331214,73369] 0 2024-03-23T15:44:57.930 INFO:tasks.workunit.client.0.smithi076.stdout:9/824: dwrite d1/d1f/d46/f80 [0,4194304] 0 2024-03-23T15:44:57.932 INFO:tasks.workunit.client.0.smithi076.stdout:8/686: write d8/fe [2486639,59523] 0 2024-03-23T15:44:57.932 INFO:tasks.workunit.client.0.smithi076.stdout:1/473: dwrite d0/d2/f48 [0,4194304] 0 2024-03-23T15:44:57.932 INFO:tasks.workunit.client.0.smithi076.stdout:6/561: dread d1/dd/df/f66 [0,4194304] 0 2024-03-23T15:44:57.932 INFO:tasks.workunit.client.0.smithi076.stdout:2/833: rename d1/d4/d64/d1d/dbc/fdf to d1/d4/d64/d82/d103/d79/dac/dd8/f11f 0 2024-03-23T15:44:57.933 INFO:tasks.workunit.client.0.smithi076.stdout:8/687: creat d8/d2e/d3e/deb/fec x:0 0 0 2024-03-23T15:44:57.933 INFO:tasks.workunit.client.0.smithi076.stdout:8/688: readlink d8/d26/d33/l74 0 2024-03-23T15:44:57.933 INFO:tasks.workunit.client.0.smithi076.stdout:8/689: readlink d8/d26/d33/l42 0 2024-03-23T15:44:57.933 INFO:tasks.workunit.client.0.smithi076.stdout:6/562: mkdir d1/dd/df/dce 0 2024-03-23T15:44:57.933 INFO:tasks.workunit.client.0.smithi076.stdout:2/834: creat d1/d4/d64/d82/d103/f120 x:0 0 0 2024-03-23T15:44:57.935 INFO:tasks.workunit.client.0.smithi076.stdout:8/690: creat d8/d2e/d30/d69/dd3/fed x:0 0 0 2024-03-23T15:44:57.935 INFO:tasks.workunit.client.0.smithi076.stdout:8/691: stat d8/d2e/d30/d69/f71 0 2024-03-23T15:44:57.944 INFO:tasks.workunit.client.0.smithi076.stdout:1/474: unlink d0/f5e 0 2024-03-23T15:44:57.946 INFO:tasks.workunit.client.0.smithi076.stdout:6/563: dwrite d1/dd/d2a/d23/f2d [0,4194304] 0 2024-03-23T15:44:57.947 INFO:tasks.workunit.client.0.smithi076.stdout:4/743: rename d8/d3a/d18/d75/d8d to d8/de3 0 2024-03-23T15:44:57.950 INFO:tasks.workunit.client.0.smithi076.stdout:2/835: dwrite d1/d4/d52/d6e/fcf [0,4194304] 0 2024-03-23T15:44:57.951 INFO:tasks.workunit.client.0.smithi076.stdout:9/825: write d1/d2f/d3a/d53/f61 [7993777,6502] 0 2024-03-23T15:44:57.952 INFO:tasks.workunit.client.0.smithi076.stdout:6/564: rename d1/d81/f95 to d1/d4/fcf 0 2024-03-23T15:44:57.953 INFO:tasks.workunit.client.0.smithi076.stdout:9/826: dread d1/d2f/fbc [0,4194304] 0 2024-03-23T15:44:57.953 INFO:tasks.workunit.client.0.smithi076.stdout:9/827: truncate d1/d1f/f107 371385 0 2024-03-23T15:44:57.953 INFO:tasks.workunit.client.0.smithi076.stdout:2/836: mknod d1/d4/d64/d82/c121 0 2024-03-23T15:44:57.955 INFO:tasks.workunit.client.0.smithi076.stdout:3/551: sync 2024-03-23T15:44:57.955 INFO:tasks.workunit.client.0.smithi076.stdout:0/767: sync 2024-03-23T15:44:57.956 INFO:tasks.workunit.client.0.smithi076.stdout:4/744: rmdir d8/d3a/d69/d73/d85/da9 39 2024-03-23T15:44:57.958 INFO:tasks.workunit.client.0.smithi076.stdout:5/779: sync 2024-03-23T15:44:57.958 INFO:tasks.workunit.client.0.smithi076.stdout:7/947: sync 2024-03-23T15:44:57.958 INFO:tasks.workunit.client.0.smithi076.stdout:7/948: readlink d8/d4e/d8d/lb8 0 2024-03-23T15:44:57.959 INFO:tasks.workunit.client.0.smithi076.stdout:4/745: dread d8/f56 [0,4194304] 0 2024-03-23T15:44:57.960 INFO:tasks.workunit.client.0.smithi076.stdout:1/475: dwrite d0/d3c/d6e/f76 [4194304,4194304] 0 2024-03-23T15:44:57.960 INFO:tasks.workunit.client.0.smithi076.stdout:5/780: creat d1a/d33/d3a/d67/d84/f10e x:0 0 0 2024-03-23T15:44:57.960 INFO:tasks.workunit.client.0.smithi076.stdout:7/949: write d8/d4e/d98/df6/d104/d40/d88/f57 [5642166,45117] 0 2024-03-23T15:44:57.961 INFO:tasks.workunit.client.0.smithi076.stdout:3/552: mknod d5/d17/d1c/d31/d61/cb6 0 2024-03-23T15:44:57.961 INFO:tasks.workunit.client.0.smithi076.stdout:6/565: mkdir d1/dd/df/d5b/dcd/dd0 0 2024-03-23T15:44:57.961 INFO:tasks.workunit.client.0.smithi076.stdout:3/553: read d5/f13 [1732180,80342] 0 2024-03-23T15:44:57.961 INFO:tasks.workunit.client.0.smithi076.stdout:4/746: mkdir d8/d3a/d69/de4 0 2024-03-23T15:44:57.962 INFO:tasks.workunit.client.0.smithi076.stdout:4/747: readlink d8/d3a/d18/d1b/l2c 0 2024-03-23T15:44:57.963 INFO:tasks.workunit.client.0.smithi076.stdout:7/950: creat d8/d4e/d98/df6/d104/d18/dcc/f13a x:0 0 0 2024-03-23T15:44:57.964 INFO:tasks.workunit.client.0.smithi076.stdout:7/951: fdatasync d8/d4e/f87 0 2024-03-23T15:44:57.964 INFO:tasks.workunit.client.0.smithi076.stdout:7/952: write d8/de/d99/fff [1098281,104672] 0 2024-03-23T15:44:57.965 INFO:tasks.workunit.client.0.smithi076.stdout:5/781: dread d1a/d7e/fab [0,4194304] 0 2024-03-23T15:44:57.965 INFO:tasks.workunit.client.0.smithi076.stdout:5/782: chown d1a/d33/d3a/f8a 28 1 2024-03-23T15:44:57.965 INFO:tasks.workunit.client.0.smithi076.stdout:5/783: write d1a/f31 [514026,91600] 0 2024-03-23T15:44:57.965 INFO:tasks.workunit.client.0.smithi076.stdout:1/476: link d0/d2/f9a d0/f9d 0 2024-03-23T15:44:57.968 INFO:tasks.workunit.client.0.smithi076.stdout:3/554: mknod d5/d17/cb7 0 2024-03-23T15:44:57.969 INFO:tasks.workunit.client.0.smithi076.stdout:4/748: rmdir d8 39 2024-03-23T15:44:57.969 INFO:tasks.workunit.client.0.smithi076.stdout:3/555: fsync d5/d35/d4c/fa6 0 2024-03-23T15:44:57.969 INFO:tasks.workunit.client.0.smithi076.stdout:7/953: mknod d8/d4e/d98/df6/d104/d3a/dd2/c13b 0 2024-03-23T15:44:57.970 INFO:tasks.workunit.client.0.smithi076.stdout:7/954: truncate d8/d4e/d8d/f11c 83468 0 2024-03-23T15:44:57.971 INFO:tasks.workunit.client.0.smithi076.stdout:4/749: truncate d8/ff 11862177 0 2024-03-23T15:44:57.971 INFO:tasks.workunit.client.0.smithi076.stdout:3/556: creat d5/d17/d1c/d31/d61/fb8 x:0 0 0 2024-03-23T15:44:57.971 INFO:tasks.workunit.client.0.smithi076.stdout:3/557: fdatasync d5/d17/d1c/d32/d39/d59/f60 0 2024-03-23T15:44:57.972 INFO:tasks.workunit.client.0.smithi076.stdout:3/558: truncate d5/d17/d1c/d32/d4a/f72 1199596 0 2024-03-23T15:44:57.981 INFO:tasks.workunit.client.0.smithi076.stdout:4/750: write d8/f3f [3776200,127490] 0 2024-03-23T15:44:57.982 INFO:tasks.workunit.client.0.smithi076.stdout:2/837: dwrite d1/d4/d64/d82/d103/f38 [0,4194304] 0 2024-03-23T15:44:57.984 INFO:tasks.workunit.client.0.smithi076.stdout:0/768: dwrite d2/de/f13 [0,4194304] 0 2024-03-23T15:44:57.986 INFO:tasks.workunit.client.0.smithi076.stdout:4/751: creat d8/d3a/d69/d73/d85/dd2/de0/fe5 x:0 0 0 2024-03-23T15:44:57.997 INFO:tasks.workunit.client.0.smithi076.stdout:6/566: dwrite d1/dd/df/d9b/db3/d8d/f9f [0,4194304] 0 2024-03-23T15:44:57.997 INFO:tasks.workunit.client.0.smithi076.stdout:6/567: stat d1/dd/df/fa4 0 2024-03-23T15:44:57.999 INFO:tasks.workunit.client.0.smithi076.stdout:6/568: mkdir d1/dd1 0 2024-03-23T15:44:58.000 INFO:tasks.workunit.client.0.smithi076.stdout:0/769: truncate d2/d8/d2d/f3f 2078040 0 2024-03-23T15:44:58.005 INFO:tasks.workunit.client.0.smithi076.stdout:9/828: dread d1/d1f/d46/d4a/d5a/daa/ffc [0,4194304] 0 2024-03-23T15:44:58.006 INFO:tasks.workunit.client.0.smithi076.stdout:9/829: stat d1/d1f/d46/d83/l58 0 2024-03-23T15:44:58.007 INFO:tasks.workunit.client.0.smithi076.stdout:7/955: rename d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/f11b to d8/d4e/d98/df6/d104/d3a/dbf/f13c 0 2024-03-23T15:44:58.008 INFO:tasks.workunit.client.0.smithi076.stdout:2/838: read d1/d4/d64/f18 [2249977,45520] 0 2024-03-23T15:44:58.008 INFO:tasks.workunit.client.0.smithi076.stdout:9/830: creat d1/d2f/d3a/f109 x:0 0 0 2024-03-23T15:44:58.009 INFO:tasks.workunit.client.0.smithi076.stdout:4/752: dwrite d8/d3a/d69/fbf [4194304,4194304] 0 2024-03-23T15:44:58.009 INFO:tasks.workunit.client.0.smithi076.stdout:4/753: write d8/d3a/d18/d52/f79 [4391131,79768] 0 2024-03-23T15:44:58.010 INFO:tasks.workunit.client.0.smithi076.stdout:9/831: creat d1/d1f/d46/d4a/f10a x:0 0 0 2024-03-23T15:44:58.014 INFO:tasks.workunit.client.0.smithi076.stdout:0/770: dread d2/de/d10/d43/f79 [0,4194304] 0 2024-03-23T15:44:58.014 INFO:tasks.workunit.client.0.smithi076.stdout:1/477: dwrite d0/d3c/f57 [0,4194304] 0 2024-03-23T15:44:58.014 INFO:tasks.workunit.client.0.smithi076.stdout:0/771: chown d2/de/d10/d3a/f5c 62 1 2024-03-23T15:44:58.014 INFO:tasks.workunit.client.0.smithi076.stdout:1/478: chown d0/d35/d40/d47/d74/d83/f98 224692 1 2024-03-23T15:44:58.014 INFO:tasks.workunit.client.0.smithi076.stdout:3/559: rename d5/f14 to d5/d35/d4c/d5a/fb9 0 2024-03-23T15:44:58.015 INFO:tasks.workunit.client.0.smithi076.stdout:7/956: creat d8/d4e/d98/df6/d104/d3a/dd2/f13d x:0 0 0 2024-03-23T15:44:58.015 INFO:tasks.workunit.client.0.smithi076.stdout:7/957: dread - d8/de/d99/f138 zero size 2024-03-23T15:44:58.015 INFO:tasks.workunit.client.0.smithi076.stdout:7/958: getdents d8/d4e/d98/df6/d104/d24/d91/d96/d7a 0 2024-03-23T15:44:58.015 INFO:tasks.workunit.client.0.smithi076.stdout:7/959: dread - d8/de/d99/f138 zero size 2024-03-23T15:44:58.015 INFO:tasks.workunit.client.0.smithi076.stdout:5/784: dwrite d1a/d33/d3a/d67/da8/dc4/ffe [0,4194304] 0 2024-03-23T15:44:58.017 INFO:tasks.workunit.client.0.smithi076.stdout:7/960: rmdir d8/de/d48/dd8 39 2024-03-23T15:44:58.017 INFO:tasks.workunit.client.0.smithi076.stdout:2/839: dread d1/d2e/f42 [0,4194304] 0 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:2/840: readlink l0 0 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:2/841: chown d1/d4/db/d15/l74 0 1 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:2/842: fsync d1/d4/d64/d82/d103/d79/fb5 0 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:2/843: chown d1/d4/d64/d1d/dbc/de6 0 1 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:2/844: fdatasync d1/d4/db/d15/d29/d7d/f9d 0 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:5/785: creat d1a/d33/d36/f10f x:0 0 0 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:4/754: link d8/f42 d8/d3a/d69/fe6 0 2024-03-23T15:44:58.018 INFO:tasks.workunit.client.0.smithi076.stdout:7/961: creat d8/d4e/d98/df6/d104/f13e x:0 0 0 2024-03-23T15:44:58.019 INFO:tasks.workunit.client.0.smithi076.stdout:1/479: write d0/d35/d40/d47/f4e [2894625,60280] 0 2024-03-23T15:44:58.019 INFO:tasks.workunit.client.0.smithi076.stdout:5/786: mkdir d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/d110 0 2024-03-23T15:44:58.019 INFO:tasks.workunit.client.0.smithi076.stdout:5/787: write d1a/f98 [124416,6985] 0 2024-03-23T15:44:58.019 INFO:tasks.workunit.client.0.smithi076.stdout:7/962: mknod d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/d12c/c13f 0 2024-03-23T15:44:58.020 INFO:tasks.workunit.client.0.smithi076.stdout:7/963: truncate d8/de/d99/f138 755303 0 2024-03-23T15:44:58.020 INFO:tasks.workunit.client.0.smithi076.stdout:0/772: mkdir d2/ded 0 2024-03-23T15:44:58.021 INFO:tasks.workunit.client.0.smithi076.stdout:2/845: rmdir d1/d4/d64/d7c 39 2024-03-23T15:44:58.021 INFO:tasks.workunit.client.0.smithi076.stdout:2/846: stat d1/d4/db/d15/d29/d7d/dbd 0 2024-03-23T15:44:58.021 INFO:tasks.workunit.client.0.smithi076.stdout:6/569: rename d1/d4/d10/f78 to d1/d81/fd2 0 2024-03-23T15:44:58.022 INFO:tasks.workunit.client.0.smithi076.stdout:4/755: mkdir d8/d3a/d69/de7 0 2024-03-23T15:44:58.024 INFO:tasks.workunit.client.0.smithi076.stdout:4/756: dread - d8/d3a/d69/d73/d76/f83 zero size 2024-03-23T15:44:58.024 INFO:tasks.workunit.client.0.smithi076.stdout:4/757: fdatasync d8/d80/fc0 0 2024-03-23T15:44:58.024 INFO:tasks.workunit.client.0.smithi076.stdout:4/758: write d8/d3a/d69/d73/d85/da9/fc2 [975927,91074] 0 2024-03-23T15:44:58.024 INFO:tasks.workunit.client.0.smithi076.stdout:6/570: creat d1/dd/d2a/fd3 x:0 0 0 2024-03-23T15:44:58.025 INFO:tasks.workunit.client.0.smithi076.stdout:7/964: unlink d8/d4e/d98/df6/d104/d40/d88/f33 0 2024-03-23T15:44:58.025 INFO:tasks.workunit.client.0.smithi076.stdout:1/480: creat d0/d2/d62/f9e x:0 0 0 2024-03-23T15:44:58.026 INFO:tasks.workunit.client.0.smithi076.stdout:7/965: rmdir d8/de/de8 39 2024-03-23T15:44:58.027 INFO:tasks.workunit.client.0.smithi076.stdout:3/560: write d5/d17/d1c/d31/fa1 [3694326,101328] 0 2024-03-23T15:44:58.027 INFO:tasks.workunit.client.0.smithi076.stdout:6/571: dread d1/d2/f87 [0,4194304] 0 2024-03-23T15:44:58.028 INFO:tasks.workunit.client.0.smithi076.stdout:7/966: unlink d8/d4e/fd0 0 2024-03-23T15:44:58.029 INFO:tasks.workunit.client.0.smithi076.stdout:9/832: rename d1/d2f/d3a/f103 to d1/d1f/f10b 0 2024-03-23T15:44:58.029 INFO:tasks.workunit.client.0.smithi076.stdout:7/967: write d8/d4e/d98/df6/d104/d40/d5d/f3e [412003,102158] 0 2024-03-23T15:44:58.030 INFO:tasks.workunit.client.0.smithi076.stdout:4/759: fsync d8/d3a/d18/d52/f79 0 2024-03-23T15:44:58.031 INFO:tasks.workunit.client.0.smithi076.stdout:0/773: rename d2/de/d10/d12/f23 to d2/d8/d2d/fee 0 2024-03-23T15:44:58.033 INFO:tasks.workunit.client.0.smithi076.stdout:9/833: symlink d1/d2f/d3f/da0/dd5/l10c 0 2024-03-23T15:44:58.033 INFO:tasks.workunit.client.0.smithi076.stdout:9/834: stat d1/d2f 0 2024-03-23T15:44:58.035 INFO:tasks.workunit.client.0.smithi076.stdout:1/481: dread d0/d35/f64 [0,4194304] 0 2024-03-23T15:44:58.037 INFO:tasks.workunit.client.0.smithi076.stdout:7/968: write d8/d4e/d98/df6/d104/d40/d88/fb2 [3517531,70495] 0 2024-03-23T15:44:58.038 INFO:tasks.workunit.client.0.smithi076.stdout:4/760: dread f6 [0,4194304] 0 2024-03-23T15:44:58.038 INFO:tasks.workunit.client.0.smithi076.stdout:5/788: dwrite d1a/d33/dc3/da6/fea [0,4194304] 0 2024-03-23T15:44:58.038 INFO:tasks.workunit.client.0.smithi076.stdout:6/572: write d1/d4/d10/f74 [450115,22959] 0 2024-03-23T15:44:58.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/774: rmdir d2/de/d71/dec 0 2024-03-23T15:44:58.039 INFO:tasks.workunit.client.0.smithi076.stdout:4/761: dread d8/d9/f2d [0,4194304] 0 2024-03-23T15:44:58.041 INFO:tasks.workunit.client.0.smithi076.stdout:9/835: rename d1/d2f/d3f/f49 to d1/d2f/d9f/f10d 0 2024-03-23T15:44:58.043 INFO:tasks.workunit.client.0.smithi076.stdout:6/573: mkdir d1/dd4 0 2024-03-23T15:44:58.045 INFO:tasks.workunit.client.0.smithi076.stdout:9/836: mknod d1/d2f/c10e 0 2024-03-23T15:44:58.045 INFO:tasks.workunit.client.0.smithi076.stdout:1/482: write d0/d3c/f57 [908479,125126] 0 2024-03-23T15:44:58.045 INFO:tasks.workunit.client.0.smithi076.stdout:1/483: dread - d0/d2/d62/f9e zero size 2024-03-23T15:44:58.047 INFO:tasks.workunit.client.0.smithi076.stdout:9/837: dread d1/d1f/d46/d83/dad/fd3 [0,4194304] 0 2024-03-23T15:44:58.047 INFO:tasks.workunit.client.0.smithi076.stdout:9/838: fsync d1/d20/f25 0 2024-03-23T15:44:58.048 INFO:tasks.workunit.client.0.smithi076.stdout:9/839: chown d1/d1f/d29 631361107 1 2024-03-23T15:44:58.048 INFO:tasks.workunit.client.0.smithi076.stdout:0/775: dread d2/d8/d52/f64 [0,4194304] 0 2024-03-23T15:44:58.049 INFO:tasks.workunit.client.0.smithi076.stdout:1/484: rmdir d0/d35/d40/d47/d74/d83 39 2024-03-23T15:44:58.049 INFO:tasks.workunit.client.0.smithi076.stdout:1/485: chown d0/d2/d91/l4c 3426 1 2024-03-23T15:44:58.051 INFO:tasks.workunit.client.0.smithi076.stdout:1/486: mknod d0/d35/d40/d47/d17/c9f 0 2024-03-23T15:44:58.053 INFO:tasks.workunit.client.0.smithi076.stdout:9/840: unlink d1/d2f/d3a/fee 0 2024-03-23T15:44:58.053 INFO:tasks.workunit.client.0.smithi076.stdout:1/487: truncate d0/d35/d82/f89 1769917 0 2024-03-23T15:44:58.055 INFO:tasks.workunit.client.0.smithi076.stdout:6/574: write d1/d2/f6d [5140438,39546] 0 2024-03-23T15:44:58.055 INFO:tasks.workunit.client.0.smithi076.stdout:6/575: readlink d1/d4/l15 0 2024-03-23T15:44:58.056 INFO:tasks.workunit.client.0.smithi076.stdout:6/576: write d1/dd/df/f5f [1001959,113974] 0 2024-03-23T15:44:58.056 INFO:tasks.workunit.client.0.smithi076.stdout:0/776: mkdir d2/de/d10/def 0 2024-03-23T15:44:58.057 INFO:tasks.workunit.client.0.smithi076.stdout:4/762: dwrite d8/d3a/d69/d73/d76/db4/fc5 [0,4194304] 0 2024-03-23T15:44:58.061 INFO:tasks.workunit.client.0.smithi076.stdout:1/488: dread d0/d35/d40/d47/d17/f30 [0,4194304] 0 2024-03-23T15:44:58.066 INFO:tasks.workunit.client.0.smithi076.stdout:4/763: symlink d8/dd7/le8 0 2024-03-23T15:44:58.066 INFO:tasks.workunit.client.0.smithi076.stdout:6/577: dread d1/dd/df/d5b/f99 [0,4194304] 0 2024-03-23T15:44:58.070 INFO:tasks.workunit.client.0.smithi076.stdout:9/841: dwrite d1/d20/f45 [0,4194304] 0 2024-03-23T15:44:58.073 INFO:tasks.workunit.client.0.smithi076.stdout:4/764: dread d8/d41/f54 [0,4194304] 0 2024-03-23T15:44:58.073 INFO:tasks.workunit.client.0.smithi076.stdout:9/842: mknod d1/d2f/d3a/d74/dd7/c10f 0 2024-03-23T15:44:58.075 INFO:tasks.workunit.client.0.smithi076.stdout:6/578: dread d1/dd/d2a/d23/f2d [0,4194304] 0 2024-03-23T15:44:58.075 INFO:tasks.workunit.client.0.smithi076.stdout:1/489: dread d0/f2e [4194304,4194304] 0 2024-03-23T15:44:58.076 INFO:tasks.workunit.client.0.smithi076.stdout:1/490: getdents d0/d35/d40/d47/d55 0 2024-03-23T15:44:58.076 INFO:tasks.workunit.client.0.smithi076.stdout:4/765: dread d8/d3a/d69/d73/d85/da9/fc2 [0,4194304] 0 2024-03-23T15:44:58.077 INFO:tasks.workunit.client.0.smithi076.stdout:1/491: creat d0/d3c/fa0 x:0 0 0 2024-03-23T15:44:58.078 INFO:tasks.workunit.client.0.smithi076.stdout:4/766: truncate d8/d3a/d18/f4d 281900 0 2024-03-23T15:44:58.078 INFO:tasks.workunit.client.0.smithi076.stdout:4/767: fsync d8/d3a/d3b/f4b 0 2024-03-23T15:44:58.079 INFO:tasks.workunit.client.0.smithi076.stdout:1/492: mkdir d0/d2/d70/da1 0 2024-03-23T15:44:58.079 INFO:tasks.workunit.client.0.smithi076.stdout:1/493: dread - d0/d35/d40/d47/d17/f8a zero size 2024-03-23T15:44:58.081 INFO:tasks.workunit.client.0.smithi076.stdout:4/768: mkdir d8/de3/db0/de9 0 2024-03-23T15:44:58.085 INFO:tasks.workunit.client.0.smithi076.stdout:7/969: rename d8/d4e/d98/df6/d104/d18/fc9 to d8/d4e/d98/df6/d104/d3a/dd2/f140 0 2024-03-23T15:44:58.090 INFO:tasks.workunit.client.0.smithi076.stdout:0/777: rename d2/d8/f65 to d2/d8/d2d/ff0 0 2024-03-23T15:44:58.090 INFO:tasks.workunit.client.0.smithi076.stdout:7/970: write d8/d4e/d98/df6/d104/d3a/f89 [2260452,56342] 0 2024-03-23T15:44:58.090 INFO:tasks.workunit.client.0.smithi076.stdout:7/971: dread - d8/d4e/d98/df6/d104/d40/d88/d5a/f95 zero size 2024-03-23T15:44:58.090 INFO:tasks.workunit.client.0.smithi076.stdout:7/972: fdatasync d8/d4e/d98/df6/f111 0 2024-03-23T15:44:58.090 INFO:tasks.workunit.client.0.smithi076.stdout:9/843: rename d1/d2f/d3f/f8b to d1/d2f/d3a/d74/da5/f110 0 2024-03-23T15:44:58.091 INFO:tasks.workunit.client.0.smithi076.stdout:0/778: mknod d2/d9a/cf1 0 2024-03-23T15:44:58.092 INFO:tasks.workunit.client.0.smithi076.stdout:0/779: write d2/de/d10/d12/da6/fd2 [89577,125847] 0 2024-03-23T15:44:58.092 INFO:tasks.workunit.client.0.smithi076.stdout:7/973: mknod d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/c141 0 2024-03-23T15:44:58.108 INFO:tasks.workunit.client.0.smithi076.stdout:0/780: dwrite d2/d1b/fc4 [0,4194304] 0 2024-03-23T15:44:58.110 INFO:tasks.workunit.client.0.smithi076.stdout:7/974: dwrite d8/d4e/d98/df6/d104/d40/d88/d5a/dde/d102/f122 [0,4194304] 0 2024-03-23T15:44:58.115 INFO:tasks.workunit.client.0.smithi076.stdout:6/579: rename d1/d4/fb9 to d1/d2/dc8/fd5 0 2024-03-23T15:44:58.116 INFO:tasks.workunit.client.0.smithi076.stdout:6/580: rmdir d1/dd/df/d9b/db3/d8d/db8 39 2024-03-23T15:44:58.117 INFO:tasks.workunit.client.0.smithi076.stdout:4/769: rename d8/d3a/d69/d73/d76/ldb to d8/d9/d1d/lea 0 2024-03-23T15:44:58.117 INFO:tasks.workunit.client.0.smithi076.stdout:6/581: getdents d1/dd/df/d9b 0 2024-03-23T15:44:58.118 INFO:tasks.workunit.client.0.smithi076.stdout:0/781: dread d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:58.119 INFO:tasks.workunit.client.0.smithi076.stdout:4/770: creat d8/de3/feb x:0 0 0 2024-03-23T15:44:58.120 INFO:tasks.workunit.client.0.smithi076.stdout:0/782: creat d2/ff2 x:0 0 0 2024-03-23T15:44:58.120 INFO:tasks.workunit.client.0.smithi076.stdout:7/975: read d8/d4e/d98/df6/d104/d18/dcc/f12a [3031422,122708] 0 2024-03-23T15:44:58.121 INFO:tasks.workunit.client.0.smithi076.stdout:4/771: write d8/d3a/d18/d1b/fa5 [716771,47556] 0 2024-03-23T15:44:58.121 INFO:tasks.workunit.client.0.smithi076.stdout:0/783: getdents d2/d8 0 2024-03-23T15:44:58.121 INFO:tasks.workunit.client.0.smithi076.stdout:7/976: getdents d8/d4e/d98/df6 0 2024-03-23T15:44:58.122 INFO:tasks.workunit.client.0.smithi076.stdout:4/772: creat d8/de3/fec x:0 0 0 2024-03-23T15:44:58.122 INFO:tasks.workunit.client.0.smithi076.stdout:7/977: truncate d8/f62 4131686 0 2024-03-23T15:44:58.122 INFO:tasks.workunit.client.0.smithi076.stdout:7/978: fdatasync d8/d4e/d98/df6/d104/d40/d5d/f101 0 2024-03-23T15:44:58.122 INFO:tasks.workunit.client.0.smithi076.stdout:0/784: readlink d2/de/d10/d12/l2b 0 2024-03-23T15:44:58.124 INFO:tasks.workunit.client.0.smithi076.stdout:4/773: mknod d8/d3a/d69/d73/d85/dd2/ced 0 2024-03-23T15:44:58.125 INFO:tasks.workunit.client.0.smithi076.stdout:4/774: creat d8/d3a/d18/d75/fee x:0 0 0 2024-03-23T15:44:58.127 INFO:tasks.workunit.client.0.smithi076.stdout:4/775: creat d8/d3a/d69/d73/d9a/fef x:0 0 0 2024-03-23T15:44:58.128 INFO:tasks.workunit.client.0.smithi076.stdout:4/776: creat d8/d9/d1d/ff0 x:0 0 0 2024-03-23T15:44:58.135 INFO:tasks.workunit.client.0.smithi076.stdout:0/785: dwrite d2/d9a/fd0 [0,4194304] 0 2024-03-23T15:44:58.135 INFO:tasks.workunit.client.0.smithi076.stdout:0/786: chown d2/d8/d52/cc1 1533585 1 2024-03-23T15:44:58.136 INFO:tasks.workunit.client.0.smithi076.stdout:9/844: rename d1/d1f/d46/d4a/d5a/daa/fcb to d1/d1f/d29/d7f/f111 0 2024-03-23T15:44:58.137 INFO:tasks.workunit.client.0.smithi076.stdout:4/777: dread d8/d3a/d69/fbf [4194304,4194304] 0 2024-03-23T15:44:58.138 INFO:tasks.workunit.client.0.smithi076.stdout:4/778: fsync d8/d9/f12 0 2024-03-23T15:44:58.138 INFO:tasks.workunit.client.0.smithi076.stdout:0/787: dread d2/de/d10/f68 [0,4194304] 0 2024-03-23T15:44:58.138 INFO:tasks.workunit.client.0.smithi076.stdout:6/582: rename d1/dd/df/fcc to d1/dd/df/d5b/d62/dc7/fd6 0 2024-03-23T15:44:58.141 INFO:tasks.workunit.client.0.smithi076.stdout:0/788: mkdir d2/de/d10/d43/df3 0 2024-03-23T15:44:58.141 INFO:tasks.workunit.client.0.smithi076.stdout:0/789: write d2/de/d10/d3a/fce [15963,98019] 0 2024-03-23T15:44:58.141 INFO:tasks.workunit.client.0.smithi076.stdout:0/790: chown d2/d1b/l8b 31125916 1 2024-03-23T15:44:58.142 INFO:tasks.workunit.client.0.smithi076.stdout:4/779: rename d8/d3a/d69/d73/d85/dd2/ced to d8/d9/cf1 0 2024-03-23T15:44:58.142 INFO:tasks.workunit.client.0.smithi076.stdout:4/780: write d8/d9/d32/f90 [296627,71966] 0 2024-03-23T15:44:58.142 INFO:tasks.workunit.client.0.smithi076.stdout:4/781: chown l7 832220388 1 2024-03-23T15:44:58.142 INFO:tasks.workunit.client.0.smithi076.stdout:4/782: write d8/d80/fc0 [6067966,16763] 0 2024-03-23T15:44:58.145 INFO:tasks.workunit.client.0.smithi076.stdout:0/791: rmdir d2/de/d10/d3a 39 2024-03-23T15:44:58.149 INFO:tasks.workunit.client.0.smithi076.stdout:4/783: dread d8/d9/d1d/fca [4194304,4194304] 0 2024-03-23T15:44:58.151 INFO:tasks.workunit.client.0.smithi076.stdout:4/784: creat d8/d3a/d69/de7/ff2 x:0 0 0 2024-03-23T15:44:58.155 INFO:tasks.workunit.client.0.smithi076.stdout:4/785: creat d8/d3a/d69/d73/d9a/ff3 x:0 0 0 2024-03-23T15:44:58.156 INFO:tasks.workunit.client.0.smithi076.stdout:4/786: rmdir d8/d3a 39 2024-03-23T15:44:58.156 INFO:tasks.workunit.client.0.smithi076.stdout:4/787: chown d8/d3a/d69/d73/d9a/ff3 709783505 1 2024-03-23T15:44:58.158 INFO:tasks.workunit.client.0.smithi076.stdout:6/583: dwrite d1/d4/d10/d39/fb1 [0,4194304] 0 2024-03-23T15:44:58.158 INFO:tasks.workunit.client.0.smithi076.stdout:6/584: readlink d1/dd/l58 0 2024-03-23T15:44:58.158 INFO:tasks.workunit.client.0.smithi076.stdout:6/585: getdents d1/dd/dbc 0 2024-03-23T15:44:58.158 INFO:tasks.workunit.client.0.smithi076.stdout:6/586: fdatasync d1/dd/d2a/d23/f2d 0 2024-03-23T15:44:58.159 INFO:tasks.workunit.client.0.smithi076.stdout:0/792: dread d2/d1b/d77/fd6 [0,4194304] 0 2024-03-23T15:44:58.165 INFO:tasks.workunit.client.0.smithi076.stdout:0/793: truncate d2/de/d10/d43/f63 3787584 0 2024-03-23T15:44:58.166 INFO:tasks.workunit.client.0.smithi076.stdout:9/845: dwrite d1/d1f/d29/d7f/ffb [0,4194304] 0 2024-03-23T15:44:58.166 INFO:tasks.workunit.client.0.smithi076.stdout:9/846: write d1/d2f/d3f/f92 [4559402,48917] 0 2024-03-23T15:44:58.168 INFO:tasks.workunit.client.0.smithi076.stdout:9/847: symlink d1/d1f/d82/l112 0 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:4/788: rename d8/d3a/d69/d73/d85 to d8/d3a/d69/d73/df4 0 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:0/794: write d2/d8/d32/f48 [250582,23584] 0 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:4/789: truncate d8/d3a/fd4 466503 0 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:4/790: chown d8/d9/d1d/dc9/fd6 2860 1 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:0/795: write d2/d8/f28 [1342926,114864] 0 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:9/848: mknod d1/d2f/d3f/da0/c113 0 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:9/849: chown d1/d1f/d29/d2d/d37/d47/cba 60291216 1 2024-03-23T15:44:58.170 INFO:tasks.workunit.client.0.smithi076.stdout:9/850: truncate d1/d2f/d3a/d64/fea 9575 0 2024-03-23T15:44:58.171 INFO:tasks.workunit.client.0.smithi076.stdout:4/791: mkdir d8/d3a/d69/d73/df4/da9/df5 0 2024-03-23T15:44:58.173 INFO:tasks.workunit.client.0.smithi076.stdout:6/587: truncate d1/d4/d10/d39/fb1 537689 0 2024-03-23T15:44:58.173 INFO:tasks.workunit.client.0.smithi076.stdout:6/588: write d1/dd/d2a/d23/f96 [1019426,128849] 0 2024-03-23T15:44:58.173 INFO:tasks.workunit.client.0.smithi076.stdout:6/589: stat d1/dd/df/d5b/d62/f7e 0 2024-03-23T15:44:58.179 INFO:tasks.workunit.client.0.smithi076.stdout:6/590: write d1/dd/df/d9b/db3/d8d/f9f [2963593,129568] 0 2024-03-23T15:44:58.181 INFO:tasks.workunit.client.0.smithi076.stdout:0/796: dwrite d2/de/f1a [0,4194304] 0 2024-03-23T15:44:58.184 INFO:tasks.workunit.client.0.smithi076.stdout:0/797: creat d2/de/d10/d3a/d47/d69/ff4 x:0 0 0 2024-03-23T15:44:58.185 INFO:tasks.workunit.client.0.smithi076.stdout:4/792: dwrite d8/d3a/d18/d52/f92 [0,4194304] 0 2024-03-23T15:44:58.194 INFO:tasks.workunit.client.0.smithi076.stdout:6/591: mknod d1/dd/df/d5b/dcd/cd7 0 2024-03-23T15:44:58.194 INFO:tasks.workunit.client.0.smithi076.stdout:6/592: fsync d1/f21 0 2024-03-23T15:44:58.197 INFO:tasks.workunit.client.0.smithi076.stdout:4/793: write d8/ff [10616779,46615] 0 2024-03-23T15:44:58.197 INFO:tasks.workunit.client.0.smithi076.stdout:0/798: dwrite d2/de/d71/fd4 [0,4194304] 0 2024-03-23T15:44:58.197 INFO:tasks.workunit.client.0.smithi076.stdout:0/799: chown d2/de/f13 10619302 1 2024-03-23T15:44:58.197 INFO:tasks.workunit.client.0.smithi076.stdout:6/593: creat d1/dd/df/d5b/dcd/dd0/fd8 x:0 0 0 2024-03-23T15:44:58.198 INFO:tasks.workunit.client.0.smithi076.stdout:0/800: mkdir d2/de/d10/df5 0 2024-03-23T15:44:58.199 INFO:tasks.workunit.client.0.smithi076.stdout:0/801: chown d2/d9a/fbc 900 1 2024-03-23T15:44:58.199 INFO:tasks.workunit.client.0.smithi076.stdout:0/802: write d2/d9a/db8/fe7 [280886,101372] 0 2024-03-23T15:44:58.199 INFO:tasks.workunit.client.0.smithi076.stdout:0/803: dread - d2/de/d10/d43/d56/fd7 zero size 2024-03-23T15:44:58.199 INFO:tasks.workunit.client.0.smithi076.stdout:9/851: rename d1/d1f/d46/d83/l58 to d1/l114 0 2024-03-23T15:44:58.201 INFO:tasks.workunit.client.0.smithi076.stdout:4/794: rename d8/d9/l46 to d8/d9/d1d/lf6 0 2024-03-23T15:44:58.206 INFO:tasks.workunit.client.0.smithi076.stdout:4/795: mknod d8/d9/d1d/cf7 0 2024-03-23T15:44:58.208 INFO:tasks.workunit.client.0.smithi076.stdout:4/796: mkdir d8/d3a/dab/dc3/df8 0 2024-03-23T15:44:58.210 INFO:tasks.workunit.client.0.smithi076.stdout:6/594: dwrite d1/dd/df/d5b/d62/f7e [4194304,4194304] 0 2024-03-23T15:44:58.210 INFO:tasks.workunit.client.0.smithi076.stdout:6/595: truncate d1/dd/df/fa4 1276025 0 2024-03-23T15:44:58.211 INFO:tasks.workunit.client.0.smithi076.stdout:4/797: dread d8/d80/fbc [0,4194304] 0 2024-03-23T15:44:58.211 INFO:tasks.workunit.client.0.smithi076.stdout:2/847: sync 2024-03-23T15:44:58.211 INFO:tasks.workunit.client.0.smithi076.stdout:8/692: sync 2024-03-23T15:44:58.211 INFO:tasks.workunit.client.0.smithi076.stdout:2/848: truncate d1/d4/d64/d1b/dd4/ff0 571432 0 2024-03-23T15:44:58.212 INFO:tasks.workunit.client.0.smithi076.stdout:6/596: creat d1/dd/df/d5b/dcd/dd0/fd9 x:0 0 0 2024-03-23T15:44:58.212 INFO:tasks.workunit.client.0.smithi076.stdout:6/597: chown d1/dd/df/d5b/l40 0 1 2024-03-23T15:44:58.212 INFO:tasks.workunit.client.0.smithi076.stdout:8/693: creat d8/d26/d33/d8c/fee x:0 0 0 2024-03-23T15:44:58.212 INFO:tasks.workunit.client.0.smithi076.stdout:8/694: read - d8/d26/fcf zero size 2024-03-23T15:44:58.219 INFO:tasks.workunit.client.0.smithi076.stdout:4/798: creat d8/d3a/ff9 x:0 0 0 2024-03-23T15:44:58.219 INFO:tasks.workunit.client.0.smithi076.stdout:4/799: creat d8/d3a/d69/de7/ffa x:0 0 0 2024-03-23T15:44:58.219 INFO:tasks.workunit.client.0.smithi076.stdout:4/800: write d8/d3a/d18/d52/f64 [4272359,105420] 0 2024-03-23T15:44:58.219 INFO:tasks.workunit.client.0.smithi076.stdout:2/849: write d1/d4/d64/d82/ffa [161370,43042] 0 2024-03-23T15:44:58.220 INFO:tasks.workunit.client.0.smithi076.stdout:2/850: mknod d1/d2e/d8a/c122 0 2024-03-23T15:44:58.221 INFO:tasks.workunit.client.0.smithi076.stdout:6/598: truncate d1/dd/df/d5b/d62/f7e 4586774 0 2024-03-23T15:44:58.221 INFO:tasks.workunit.client.0.smithi076.stdout:0/804: dwrite d2/de/d10/d43/f63 [4194304,4194304] 0 2024-03-23T15:44:58.221 INFO:tasks.workunit.client.0.smithi076.stdout:0/805: write d2/d9a/fc8 [485370,128153] 0 2024-03-23T15:44:58.221 INFO:tasks.workunit.client.0.smithi076.stdout:0/806: chown d2/de/d10/d3a/fca 14816936 1 2024-03-23T15:44:58.221 INFO:tasks.workunit.client.0.smithi076.stdout:2/851: creat d1/d4/db/d3e/f123 x:0 0 0 2024-03-23T15:44:58.222 INFO:tasks.workunit.client.0.smithi076.stdout:2/852: stat d1/d4/d52/fbb 0 2024-03-23T15:44:58.222 INFO:tasks.workunit.client.0.smithi076.stdout:2/853: read - d1/d4/db/d15/f10e zero size 2024-03-23T15:44:58.227 INFO:tasks.workunit.client.0.smithi076.stdout:0/807: unlink d2/de/d10/d3a/d47/fab 0 2024-03-23T15:44:58.231 INFO:tasks.workunit.client.0.smithi076.stdout:6/599: mknod d1/dd/df/d5b/d62/cda 0 2024-03-23T15:44:58.232 INFO:tasks.workunit.client.0.smithi076.stdout:6/600: dread d1/d4/f35 [0,4194304] 0 2024-03-23T15:44:58.232 INFO:tasks.workunit.client.0.smithi076.stdout:6/601: dread - d1/d81/fcb zero size 2024-03-23T15:44:58.232 INFO:tasks.workunit.client.0.smithi076.stdout:6/602: write d1/dd/df/d9b/db3/d8d/d60/d7c/f83 [2682609,8270] 0 2024-03-23T15:44:58.233 INFO:tasks.workunit.client.0.smithi076.stdout:6/603: truncate d1/dd/df/d9b/db3/d8d/f9e 264367 0 2024-03-23T15:44:58.233 INFO:tasks.workunit.client.0.smithi076.stdout:6/604: dread d1/f21 [4194304,4194304] 0 2024-03-23T15:44:58.241 INFO:tasks.workunit.client.0.smithi076.stdout:2/854: dwrite d1/d4/db/d3e/f84 [0,4194304] 0 2024-03-23T15:44:58.243 INFO:tasks.workunit.client.0.smithi076.stdout:2/855: mknod d1/d4/d52/d6e/c124 0 2024-03-23T15:44:58.247 INFO:tasks.workunit.client.0.smithi076.stdout:6/605: dread d1/d2/f1c [0,4194304] 0 2024-03-23T15:44:58.249 INFO:tasks.workunit.client.0.smithi076.stdout:6/606: unlink d1/dd/df/d5b/dcd/dd0/fd8 0 2024-03-23T15:44:58.250 INFO:tasks.workunit.client.0.smithi076.stdout:4/801: dwrite d8/d9/f9c [0,4194304] 0 2024-03-23T15:44:58.253 INFO:tasks.workunit.client.0.smithi076.stdout:4/802: creat d8/d80/ffb x:0 0 0 2024-03-23T15:44:58.259 INFO:tasks.workunit.client.0.smithi076.stdout:5/789: sync 2024-03-23T15:44:58.259 INFO:tasks.workunit.client.0.smithi076.stdout:3/561: sync 2024-03-23T15:44:58.259 INFO:tasks.workunit.client.0.smithi076.stdout:9/852: rename d1/d1f/d29/d2d/d37/cce to d1/d2f/d3a/d74/dd7/c115 0 2024-03-23T15:44:58.259 INFO:tasks.workunit.client.0.smithi076.stdout:2/856: dwrite d1/d2e/dd1/ddc/ff8 [0,4194304] 0 2024-03-23T15:44:58.259 INFO:tasks.workunit.client.0.smithi076.stdout:6/607: dread d1/dd/df/f66 [0,4194304] 0 2024-03-23T15:44:58.259 INFO:tasks.workunit.client.0.smithi076.stdout:9/853: fdatasync d1/d1f/d29/d2d/d37/f3c 0 2024-03-23T15:44:58.259 INFO:tasks.workunit.client.0.smithi076.stdout:6/608: chown d1/d4/d10/d39/l53 22475 1 2024-03-23T15:44:58.260 INFO:tasks.workunit.client.0.smithi076.stdout:3/562: getdents d5/d35/d3c 0 2024-03-23T15:44:58.261 INFO:tasks.workunit.client.0.smithi076.stdout:2/857: creat d1/d4/db/d15/f125 x:0 0 0 2024-03-23T15:44:58.262 INFO:tasks.workunit.client.0.smithi076.stdout:2/858: creat d1/d4/d64/d82/d103/f126 x:0 0 0 2024-03-23T15:44:58.263 INFO:tasks.workunit.client.0.smithi076.stdout:2/859: write d1/d4/d64/d82/d103/d79/fa9 [1540908,36614] 0 2024-03-23T15:44:58.264 INFO:tasks.workunit.client.0.smithi076.stdout:2/860: creat d1/d4/db/f127 x:0 0 0 2024-03-23T15:44:58.264 INFO:tasks.workunit.client.0.smithi076.stdout:2/861: chown d1/d4/d64/d1b 85 1 2024-03-23T15:44:58.264 INFO:tasks.workunit.client.0.smithi076.stdout:2/862: write d1/d4/db/d15/f125 [862359,68138] 0 2024-03-23T15:44:58.265 INFO:tasks.workunit.client.0.smithi076.stdout:5/790: unlink d1a/d33/d3a/d67/da8/dc4/d5f/d8f/lee 0 2024-03-23T15:44:58.265 INFO:tasks.workunit.client.0.smithi076.stdout:2/863: getdents d1/d4/db/dc8 0 2024-03-23T15:44:58.266 INFO:tasks.workunit.client.0.smithi076.stdout:2/864: chown d1/d4/d52/ded 13844125 1 2024-03-23T15:44:58.266 INFO:tasks.workunit.client.0.smithi076.stdout:2/865: creat d1/d4/d64/d82/f128 x:0 0 0 2024-03-23T15:44:58.267 INFO:tasks.workunit.client.0.smithi076.stdout:2/866: mknod d1/d4/d64/d82/d7f/c129 0 2024-03-23T15:44:58.267 INFO:tasks.workunit.client.0.smithi076.stdout:5/791: creat d1a/d33/f111 x:0 0 0 2024-03-23T15:44:58.275 INFO:tasks.workunit.client.0.smithi076.stdout:3/563: dwrite d5/d17/d1c/d32/d39/f7c [0,4194304] 0 2024-03-23T15:44:58.277 INFO:tasks.workunit.client.0.smithi076.stdout:3/564: mknod d5/d17/d1c/d31/d61/d96/cba 0 2024-03-23T15:44:58.277 INFO:tasks.workunit.client.0.smithi076.stdout:3/565: truncate d5/d17/d1c/d32/d39/f44 3509347 0 2024-03-23T15:44:58.279 INFO:tasks.workunit.client.0.smithi076.stdout:2/867: dwrite d1/d4/db/d3e/f84 [0,4194304] 0 2024-03-23T15:44:58.295 INFO:tasks.workunit.client.0.smithi076.stdout:3/566: dwrite d5/d35/d4c/d5a/fb9 [0,4194304] 0 2024-03-23T15:44:58.295 INFO:tasks.workunit.client.0.smithi076.stdout:3/567: readlink d5/d17/d1c/d32/l42 0 2024-03-23T15:44:58.310 INFO:tasks.workunit.client.0.smithi076.stdout:3/568: dwrite d5/d35/d4c/d5a/fb0 [0,4194304] 0 2024-03-23T15:44:58.310 INFO:tasks.workunit.client.0.smithi076.stdout:3/569: truncate d5/d17/d1c/d32/d4a/f72 1973606 0 2024-03-23T15:44:58.311 INFO:tasks.workunit.client.0.smithi076.stdout:3/570: chown d5/d17/d1c/d32/d39/f7c 30197 1 2024-03-23T15:44:58.311 INFO:tasks.workunit.client.0.smithi076.stdout:3/571: creat d5/d17/d1c/d32/d4a/fbb x:0 0 0 2024-03-23T15:44:58.312 INFO:tasks.workunit.client.0.smithi076.stdout:3/572: creat d5/d35/d3c/d67/fbc x:0 0 0 2024-03-23T15:44:58.313 INFO:tasks.workunit.client.0.smithi076.stdout:3/573: creat d5/d35/d4c/d5a/fbd x:0 0 0 2024-03-23T15:44:58.316 INFO:tasks.workunit.client.0.smithi076.stdout:3/574: link d5/d17/d1c/d31/d47/f7d d5/d35/d6d/fbe 0 2024-03-23T15:44:58.319 INFO:tasks.workunit.client.0.smithi076.stdout:5/792: write d1a/d33/d3a/d67/da8/dc4/d5f/d8f/fc5 [313040,89649] 0 2024-03-23T15:44:58.319 INFO:tasks.workunit.client.0.smithi076.stdout:5/793: chown d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fec 0 1 2024-03-23T15:44:58.320 INFO:tasks.workunit.client.0.smithi076.stdout:5/794: read d1a/d4b/fb8 [565667,120695] 0 2024-03-23T15:44:58.321 INFO:tasks.workunit.client.0.smithi076.stdout:3/575: link d5/fb1 d5/d17/fbf 0 2024-03-23T15:44:58.321 INFO:tasks.workunit.client.0.smithi076.stdout:4/803: dwrite d8/d3a/d69/d73/d76/db4/fc5 [0,4194304] 0 2024-03-23T15:44:58.322 INFO:tasks.workunit.client.0.smithi076.stdout:0/808: dwrite d2/de/d10/d3a/d47/d69/d84/fa2 [4194304,4194304] 0 2024-03-23T15:44:58.323 INFO:tasks.workunit.client.0.smithi076.stdout:3/576: rmdir d5/d17/d1c/d31 39 2024-03-23T15:44:58.324 INFO:tasks.workunit.client.0.smithi076.stdout:5/795: dread d1a/d33/d3a/d67/da8/dc4/d5f/fd2 [0,4194304] 0 2024-03-23T15:44:58.327 INFO:tasks.workunit.client.0.smithi076.stdout:4/804: creat d8/de3/db0/de9/ffc x:0 0 0 2024-03-23T15:44:58.327 INFO:tasks.workunit.client.0.smithi076.stdout:9/854: rename d1/c14 to d1/d1f/d46/d4a/d5a/daa/c116 0 2024-03-23T15:44:58.330 INFO:tasks.workunit.client.0.smithi076.stdout:3/577: symlink d5/d17/d1c/d32/d8e/lc0 0 2024-03-23T15:44:58.332 INFO:tasks.workunit.client.0.smithi076.stdout:0/809: mknod d2/de/d10/d3a/d47/d69/cf6 0 2024-03-23T15:44:58.333 INFO:tasks.workunit.client.0.smithi076.stdout:6/609: rename d1/dd/c63 to d1/dd/df/cdb 0 2024-03-23T15:44:58.333 INFO:tasks.workunit.client.0.smithi076.stdout:9/855: dread d1/d2f/d3f/f92 [0,4194304] 0 2024-03-23T15:44:58.333 INFO:tasks.workunit.client.0.smithi076.stdout:9/856: stat d1/d1f/d29/d2d/d37 0 2024-03-23T15:44:58.339 INFO:tasks.workunit.client.0.smithi076.stdout:5/796: dread fc [0,4194304] 0 2024-03-23T15:44:58.339 INFO:tasks.workunit.client.0.smithi076.stdout:5/797: readlink d1a/d20/lb0 0 2024-03-23T15:44:58.339 INFO:tasks.workunit.client.0.smithi076.stdout:0/810: rmdir d2/d9a/db8 39 2024-03-23T15:44:58.340 INFO:tasks.workunit.client.0.smithi076.stdout:6/610: creat d1/d4/d10/fdc x:0 0 0 2024-03-23T15:44:58.342 INFO:tasks.workunit.client.0.smithi076.stdout:7/979: sync 2024-03-23T15:44:58.342 INFO:tasks.workunit.client.0.smithi076.stdout:1/494: sync 2024-03-23T15:44:58.342 INFO:tasks.workunit.client.0.smithi076.stdout:1/495: fsync d0/d35/f6f 0 2024-03-23T15:44:58.342 INFO:tasks.workunit.client.0.smithi076.stdout:1/496: fsync d0/f34 0 2024-03-23T15:44:58.343 INFO:tasks.workunit.client.0.smithi076.stdout:6/611: mknod d1/dd/df/d5b/d62/dc7/cdd 0 2024-03-23T15:44:58.343 INFO:tasks.workunit.client.0.smithi076.stdout:6/612: fdatasync d1/dd/d2a/f91 0 2024-03-23T15:44:58.343 INFO:tasks.workunit.client.0.smithi076.stdout:6/613: chown d1/d4/d10/d39/ca3 62525932 1 2024-03-23T15:44:58.344 INFO:tasks.workunit.client.0.smithi076.stdout:4/805: dwrite d8/d9/f9c [0,4194304] 0 2024-03-23T15:44:58.344 INFO:tasks.workunit.client.0.smithi076.stdout:3/578: read d5/d17/d1c/f2b [137696,34829] 0 2024-03-23T15:44:58.344 INFO:tasks.workunit.client.0.smithi076.stdout:3/579: chown d5/f16 46290830 1 2024-03-23T15:44:58.344 INFO:tasks.workunit.client.0.smithi076.stdout:5/798: unlink d1a/d33/d3a/c4f 0 2024-03-23T15:44:58.345 INFO:tasks.workunit.client.0.smithi076.stdout:0/811: mkdir d2/de/d10/d43/df3/df7 0 2024-03-23T15:44:58.345 INFO:tasks.workunit.client.0.smithi076.stdout:5/799: truncate d1a/d33/f73 803088 0 2024-03-23T15:44:58.347 INFO:tasks.workunit.client.0.smithi076.stdout:5/800: mknod d1a/d33/d3a/d67/da8/dc4/c112 0 2024-03-23T15:44:58.348 INFO:tasks.workunit.client.0.smithi076.stdout:1/497: mknod d0/d35/ca2 0 2024-03-23T15:44:58.351 INFO:tasks.workunit.client.0.smithi076.stdout:5/801: link d1a/d33/d3a/d67/da8/dc4/le9 d1a/d33/dc3/d105/l113 0 2024-03-23T15:44:58.352 INFO:tasks.workunit.client.0.smithi076.stdout:9/857: dwrite d1/d2f/d3f/da0/fd9 [0,4194304] 0 2024-03-23T15:44:58.355 INFO:tasks.workunit.client.0.smithi076.stdout:4/806: rmdir d8/d3a/d69/d73/df4/d88 39 2024-03-23T15:44:58.355 INFO:tasks.workunit.client.0.smithi076.stdout:3/580: rename d5/d17/d1c/d31/d47/f7d to d5/d17/d1c/d32/d70/fc1 0 2024-03-23T15:44:58.355 INFO:tasks.workunit.client.0.smithi076.stdout:3/581: chown d5/d17/d1c/l6a 888364 1 2024-03-23T15:44:58.358 INFO:tasks.workunit.client.0.smithi076.stdout:4/807: link d8/d9/lba d8/d41/d97/lfd 0 2024-03-23T15:44:58.361 INFO:tasks.workunit.client.0.smithi076.stdout:9/858: dread d1/d2f/d3a/d64/f85 [0,4194304] 0 2024-03-23T15:44:58.361 INFO:tasks.workunit.client.0.smithi076.stdout:9/859: getdents d1/d20 0 2024-03-23T15:44:58.361 INFO:tasks.workunit.client.0.smithi076.stdout:4/808: mknod d8/d9/d1d/dc9/cfe 0 2024-03-23T15:44:58.361 INFO:tasks.workunit.client.0.smithi076.stdout:4/809: write d8/d3a/d18/d1b/fa5 [1044257,113105] 0 2024-03-23T15:44:58.363 INFO:tasks.workunit.client.0.smithi076.stdout:9/860: creat d1/d2f/d3a/d74/d102/f117 x:0 0 0 2024-03-23T15:44:58.363 INFO:tasks.workunit.client.0.smithi076.stdout:5/802: dread d1a/d33/d3a/d80/fd8 [0,4194304] 0 2024-03-23T15:44:58.363 INFO:tasks.workunit.client.0.smithi076.stdout:7/980: dwrite d8/d4e/d98/df6/d104/d40/d5d/f34 [0,4194304] 0 2024-03-23T15:44:58.364 INFO:tasks.workunit.client.0.smithi076.stdout:3/582: write d5/d35/d4c/d5a/fb0 [2766533,79133] 0 2024-03-23T15:44:58.366 INFO:tasks.workunit.client.0.smithi076.stdout:1/498: dwrite d0/d2/f75 [0,4194304] 0 2024-03-23T15:44:58.368 INFO:tasks.workunit.client.0.smithi076.stdout:6/614: dwrite d1/d2/f87 [0,4194304] 0 2024-03-23T15:44:58.368 INFO:tasks.workunit.client.0.smithi076.stdout:6/615: readlink d1/d4/d10/d39/lbb 0 2024-03-23T15:44:58.368 INFO:tasks.workunit.client.0.smithi076.stdout:4/810: symlink d8/de3/lff 0 2024-03-23T15:44:58.368 INFO:tasks.workunit.client.0.smithi076.stdout:3/583: mkdir d5/dc2 0 2024-03-23T15:44:58.368 INFO:tasks.workunit.client.0.smithi076.stdout:7/981: creat d8/d4e/d98/df6/d104/d3a/f142 x:0 0 0 2024-03-23T15:44:58.368 INFO:tasks.workunit.client.0.smithi076.stdout:7/982: stat d8/d4e/d98/df6/d104 0 2024-03-23T15:44:58.371 INFO:tasks.workunit.client.0.smithi076.stdout:7/983: rename d8/d4e/d98/df6/d104/d18/dcc/f13a to d8/d4e/d98/df6/d104/d18/dfd/d125/f143 0 2024-03-23T15:44:58.373 INFO:tasks.workunit.client.0.smithi076.stdout:6/616: mknod d1/dd/df/d5b/d62/cde 0 2024-03-23T15:44:58.374 INFO:tasks.workunit.client.0.smithi076.stdout:4/811: creat d8/d9/d1d/f100 x:0 0 0 2024-03-23T15:44:58.374 INFO:tasks.workunit.client.0.smithi076.stdout:3/584: symlink d5/d17/d1c/lc3 0 2024-03-23T15:44:58.376 INFO:tasks.workunit.client.0.smithi076.stdout:4/812: symlink d8/d3a/d69/de7/l101 0 2024-03-23T15:44:58.377 INFO:tasks.workunit.client.0.smithi076.stdout:6/617: mknod d1/dd/df/d9b/db3/d8d/cdf 0 2024-03-23T15:44:58.377 INFO:tasks.workunit.client.0.smithi076.stdout:3/585: link d5/d35/c78 d5/d17/d1c/d31/d61/d92/cc4 0 2024-03-23T15:44:58.378 INFO:tasks.workunit.client.0.smithi076.stdout:3/586: write d5/d17/d1c/f52 [681388,49738] 0 2024-03-23T15:44:58.378 INFO:tasks.workunit.client.0.smithi076.stdout:4/813: rmdir d8/d3a/d18/d75 39 2024-03-23T15:44:58.379 INFO:tasks.workunit.client.0.smithi076.stdout:6/618: mkdir d1/dd/df/d9b/db3/d8d/db8/de0 0 2024-03-23T15:44:58.381 INFO:tasks.workunit.client.0.smithi076.stdout:4/814: creat d8/d3a/d69/f102 x:0 0 0 2024-03-23T15:44:58.382 INFO:tasks.workunit.client.0.smithi076.stdout:5/803: dwrite d1a/d4b/fb8 [0,4194304] 0 2024-03-23T15:44:58.384 INFO:tasks.workunit.client.0.smithi076.stdout:5/804: creat d1a/d33/dc3/f114 x:0 0 0 2024-03-23T15:44:58.385 INFO:tasks.workunit.client.0.smithi076.stdout:9/861: dwrite d1/d1f/d29/d2d/d37/d47/fae [0,4194304] 0 2024-03-23T15:44:58.386 INFO:tasks.workunit.client.0.smithi076.stdout:5/805: getdents d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96 0 2024-03-23T15:44:58.386 INFO:tasks.workunit.client.0.smithi076.stdout:5/806: chown d1a/d33/c108 592900027 1 2024-03-23T15:44:58.386 INFO:tasks.workunit.client.0.smithi076.stdout:4/815: rename d8/de3/db0/fce to d8/d3a/d69/d73/df4/f103 0 2024-03-23T15:44:58.386 INFO:tasks.workunit.client.0.smithi076.stdout:4/816: write d8/d3a/d69/de7/ff2 [600965,117324] 0 2024-03-23T15:44:58.391 INFO:tasks.workunit.client.0.smithi076.stdout:1/499: dwrite d0/d35/d40/d47/d74/d83/f9c [0,4194304] 0 2024-03-23T15:44:58.391 INFO:tasks.workunit.client.0.smithi076.stdout:5/807: readlink d1a/l7b 0 2024-03-23T15:44:58.392 INFO:tasks.workunit.client.0.smithi076.stdout:1/500: truncate d0/d3c/d6e/f80 2110291 0 2024-03-23T15:44:58.393 INFO:tasks.workunit.client.0.smithi076.stdout:4/817: dread d8/d3a/d69/d73/d76/db4/fc5 [0,4194304] 0 2024-03-23T15:44:58.394 INFO:tasks.workunit.client.0.smithi076.stdout:4/818: stat d8/d9/d1d/lf6 0 2024-03-23T15:44:58.395 INFO:tasks.workunit.client.0.smithi076.stdout:1/501: mknod d0/d35/d40/d47/d74/d83/ca3 0 2024-03-23T15:44:58.395 INFO:tasks.workunit.client.0.smithi076.stdout:4/819: creat d8/d3a/d18/d75/f104 x:0 0 0 2024-03-23T15:44:58.396 INFO:tasks.workunit.client.0.smithi076.stdout:1/502: creat d0/d35/d40/d47/d8c/d99/fa4 x:0 0 0 2024-03-23T15:44:58.397 INFO:tasks.workunit.client.0.smithi076.stdout:5/808: write d1a/d33/d3a/d67/da8/dc4/f52 [3626954,117519] 0 2024-03-23T15:44:58.397 INFO:tasks.workunit.client.0.smithi076.stdout:5/809: readlink d1a/d33/d3a/d67/da8/dc4/d65/l89 0 2024-03-23T15:44:58.399 INFO:tasks.workunit.client.0.smithi076.stdout:5/810: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/f115 x:0 0 0 2024-03-23T15:44:58.399 INFO:tasks.workunit.client.0.smithi076.stdout:5/811: rmdir d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf 39 2024-03-23T15:44:58.402 INFO:tasks.workunit.client.0.smithi076.stdout:5/812: creat d1a/f116 x:0 0 0 2024-03-23T15:44:58.404 INFO:tasks.workunit.client.0.smithi076.stdout:9/862: read d1/d1f/f10b [1894291,75080] 0 2024-03-23T15:44:58.404 INFO:tasks.workunit.client.0.smithi076.stdout:6/619: dread d1/d2/f6d [4194304,4194304] 0 2024-03-23T15:44:58.404 INFO:tasks.workunit.client.0.smithi076.stdout:6/620: chown d1/dd/df/d9b/db3 43162325 1 2024-03-23T15:44:58.405 INFO:tasks.workunit.client.0.smithi076.stdout:3/587: dwrite d5/d17/d1c/d32/d39/f65 [0,4194304] 0 2024-03-23T15:44:58.412 INFO:tasks.workunit.client.0.smithi076.stdout:3/588: link d5/d17/d1c/d32/d8e/c7e d5/d17/d1c/d32/d5d/cc5 0 2024-03-23T15:44:58.416 INFO:tasks.workunit.client.0.smithi076.stdout:3/589: unlink d5/d17/d1c/l6a 0 2024-03-23T15:44:58.416 INFO:tasks.workunit.client.0.smithi076.stdout:3/590: write d5/fd [1133200,97228] 0 2024-03-23T15:44:58.416 INFO:tasks.workunit.client.0.smithi076.stdout:5/813: dread d1a/d4b/f78 [0,4194304] 0 2024-03-23T15:44:58.416 INFO:tasks.workunit.client.0.smithi076.stdout:1/503: dwrite d0/f9d [0,4194304] 0 2024-03-23T15:44:58.417 INFO:tasks.workunit.client.0.smithi076.stdout:5/814: symlink d1a/d33/dc3/da6/l117 0 2024-03-23T15:44:58.417 INFO:tasks.workunit.client.0.smithi076.stdout:5/815: readlink d1a/d33/d3a/laf 0 2024-03-23T15:44:58.417 INFO:tasks.workunit.client.0.smithi076.stdout:5/816: chown d1a/d33/d3a/d67/da8/dc4/c112 213 1 2024-03-23T15:44:58.426 INFO:tasks.workunit.client.0.smithi076.stdout:3/591: rmdir d5/d17/d1c/d32/d39/da5 0 2024-03-23T15:44:58.426 INFO:tasks.workunit.client.0.smithi076.stdout:3/592: truncate d5/d35/d6d/fa7 52603 0 2024-03-23T15:44:58.426 INFO:tasks.workunit.client.0.smithi076.stdout:3/593: readlink d5/d17/l22 0 2024-03-23T15:44:58.426 INFO:tasks.workunit.client.0.smithi076.stdout:5/817: link d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/ff9 d1a/f118 0 2024-03-23T15:44:58.426 INFO:tasks.workunit.client.0.smithi076.stdout:9/863: getdents d1/d1f/d46/d4a/d5a 0 2024-03-23T15:44:58.426 INFO:tasks.workunit.client.0.smithi076.stdout:5/818: mkdir d1a/d33/d3a/d67/da8/dc4/d119 0 2024-03-23T15:44:58.429 INFO:tasks.workunit.client.0.smithi076.stdout:5/819: dread d1a/d33/d3a/d80/fca [0,4194304] 0 2024-03-23T15:44:58.429 INFO:tasks.workunit.client.0.smithi076.stdout:5/820: chown d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/fc2 52696714 1 2024-03-23T15:44:58.429 INFO:tasks.workunit.client.0.smithi076.stdout:5/821: chown d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fc1 2986 1 2024-03-23T15:44:58.429 INFO:tasks.workunit.client.0.smithi076.stdout:5/822: chown d1a/d33/d3a/d67/da8/dc4/d5f/f81 124 1 2024-03-23T15:44:58.430 INFO:tasks.workunit.client.0.smithi076.stdout:6/621: link d1/dd/d2a/d23/c9c d1/ce1 0 2024-03-23T15:44:58.430 INFO:tasks.workunit.client.0.smithi076.stdout:5/823: symlink d1a/d33/dc3/da6/l11a 0 2024-03-23T15:44:58.431 INFO:tasks.workunit.client.0.smithi076.stdout:6/622: truncate d1/d2/f71 839245 0 2024-03-23T15:44:58.433 INFO:tasks.workunit.client.0.smithi076.stdout:5/824: creat d1a/d33/d3a/d67/d84/f11b x:0 0 0 2024-03-23T15:44:58.434 INFO:tasks.workunit.client.0.smithi076.stdout:5/825: creat d1a/d33/d3a/f11c x:0 0 0 2024-03-23T15:44:58.434 INFO:tasks.workunit.client.0.smithi076.stdout:5/826: chown d1a/d7e/fe7 2 1 2024-03-23T15:44:58.436 INFO:tasks.workunit.client.0.smithi076.stdout:5/827: creat d1a/d2c/d106/f11d x:0 0 0 2024-03-23T15:44:58.436 INFO:tasks.workunit.client.0.smithi076.stdout:9/864: rename d1/d1f/f107 to d1/d1f/d46/d83/dad/f118 0 2024-03-23T15:44:58.437 INFO:tasks.workunit.client.0.smithi076.stdout:1/504: dwrite d0/f2e [0,4194304] 0 2024-03-23T15:44:58.438 INFO:tasks.workunit.client.0.smithi076.stdout:1/505: chown d0/d2/f75 1059332214 1 2024-03-23T15:44:58.438 INFO:tasks.workunit.client.0.smithi076.stdout:5/828: write d1a/d33/d3a/d67/da8/dc4/d65/f6b [884676,127284] 0 2024-03-23T15:44:58.439 INFO:tasks.workunit.client.0.smithi076.stdout:5/829: creat d1a/d2c/f11e x:0 0 0 2024-03-23T15:44:58.439 INFO:tasks.workunit.client.0.smithi076.stdout:5/830: readlink d1a/l7b 0 2024-03-23T15:44:58.440 INFO:tasks.workunit.client.0.smithi076.stdout:1/506: symlink d0/d35/d40/d47/la5 0 2024-03-23T15:44:58.442 INFO:tasks.workunit.client.0.smithi076.stdout:5/831: rmdir d1a/d33/dfd 0 2024-03-23T15:44:58.444 INFO:tasks.workunit.client.0.smithi076.stdout:5/832: symlink d1a/l11f 0 2024-03-23T15:44:58.446 INFO:tasks.workunit.client.0.smithi076.stdout:5/833: mkdir d1a/d33/d3a/d120 0 2024-03-23T15:44:58.451 INFO:tasks.workunit.client.0.smithi076.stdout:1/507: dread d0/d2/f48 [0,4194304] 0 2024-03-23T15:44:58.451 INFO:tasks.workunit.client.0.smithi076.stdout:6/623: dwrite d1/dd/f26 [4194304,4194304] 0 2024-03-23T15:44:58.456 INFO:tasks.workunit.client.0.smithi076.stdout:9/865: dwrite d1/d1f/d46/d4a/fe1 [0,4194304] 0 2024-03-23T15:44:58.458 INFO:tasks.workunit.client.0.smithi076.stdout:5/834: dread d1a/d33/f4d [0,4194304] 0 2024-03-23T15:44:58.458 INFO:tasks.workunit.client.0.smithi076.stdout:5/835: fdatasync d1a/d33/d3a/d67/da8/dc4/ff6 0 2024-03-23T15:44:58.459 INFO:tasks.workunit.client.0.smithi076.stdout:5/836: truncate d1a/d33/d3a/d67/da8/dc4/ffe 1649977 0 2024-03-23T15:44:58.459 INFO:tasks.workunit.client.0.smithi076.stdout:9/866: mkdir d1/d2f/d3f/da0/d119 0 2024-03-23T15:44:58.461 INFO:tasks.workunit.client.0.smithi076.stdout:9/867: creat d1/d2f/d3a/d64/f11a x:0 0 0 2024-03-23T15:44:58.466 INFO:tasks.workunit.client.0.smithi076.stdout:5/837: unlink d1a/f116 0 2024-03-23T15:44:58.468 INFO:tasks.workunit.client.0.smithi076.stdout:1/508: dwrite d0/d3c/f57 [4194304,4194304] 0 2024-03-23T15:44:58.468 INFO:tasks.workunit.client.0.smithi076.stdout:1/509: getdents d0/d35/d40/d47/d8c/d99 0 2024-03-23T15:44:58.469 INFO:tasks.workunit.client.0.smithi076.stdout:1/510: write d0/f4b [302215,53320] 0 2024-03-23T15:44:58.469 INFO:tasks.workunit.client.0.smithi076.stdout:1/511: rmdir d0/d35/d40/d47/d17/d20 39 2024-03-23T15:44:58.470 INFO:tasks.workunit.client.0.smithi076.stdout:1/512: mknod d0/d35/d40/d47/d55/ca6 0 2024-03-23T15:44:58.470 INFO:tasks.workunit.client.0.smithi076.stdout:1/513: read d0/d3c/f57 [1733285,48943] 0 2024-03-23T15:44:58.472 INFO:tasks.workunit.client.0.smithi076.stdout:6/624: dwrite d1/d4/d10/f30 [0,4194304] 0 2024-03-23T15:44:58.474 INFO:tasks.workunit.client.0.smithi076.stdout:6/625: rmdir d1/dd/d2a/d23 39 2024-03-23T15:44:58.476 INFO:tasks.workunit.client.0.smithi076.stdout:1/514: creat d0/d2/d62/fa7 x:0 0 0 2024-03-23T15:44:58.481 INFO:tasks.workunit.client.0.smithi076.stdout:6/626: write d1/dd/df/f13 [8964629,24794] 0 2024-03-23T15:44:58.482 INFO:tasks.workunit.client.0.smithi076.stdout:6/627: chown d1/d4/d10/c3d 0 1 2024-03-23T15:44:58.483 INFO:tasks.workunit.client.0.smithi076.stdout:6/628: mkdir d1/d4/de2 0 2024-03-23T15:44:58.484 INFO:tasks.workunit.client.0.smithi076.stdout:6/629: rmdir d1/dd/df/dac 39 2024-03-23T15:44:58.489 INFO:tasks.workunit.client.0.smithi076.stdout:1/515: dread d0/d3c/d6e/f76 [0,4194304] 0 2024-03-23T15:44:58.489 INFO:tasks.workunit.client.0.smithi076.stdout:1/516: chown d0/d35/d40/d47/d17/d20/l2d 0 1 2024-03-23T15:44:58.489 INFO:tasks.workunit.client.0.smithi076.stdout:1/517: readlink d0/d2/d91/l3d 0 2024-03-23T15:44:58.489 INFO:tasks.workunit.client.0.smithi076.stdout:1/518: chown d0/d2/d1f/f43 55842 1 2024-03-23T15:44:58.492 INFO:tasks.workunit.client.0.smithi076.stdout:5/838: dwrite d1a/f1b [4194304,4194304] 0 2024-03-23T15:44:58.493 INFO:tasks.workunit.client.0.smithi076.stdout:9/868: dwrite d1/d1f/f72 [8388608,4194304] 0 2024-03-23T15:44:58.500 INFO:tasks.workunit.client.0.smithi076.stdout:5/839: write d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fc1 [3873041,72290] 0 2024-03-23T15:44:58.501 INFO:tasks.workunit.client.0.smithi076.stdout:6/630: rename d1/dd/df/f5f to d1/d81/fe3 0 2024-03-23T15:44:58.502 INFO:tasks.workunit.client.0.smithi076.stdout:6/631: write d1/dd/df/d9b/db3/d8d/d60/fae [679627,110800] 0 2024-03-23T15:44:58.502 INFO:tasks.workunit.client.0.smithi076.stdout:6/632: dread - d1/dd/df/dac/fc4 zero size 2024-03-23T15:44:58.502 INFO:tasks.workunit.client.0.smithi076.stdout:9/869: write d1/d1f/d29/d2d/d37/d47/f52 [3163905,4755] 0 2024-03-23T15:44:58.503 INFO:tasks.workunit.client.0.smithi076.stdout:6/633: symlink d1/d4/le4 0 2024-03-23T15:44:58.505 INFO:tasks.workunit.client.0.smithi076.stdout:6/634: mknod d1/dd/df/dce/ce5 0 2024-03-23T15:44:58.505 INFO:tasks.workunit.client.0.smithi076.stdout:6/635: chown d1/d4/d10/l50 91 1 2024-03-23T15:44:58.505 INFO:tasks.workunit.client.0.smithi076.stdout:8/695: sync 2024-03-23T15:44:58.507 INFO:tasks.workunit.client.0.smithi076.stdout:8/696: creat d8/d4f/d80/db7/fef x:0 0 0 2024-03-23T15:44:58.514 INFO:tasks.workunit.client.0.smithi076.stdout:6/636: write d1/dd/d2a/d23/f3a [6928211,68016] 0 2024-03-23T15:44:58.515 INFO:tasks.workunit.client.0.smithi076.stdout:6/637: creat d1/d4/d10/fe6 x:0 0 0 2024-03-23T15:44:58.515 INFO:tasks.workunit.client.0.smithi076.stdout:6/638: dread - d1/dd/df/fb2 zero size 2024-03-23T15:44:58.518 INFO:tasks.workunit.client.0.smithi076.stdout:6/639: unlink d1/d2/cab 0 2024-03-23T15:44:58.518 INFO:tasks.workunit.client.0.smithi076.stdout:6/640: chown d1/dd/df/d5b/d62/cda 1157316 1 2024-03-23T15:44:58.527 INFO:tasks.workunit.client.0.smithi076.stdout:1/519: rename d0/d35/d40/d47/d17/f49 to d0/d35/d40/fa8 0 2024-03-23T15:44:58.527 INFO:tasks.workunit.client.0.smithi076.stdout:1/520: stat d0/d35/d40/d47/d74/d83/f98 0 2024-03-23T15:44:58.530 INFO:tasks.workunit.client.0.smithi076.stdout:6/641: link d1/d2/laf d1/dd/df/dac/le7 0 2024-03-23T15:44:58.531 INFO:tasks.workunit.client.0.smithi076.stdout:6/642: dread - d1/dd/df/fb2 zero size 2024-03-23T15:44:58.531 INFO:tasks.workunit.client.0.smithi076.stdout:6/643: readlink d1/d4/l65 0 2024-03-23T15:44:58.541 INFO:tasks.workunit.client.0.smithi076.stdout:5/840: rename d1a/d33/d3a/d67/d84/fd3 to d1a/f121 0 2024-03-23T15:44:58.542 INFO:tasks.workunit.client.0.smithi076.stdout:5/841: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/f122 x:0 0 0 2024-03-23T15:44:58.542 INFO:tasks.workunit.client.0.smithi076.stdout:5/842: chown d1a/d33/d3a/d67/da8/dc4/f93 7039924 1 2024-03-23T15:44:58.545 INFO:tasks.workunit.client.0.smithi076.stdout:1/521: dwrite d0/d2/d91/f61 [0,4194304] 0 2024-03-23T15:44:58.545 INFO:tasks.workunit.client.0.smithi076.stdout:1/522: readlink d0/d35/d40/d47/d55/l69 0 2024-03-23T15:44:58.546 INFO:tasks.workunit.client.0.smithi076.stdout:1/523: mknod d0/d35/d40/d47/d55/ca9 0 2024-03-23T15:44:58.552 INFO:tasks.workunit.client.0.smithi076.stdout:1/524: write d0/d35/d40/d47/d17/f1d [1823946,68388] 0 2024-03-23T15:44:58.615 INFO:tasks.workunit.client.0.smithi076.stdout:5/843: dwrite d1a/d33/d3a/d67/da8/dc4/d65/f88 [0,4194304] 0 2024-03-23T15:44:58.616 INFO:tasks.workunit.client.0.smithi076.stdout:1/525: dwrite d0/d2/f37 [0,4194304] 0 2024-03-23T15:44:58.619 INFO:tasks.workunit.client.0.smithi076.stdout:1/526: mknod d0/d2/d70/da1/caa 0 2024-03-23T15:44:58.619 INFO:tasks.workunit.client.0.smithi076.stdout:5/844: dread d1a/f42 [0,4194304] 0 2024-03-23T15:44:58.622 INFO:tasks.workunit.client.0.smithi076.stdout:2/868: sync 2024-03-23T15:44:58.622 INFO:tasks.workunit.client.0.smithi076.stdout:0/812: sync 2024-03-23T15:44:58.625 INFO:tasks.workunit.client.0.smithi076.stdout:2/869: link d1/d4/d64/d82/d103/cc1 d1/d4/d64/d1b/dd4/c12a 0 2024-03-23T15:44:58.627 INFO:tasks.workunit.client.0.smithi076.stdout:5/845: getdents d1a/d33/d3a/d67/d84 0 2024-03-23T15:44:58.628 INFO:tasks.workunit.client.0.smithi076.stdout:2/870: mknod d1/c12b 0 2024-03-23T15:44:58.629 INFO:tasks.workunit.client.0.smithi076.stdout:2/871: chown d1/d4/d64/f20 461541648 1 2024-03-23T15:44:58.629 INFO:tasks.workunit.client.0.smithi076.stdout:5/846: creat d1a/d20/f123 x:0 0 0 2024-03-23T15:44:58.629 INFO:tasks.workunit.client.0.smithi076.stdout:2/872: symlink d1/d4/db/d15/d29/d7d/l12c 0 2024-03-23T15:44:58.629 INFO:tasks.workunit.client.0.smithi076.stdout:0/813: write d2/de/d10/d3a/d47/f4e [1155738,103652] 0 2024-03-23T15:44:58.629 INFO:tasks.workunit.client.0.smithi076.stdout:0/814: write d2/d1b/f42 [554794,116385] 0 2024-03-23T15:44:58.629 INFO:tasks.workunit.client.0.smithi076.stdout:2/873: creat d1/d4/db/d15/d29/d7d/f12d x:0 0 0 2024-03-23T15:44:58.630 INFO:tasks.workunit.client.0.smithi076.stdout:0/815: mkdir d2/ded/df8 0 2024-03-23T15:44:58.632 INFO:tasks.workunit.client.0.smithi076.stdout:1/527: dwrite d0/d2/d62/f9e [0,4194304] 0 2024-03-23T15:44:58.648 INFO:tasks.workunit.client.0.smithi076.stdout:5/847: dwrite d1a/d33/f4d [0,4194304] 0 2024-03-23T15:44:58.648 INFO:tasks.workunit.client.0.smithi076.stdout:5/848: dread - d1a/d2c/f11e zero size 2024-03-23T15:44:58.648 INFO:tasks.workunit.client.0.smithi076.stdout:5/849: chown c16 364179976 1 2024-03-23T15:44:58.654 INFO:tasks.workunit.client.0.smithi076.stdout:2/874: dwrite d1/d4/d64/d82/d7f/f8c [0,4194304] 0 2024-03-23T15:44:58.655 INFO:tasks.workunit.client.0.smithi076.stdout:2/875: creat d1/d2e/dd1/df6/f12e x:0 0 0 2024-03-23T15:44:58.655 INFO:tasks.workunit.client.0.smithi076.stdout:2/876: stat d1/d2e/dd1/ddc 0 2024-03-23T15:44:58.655 INFO:tasks.workunit.client.0.smithi076.stdout:2/877: chown d1/d4/db/d15/d29/d7d/fc9 17876111 1 2024-03-23T15:44:58.655 INFO:tasks.workunit.client.0.smithi076.stdout:2/878: truncate d1/d4/d52/fe8 589242 0 2024-03-23T15:44:58.657 INFO:tasks.workunit.client.0.smithi076.stdout:2/879: mknod d1/d4/db/d3e/d10c/c12f 0 2024-03-23T15:44:58.657 INFO:tasks.workunit.client.0.smithi076.stdout:2/880: read - d1/d4/db/d15/d29/d7d/f12d zero size 2024-03-23T15:44:58.657 INFO:tasks.workunit.client.0.smithi076.stdout:1/528: dwrite d0/d35/d40/f90 [0,4194304] 0 2024-03-23T15:44:58.657 INFO:tasks.workunit.client.0.smithi076.stdout:0/816: dwrite d2/de/d10/f61 [0,4194304] 0 2024-03-23T15:44:58.658 INFO:tasks.workunit.client.0.smithi076.stdout:2/881: creat d1/dbf/f130 x:0 0 0 2024-03-23T15:44:58.659 INFO:tasks.workunit.client.0.smithi076.stdout:0/817: dread d2/d9a/fc8 [0,4194304] 0 2024-03-23T15:44:58.659 INFO:tasks.workunit.client.0.smithi076.stdout:1/529: mknod d0/d35/d40/d47/d17/cab 0 2024-03-23T15:44:58.661 INFO:tasks.workunit.client.0.smithi076.stdout:0/818: unlink d2/de/d10/d3a/d47/la1 0 2024-03-23T15:44:58.661 INFO:tasks.workunit.client.0.smithi076.stdout:2/882: rmdir d1/d4/d64/d82/d103/d79/dac/dd8/dea 39 2024-03-23T15:44:58.663 INFO:tasks.workunit.client.0.smithi076.stdout:0/819: symlink d2/de/d10/d43/d56/lf9 0 2024-03-23T15:44:58.663 INFO:tasks.workunit.client.0.smithi076.stdout:0/820: write d2/de/d10/d43/f50 [606630,39709] 0 2024-03-23T15:44:58.665 INFO:tasks.workunit.client.0.smithi076.stdout:0/821: creat d2/ded/df8/ffa x:0 0 0 2024-03-23T15:44:58.666 INFO:tasks.workunit.client.0.smithi076.stdout:0/822: symlink d2/de/d10/d3a/d47/de3/lfb 0 2024-03-23T15:44:58.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/530: rename d0/d35/d40/d47/c22 to d0/d35/d40/d47/d17/d20/cac 0 2024-03-23T15:44:58.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/531: chown d0/d2/c5c 2842 1 2024-03-23T15:44:58.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/532: fsync d0/d2/f37 0 2024-03-23T15:44:58.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/533: readlink d0/d2/d70/l97 0 2024-03-23T15:44:58.668 INFO:tasks.workunit.client.0.smithi076.stdout:2/883: dread d1/d4/db/f68 [0,4194304] 0 2024-03-23T15:44:58.668 INFO:tasks.workunit.client.0.smithi076.stdout:1/534: rmdir d0/d2/d62 39 2024-03-23T15:44:58.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/884: mknod d1/d4/d64/d82/c131 0 2024-03-23T15:44:58.669 INFO:tasks.workunit.client.0.smithi076.stdout:2/885: write d1/d4/db/d3e/d10c/f111 [905576,42363] 0 2024-03-23T15:44:58.673 INFO:tasks.workunit.client.0.smithi076.stdout:0/823: dread d2/d8/f62 [4194304,4194304] 0 2024-03-23T15:44:58.678 INFO:tasks.workunit.client.0.smithi076.stdout:7/984: sync 2024-03-23T15:44:58.679 INFO:tasks.workunit.client.0.smithi076.stdout:0/824: read d2/d8/f4f [2193110,76559] 0 2024-03-23T15:44:58.684 INFO:tasks.workunit.client.0.smithi076.stdout:2/886: dwrite d1/d4/d64/f20 [0,4194304] 0 2024-03-23T15:44:58.684 INFO:tasks.workunit.client.0.smithi076.stdout:1/535: rename d0/d2/c18 to d0/d35/d40/d47/d55/cad 0 2024-03-23T15:44:58.688 INFO:tasks.workunit.client.0.smithi076.stdout:0/825: dread d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:58.688 INFO:tasks.workunit.client.0.smithi076.stdout:0/826: write d2/de/d10/d43/f50 [1676393,121713] 0 2024-03-23T15:44:58.688 INFO:tasks.workunit.client.0.smithi076.stdout:7/985: dread d8/d4e/d98/df6/d104/d40/d88/d5a/f7f [0,4194304] 0 2024-03-23T15:44:58.690 INFO:tasks.workunit.client.0.smithi076.stdout:7/986: mknod d8/d4e/d98/df6/d104/d3a/d85/c144 0 2024-03-23T15:44:58.692 INFO:tasks.workunit.client.0.smithi076.stdout:7/987: creat d8/d4e/d98/df6/d104/d3a/dbf/f145 x:0 0 0 2024-03-23T15:44:58.694 INFO:tasks.workunit.client.0.smithi076.stdout:1/536: dread d0/d35/d40/fa8 [0,4194304] 0 2024-03-23T15:44:58.694 INFO:tasks.workunit.client.0.smithi076.stdout:1/537: getdents d0/d2/d91/d86 0 2024-03-23T15:44:58.696 INFO:tasks.workunit.client.0.smithi076.stdout:1/538: creat d0/d2/d62/fae x:0 0 0 2024-03-23T15:44:58.698 INFO:tasks.workunit.client.0.smithi076.stdout:1/539: creat d0/d35/d40/d47/d17/faf x:0 0 0 2024-03-23T15:44:58.698 INFO:tasks.workunit.client.0.smithi076.stdout:1/540: dread - d0/d35/d40/d47/d74/d83/f98 zero size 2024-03-23T15:44:58.699 INFO:tasks.workunit.client.0.smithi076.stdout:2/887: dwrite d1/d2e/d6a/f75 [0,4194304] 0 2024-03-23T15:44:58.705 INFO:tasks.workunit.client.0.smithi076.stdout:7/988: write d8/d4e/d98/df6/d104/d24/f3b [1774547,125498] 0 2024-03-23T15:44:58.705 INFO:tasks.workunit.client.0.smithi076.stdout:1/541: read d0/d2/f37 [1995488,112231] 0 2024-03-23T15:44:58.706 INFO:tasks.workunit.client.0.smithi076.stdout:0/827: dwrite d2/d1b/f42 [4194304,4194304] 0 2024-03-23T15:44:58.707 INFO:tasks.workunit.client.0.smithi076.stdout:1/542: mknod d0/d2/d70/d7a/cb0 0 2024-03-23T15:44:58.710 INFO:tasks.workunit.client.0.smithi076.stdout:1/543: truncate d0/d35/d40/fa8 3653436 0 2024-03-23T15:44:58.710 INFO:tasks.workunit.client.0.smithi076.stdout:1/544: stat d0/d2/d70/d7a/cb0 0 2024-03-23T15:44:58.713 INFO:tasks.workunit.client.0.smithi076.stdout:1/545: creat d0/d35/d40/d47/d74/d83/d8f/fb1 x:0 0 0 2024-03-23T15:44:58.713 INFO:tasks.workunit.client.0.smithi076.stdout:1/546: truncate d0/d35/d40/d47/d17/d20/f58 766639 0 2024-03-23T15:44:58.713 INFO:tasks.workunit.client.0.smithi076.stdout:1/547: readlink d0/d2/d91/l59 0 2024-03-23T15:44:58.714 INFO:tasks.workunit.client.0.smithi076.stdout:1/548: dread d0/d35/d40/d47/d17/d20/f58 [0,4194304] 0 2024-03-23T15:44:58.715 INFO:tasks.workunit.client.0.smithi076.stdout:1/549: write d0/d35/d40/d47/d17/d20/f25 [4744213,11712] 0 2024-03-23T15:44:58.715 INFO:tasks.workunit.client.0.smithi076.stdout:1/550: write d0/d35/d40/d47/d17/d20/f58 [981718,112870] 0 2024-03-23T15:44:58.716 INFO:tasks.workunit.client.0.smithi076.stdout:1/551: creat d0/d2/d1f/fb2 x:0 0 0 2024-03-23T15:44:58.716 INFO:tasks.workunit.client.0.smithi076.stdout:1/552: truncate d0/d35/d40/d47/d74/f7f 768418 0 2024-03-23T15:44:58.720 INFO:tasks.workunit.client.0.smithi076.stdout:1/553: write d0/d2/d1f/f43 [516330,62207] 0 2024-03-23T15:44:58.721 INFO:tasks.workunit.client.0.smithi076.stdout:8/697: sync 2024-03-23T15:44:58.721 INFO:tasks.workunit.client.0.smithi076.stdout:9/870: sync 2024-03-23T15:44:58.721 INFO:tasks.workunit.client.0.smithi076.stdout:4/820: sync 2024-03-23T15:44:58.721 INFO:tasks.workunit.client.0.smithi076.stdout:3/594: sync 2024-03-23T15:44:58.721 INFO:tasks.workunit.client.0.smithi076.stdout:9/871: chown d1/d1f/d29/d7f/ld6 136 1 2024-03-23T15:44:58.721 INFO:tasks.workunit.client.0.smithi076.stdout:4/821: write d8/f71 [5049530,44776] 0 2024-03-23T15:44:58.723 INFO:tasks.workunit.client.0.smithi076.stdout:3/595: mknod d5/cc6 0 2024-03-23T15:44:58.724 INFO:tasks.workunit.client.0.smithi076.stdout:8/698: mkdir d8/dd0/df0 0 2024-03-23T15:44:58.725 INFO:tasks.workunit.client.0.smithi076.stdout:9/872: truncate d1/f35 1245477 0 2024-03-23T15:44:58.726 INFO:tasks.workunit.client.0.smithi076.stdout:4/822: mknod d8/de3/db0/de9/c105 0 2024-03-23T15:44:58.727 INFO:tasks.workunit.client.0.smithi076.stdout:3/596: symlink d5/d17/d1c/d32/d8e/lc7 0 2024-03-23T15:44:58.729 INFO:tasks.workunit.client.0.smithi076.stdout:2/888: dwrite d1/d2e/d6a/f9b [0,4194304] 0 2024-03-23T15:44:58.729 INFO:tasks.workunit.client.0.smithi076.stdout:2/889: chown d1/d4/d64/d82/d103/c114 44133785 1 2024-03-23T15:44:58.729 INFO:tasks.workunit.client.0.smithi076.stdout:4/823: dread d8/f6a [0,4194304] 0 2024-03-23T15:44:58.729 INFO:tasks.workunit.client.0.smithi076.stdout:4/824: dread - d8/d3a/d69/d73/df4/fcf zero size 2024-03-23T15:44:58.729 INFO:tasks.workunit.client.0.smithi076.stdout:8/699: rename d8/d26/d27/da9/dea to d8/d2e/d3e/deb/df1 0 2024-03-23T15:44:58.729 INFO:tasks.workunit.client.0.smithi076.stdout:0/828: dwrite d2/de/d10/f68 [0,4194304] 0 2024-03-23T15:44:58.730 INFO:tasks.workunit.client.0.smithi076.stdout:3/597: mknod d5/d35/d6d/cc8 0 2024-03-23T15:44:58.731 INFO:tasks.workunit.client.0.smithi076.stdout:4/825: dread d8/d9/f5a [0,4194304] 0 2024-03-23T15:44:58.732 INFO:tasks.workunit.client.0.smithi076.stdout:1/554: dread d0/d35/d40/d47/d17/f2c [0,4194304] 0 2024-03-23T15:44:58.733 INFO:tasks.workunit.client.0.smithi076.stdout:2/890: dread d1/d4/db/d3e/d10c/f111 [0,4194304] 0 2024-03-23T15:44:58.735 INFO:tasks.workunit.client.0.smithi076.stdout:2/891: mknod d1/d2e/dd1/ddc/c132 0 2024-03-23T15:44:58.736 INFO:tasks.workunit.client.0.smithi076.stdout:0/829: creat d2/de/d10/d43/df3/df7/ffc x:0 0 0 2024-03-23T15:44:58.737 INFO:tasks.workunit.client.0.smithi076.stdout:1/555: dread d0/f26 [0,4194304] 0 2024-03-23T15:44:58.739 INFO:tasks.workunit.client.0.smithi076.stdout:0/830: getdents d2/de/d10/d12/da6 0 2024-03-23T15:44:58.740 INFO:tasks.workunit.client.0.smithi076.stdout:8/700: dread f7 [4194304,4194304] 0 2024-03-23T15:44:58.740 INFO:tasks.workunit.client.0.smithi076.stdout:1/556: mknod d0/d2/d70/d7a/cb3 0 2024-03-23T15:44:58.740 INFO:tasks.workunit.client.0.smithi076.stdout:7/989: dwrite d8/f11e [0,4194304] 0 2024-03-23T15:44:58.741 INFO:tasks.workunit.client.0.smithi076.stdout:1/557: symlink d0/d35/d40/d47/d74/d83/d8f/lb4 0 2024-03-23T15:44:58.742 INFO:tasks.workunit.client.0.smithi076.stdout:1/558: fdatasync d0/d2/d62/f9e 0 2024-03-23T15:44:58.742 INFO:tasks.workunit.client.0.smithi076.stdout:8/701: dread d8/f23 [0,4194304] 0 2024-03-23T15:44:58.743 INFO:tasks.workunit.client.0.smithi076.stdout:3/598: write d5/d17/f29 [1546786,2190] 0 2024-03-23T15:44:58.746 INFO:tasks.workunit.client.0.smithi076.stdout:9/873: link d1/d1f/d29/d2d/d37/f59 d1/d1f/d46/d4a/d5a/daa/f11b 0 2024-03-23T15:44:58.751 INFO:tasks.workunit.client.0.smithi076.stdout:2/892: dread d1/d4/db/d3e/f84 [0,4194304] 0 2024-03-23T15:44:58.753 INFO:tasks.workunit.client.0.smithi076.stdout:7/990: mkdir d8/d4e/d98/df6/d104/d24/de4/d146 0 2024-03-23T15:44:58.754 INFO:tasks.workunit.client.0.smithi076.stdout:7/991: chown d8/d4e/d98/df6/d104/d18/dcc/c12b 59041688 1 2024-03-23T15:44:58.755 INFO:tasks.workunit.client.0.smithi076.stdout:2/893: dread d1/d4/d64/d1b/f63 [0,4194304] 0 2024-03-23T15:44:58.758 INFO:tasks.workunit.client.0.smithi076.stdout:3/599: link d5/d17/d1c/d32/d39/d59/f88 d5/d35/d3c/d67/fc9 0 2024-03-23T15:44:58.758 INFO:tasks.workunit.client.0.smithi076.stdout:3/600: stat d5/d35/d4c/d5a/f49 0 2024-03-23T15:44:58.759 INFO:tasks.workunit.client.0.smithi076.stdout:2/894: write d1/d4/db/d15/d29/d7d/f9d [56728,73319] 0 2024-03-23T15:44:58.759 INFO:tasks.workunit.client.0.smithi076.stdout:7/992: rename d8/d4e/d98/df6/d104/f2f to d8/d4e/d98/df6/d104/d40/d88/d5a/dd9/f147 0 2024-03-23T15:44:58.760 INFO:tasks.workunit.client.0.smithi076.stdout:3/601: mknod d5/d35/d3c/d67/cca 0 2024-03-23T15:44:58.760 INFO:tasks.workunit.client.0.smithi076.stdout:2/895: symlink d1/d4/db/d15/d29/l133 0 2024-03-23T15:44:58.765 INFO:tasks.workunit.client.0.smithi076.stdout:0/831: dwrite d2/de/d10/d43/df3/df7/ffc [0,4194304] 0 2024-03-23T15:44:58.767 INFO:tasks.workunit.client.0.smithi076.stdout:9/874: dread d1/d1f/d46/d4a/d5a/daa/f11b [0,4194304] 0 2024-03-23T15:44:58.768 INFO:tasks.workunit.client.0.smithi076.stdout:3/602: read d5/d17/d1c/d32/d70/fc1 [3623857,2650] 0 2024-03-23T15:44:58.769 INFO:tasks.workunit.client.0.smithi076.stdout:9/875: mkdir d1/d1f/d29/d2d/d37/d47/d11c 0 2024-03-23T15:44:58.769 INFO:tasks.workunit.client.0.smithi076.stdout:9/876: stat d1/d1f/d46/d83/dad/l6d 0 2024-03-23T15:44:58.769 INFO:tasks.workunit.client.0.smithi076.stdout:0/832: getdents d2/daf 0 2024-03-23T15:44:58.770 INFO:tasks.workunit.client.0.smithi076.stdout:0/833: write d2/de/f1a [5020156,15880] 0 2024-03-23T15:44:58.770 INFO:tasks.workunit.client.0.smithi076.stdout:0/834: stat d2/de/d10/d3a/fce 0 2024-03-23T15:44:58.771 INFO:tasks.workunit.client.0.smithi076.stdout:7/993: write d8/f62 [2722929,103653] 0 2024-03-23T15:44:58.773 INFO:tasks.workunit.client.0.smithi076.stdout:9/877: creat d1/d2f/d3f/f11d x:0 0 0 2024-03-23T15:44:58.773 INFO:tasks.workunit.client.0.smithi076.stdout:2/896: getdents d1 0 2024-03-23T15:44:58.773 INFO:tasks.workunit.client.0.smithi076.stdout:2/897: fdatasync d1/d4/d64/d1b/f6c 0 2024-03-23T15:44:58.774 INFO:tasks.workunit.client.0.smithi076.stdout:0/835: symlink d2/de/d10/d3a/lfd 0 2024-03-23T15:44:58.774 INFO:tasks.workunit.client.0.smithi076.stdout:0/836: truncate d2/de/d10/d3a/feb 690351 0 2024-03-23T15:44:58.775 INFO:tasks.workunit.client.0.smithi076.stdout:2/898: link d1/d4/d64/d7c/l105 d1/d4/d64/d82/d103/d79/dac/l134 0 2024-03-23T15:44:58.777 INFO:tasks.workunit.client.0.smithi076.stdout:7/994: mkdir d8/d4e/d98/df6/d104/d3a/d85/daf/dca/d11a/d148 0 2024-03-23T15:44:58.777 INFO:tasks.workunit.client.0.smithi076.stdout:1/559: dwrite d0/d35/f68 [0,4194304] 0 2024-03-23T15:44:58.778 INFO:tasks.workunit.client.0.smithi076.stdout:8/702: dwrite d8/d4f/d80/da3/fdf [0,4194304] 0 2024-03-23T15:44:58.778 INFO:tasks.workunit.client.0.smithi076.stdout:8/703: dread - d8/d26/d27/d56/fd6 zero size 2024-03-23T15:44:58.780 INFO:tasks.workunit.client.0.smithi076.stdout:8/704: symlink d8/d6b/d9d/lf2 0 2024-03-23T15:44:58.781 INFO:tasks.workunit.client.0.smithi076.stdout:2/899: rename d1/d4/d64/d82/d103/f115 to d1/d2e/d6a/f135 0 2024-03-23T15:44:58.781 INFO:tasks.workunit.client.0.smithi076.stdout:2/900: write d1/d4/d64/d82/d103/d79/dac/dd8/dea/fef [804797,28119] 0 2024-03-23T15:44:58.782 INFO:tasks.workunit.client.0.smithi076.stdout:8/705: mknod d8/d2e/d3e/cf3 0 2024-03-23T15:44:58.783 INFO:tasks.workunit.client.0.smithi076.stdout:2/901: creat d1/d4/d64/d1b/dd4/f136 x:0 0 0 2024-03-23T15:44:58.784 INFO:tasks.workunit.client.0.smithi076.stdout:8/706: rename d8/d26/f46 to d8/d6b/ff4 0 2024-03-23T15:44:58.784 INFO:tasks.workunit.client.0.smithi076.stdout:8/707: fdatasync f3 0 2024-03-23T15:44:58.784 INFO:tasks.workunit.client.0.smithi076.stdout:8/708: fdatasync d8/f16 0 2024-03-23T15:44:58.785 INFO:tasks.workunit.client.0.smithi076.stdout:2/902: creat d1/d4/d64/d82/d103/d79/dac/dd8/dea/f137 x:0 0 0 2024-03-23T15:44:58.785 INFO:tasks.workunit.client.0.smithi076.stdout:2/903: read - d1/d4/d64/d82/d103/f126 zero size 2024-03-23T15:44:58.786 INFO:tasks.workunit.client.0.smithi076.stdout:9/878: dread d1/d1f/d46/d83/dad/fc5 [0,4194304] 0 2024-03-23T15:44:58.786 INFO:tasks.workunit.client.0.smithi076.stdout:9/879: chown d1/d2f/d9f/cf3 2873 1 2024-03-23T15:44:58.787 INFO:tasks.workunit.client.0.smithi076.stdout:9/880: getdents d1/d2f/d3f/da0/d119 0 2024-03-23T15:44:58.787 INFO:tasks.workunit.client.0.smithi076.stdout:9/881: fdatasync d1/d2f/d3a/d74/d102/f117 0 2024-03-23T15:44:58.789 INFO:tasks.workunit.client.0.smithi076.stdout:9/882: creat d1/dfd/f11e x:0 0 0 2024-03-23T15:44:58.789 INFO:tasks.workunit.client.0.smithi076.stdout:9/883: chown d1/c27 3094 1 2024-03-23T15:44:58.790 INFO:tasks.workunit.client.0.smithi076.stdout:2/904: mknod d1/d4/d52/ded/c138 0 2024-03-23T15:44:58.790 INFO:tasks.workunit.client.0.smithi076.stdout:3/603: dwrite d5/d35/d3c/d67/fbc [0,4194304] 0 2024-03-23T15:44:58.790 INFO:tasks.workunit.client.0.smithi076.stdout:2/905: fsync d1/d4/d64/d1b/f44 0 2024-03-23T15:44:58.798 INFO:tasks.workunit.client.0.smithi076.stdout:1/560: dwrite d0/d35/d40/d47/d17/f1d [0,4194304] 0 2024-03-23T15:44:58.799 INFO:tasks.workunit.client.0.smithi076.stdout:0/837: dwrite d2/d8/f25 [0,4194304] 0 2024-03-23T15:44:58.799 INFO:tasks.workunit.client.0.smithi076.stdout:5/850: sync 2024-03-23T15:44:58.799 INFO:tasks.workunit.client.0.smithi076.stdout:6/644: sync 2024-03-23T15:44:58.800 INFO:tasks.workunit.client.0.smithi076.stdout:8/709: dwrite d8/d2e/d3e/f44 [0,4194304] 0 2024-03-23T15:44:58.800 INFO:tasks.workunit.client.0.smithi076.stdout:0/838: rmdir d2/d1b/d77 39 2024-03-23T15:44:58.800 INFO:tasks.workunit.client.0.smithi076.stdout:0/839: chown d2/de/d10/d3a/feb 41669795 1 2024-03-23T15:44:58.801 INFO:tasks.workunit.client.0.smithi076.stdout:8/710: symlink d8/d26/d33/dae/lf5 0 2024-03-23T15:44:58.801 INFO:tasks.workunit.client.0.smithi076.stdout:8/711: read - d8/d6b/d91/f98 zero size 2024-03-23T15:44:58.801 INFO:tasks.workunit.client.0.smithi076.stdout:8/712: write d8/d26/d33/f86 [1506243,127398] 0 2024-03-23T15:44:58.801 INFO:tasks.workunit.client.0.smithi076.stdout:5/851: dread d1a/d33/d3a/d67/da8/dc4/f93 [0,4194304] 0 2024-03-23T15:44:58.806 INFO:tasks.workunit.client.0.smithi076.stdout:1/561: dread d0/d2/f75 [0,4194304] 0 2024-03-23T15:44:58.806 INFO:tasks.workunit.client.0.smithi076.stdout:1/562: fsync d0/f2e 0 2024-03-23T15:44:58.808 INFO:tasks.workunit.client.0.smithi076.stdout:0/840: truncate d2/de/d10/d43/f88 851519 0 2024-03-23T15:44:58.809 INFO:tasks.workunit.client.0.smithi076.stdout:0/841: truncate d2/de/d10/d3a/d47/f59 131403 0 2024-03-23T15:44:58.809 INFO:tasks.workunit.client.0.smithi076.stdout:0/842: readlink d2/d1b/le8 0 2024-03-23T15:44:58.810 INFO:tasks.workunit.client.0.smithi076.stdout:5/852: dread d1a/d33/f54 [0,4194304] 0 2024-03-23T15:44:58.811 INFO:tasks.workunit.client.0.smithi076.stdout:0/843: creat d2/de/d10/ffe x:0 0 0 2024-03-23T15:44:58.812 INFO:tasks.workunit.client.0.smithi076.stdout:0/844: stat d2/de/f1a 0 2024-03-23T15:44:58.813 INFO:tasks.workunit.client.0.smithi076.stdout:5/853: rmdir d1a/d33 39 2024-03-23T15:44:58.813 INFO:tasks.workunit.client.0.smithi076.stdout:8/713: write d8/d2e/d30/f6a [375604,62397] 0 2024-03-23T15:44:58.817 INFO:tasks.workunit.client.0.smithi076.stdout:6/645: dread d1/d2/f38 [0,4194304] 0 2024-03-23T15:44:58.817 INFO:tasks.workunit.client.0.smithi076.stdout:6/646: stat d1/dd/df/d9b/db3/d8d/d60/fae 0 2024-03-23T15:44:58.817 INFO:tasks.workunit.client.0.smithi076.stdout:0/845: dread d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:58.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:58 smithi192 ceph-mon[36653]: scrub summary: idle 2024-03-23T15:44:58.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:58 smithi192 ceph-mon[36653]: scrub summary: idle 2024-03-23T15:44:58.820 INFO:tasks.workunit.client.0.smithi076.stdout:0/846: mknod d2/de/d10/d3a/d47/d69/da3/cff 0 2024-03-23T15:44:58.820 INFO:tasks.workunit.client.0.smithi076.stdout:0/847: chown d2/de/d10/l72 23 1 2024-03-23T15:44:58.821 INFO:tasks.workunit.client.0.smithi076.stdout:0/848: write d2/de/d10/d43/f81 [2098604,103658] 0 2024-03-23T15:44:58.823 INFO:tasks.workunit.client.0.smithi076.stdout:0/849: symlink d2/daf/l100 0 2024-03-23T15:44:58.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:58 smithi076 ceph-mon[32176]: scrub summary: idle 2024-03-23T15:44:58.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:58 smithi076 ceph-mon[32176]: scrub summary: idle 2024-03-23T15:44:58.826 INFO:tasks.workunit.client.0.smithi076.stdout:0/850: truncate d2/de/d10/f8c 2965694 0 2024-03-23T15:44:58.830 INFO:tasks.workunit.client.0.smithi076.stdout:1/563: dwrite d0/d35/d40/d47/d74/f7f [0,4194304] 0 2024-03-23T15:44:58.831 INFO:tasks.workunit.client.0.smithi076.stdout:1/564: fdatasync d0/d35/d40/d47/d55/f56 0 2024-03-23T15:44:58.832 INFO:tasks.workunit.client.0.smithi076.stdout:1/565: symlink d0/d2/d70/d7a/lb5 0 2024-03-23T15:44:58.835 INFO:tasks.workunit.client.0.smithi076.stdout:5/854: dwrite d1a/f3c [0,4194304] 0 2024-03-23T15:44:58.835 INFO:tasks.workunit.client.0.smithi076.stdout:1/566: creat d0/d2/d70/fb6 x:0 0 0 2024-03-23T15:44:58.836 INFO:tasks.workunit.client.0.smithi076.stdout:6/647: dwrite d1/d81/f9a [0,4194304] 0 2024-03-23T15:44:58.836 INFO:tasks.workunit.client.0.smithi076.stdout:5/855: fdatasync d1a/d33/f51 0 2024-03-23T15:44:58.838 INFO:tasks.workunit.client.0.smithi076.stdout:6/648: getdents d1/dd/d2a 0 2024-03-23T15:44:58.838 INFO:tasks.workunit.client.0.smithi076.stdout:1/567: creat d0/d2/d70/d7a/d8b/fb7 x:0 0 0 2024-03-23T15:44:58.839 INFO:tasks.workunit.client.0.smithi076.stdout:5/856: truncate f4 3058173 0 2024-03-23T15:44:58.841 INFO:tasks.workunit.client.0.smithi076.stdout:9/884: rename d1/d1f/d46/d83/da7 to d1/d2f/d3a/d11f 0 2024-03-23T15:44:58.845 INFO:tasks.workunit.client.0.smithi076.stdout:0/851: dwrite d2/de/d10/d3a/fce [0,4194304] 0 2024-03-23T15:44:58.846 INFO:tasks.workunit.client.0.smithi076.stdout:8/714: rename d8/d26/fcf to d8/d4f/ff6 0 2024-03-23T15:44:58.850 INFO:tasks.workunit.client.0.smithi076.stdout:6/649: dread d1/dd/f54 [0,4194304] 0 2024-03-23T15:44:58.850 INFO:tasks.workunit.client.0.smithi076.stdout:1/568: dread d0/d2/d91/f5b [8388608,4194304] 0 2024-03-23T15:44:58.852 INFO:tasks.workunit.client.0.smithi076.stdout:0/852: rename d2/d1b/fa8 to d2/f101 0 2024-03-23T15:44:58.856 INFO:tasks.workunit.client.0.smithi076.stdout:1/569: dread d0/d35/f64 [0,4194304] 0 2024-03-23T15:44:58.856 INFO:tasks.workunit.client.0.smithi076.stdout:9/885: dread d1/d1f/d46/d4a/f50 [0,4194304] 0 2024-03-23T15:44:58.857 INFO:tasks.workunit.client.0.smithi076.stdout:5/857: dwrite d1a/d33/d3a/d67/da8/dc4/f93 [0,4194304] 0 2024-03-23T15:44:58.858 INFO:tasks.workunit.client.0.smithi076.stdout:0/853: unlink d2/de/d10/d3a/d47/d69/d84/f93 0 2024-03-23T15:44:58.858 INFO:tasks.workunit.client.0.smithi076.stdout:5/858: symlink d1a/d33/d3a/d67/da8/dc4/d65/l124 0 2024-03-23T15:44:58.858 INFO:tasks.workunit.client.0.smithi076.stdout:5/859: write f19 [1581871,102328] 0 2024-03-23T15:44:58.861 INFO:tasks.workunit.client.0.smithi076.stdout:0/854: symlink d2/de/d10/d3a/d47/de3/l102 0 2024-03-23T15:44:58.861 INFO:tasks.workunit.client.0.smithi076.stdout:0/855: chown d2/l2a 4843475 1 2024-03-23T15:44:58.861 INFO:tasks.workunit.client.0.smithi076.stdout:0/856: fsync d2/de/d10/d12/da6/fd2 0 2024-03-23T15:44:58.866 INFO:tasks.workunit.client.0.smithi076.stdout:0/857: dread d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:58.867 INFO:tasks.workunit.client.0.smithi076.stdout:6/650: dwrite d1/dd/df/fa7 [0,4194304] 0 2024-03-23T15:44:58.868 INFO:tasks.workunit.client.0.smithi076.stdout:0/858: getdents d2/de/d10/d3a 0 2024-03-23T15:44:58.873 INFO:tasks.workunit.client.0.smithi076.stdout:0/859: rmdir d2/de/de0 39 2024-03-23T15:44:58.877 INFO:tasks.workunit.client.0.smithi076.stdout:5/860: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/ffc [0,4194304] 0 2024-03-23T15:44:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:58 smithi183 ceph-mon[36296]: scrub summary: idle 2024-03-23T15:44:58.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:58 smithi183 ceph-mon[36296]: scrub summary: idle 2024-03-23T15:44:58.884 INFO:tasks.workunit.client.0.smithi076.stdout:5/861: write d1a/d4b/fb4 [2485547,125868] 0 2024-03-23T15:44:58.884 INFO:tasks.workunit.client.0.smithi076.stdout:5/862: dread - d1a/d33/d3a/d80/ff8 zero size 2024-03-23T15:44:58.890 INFO:tasks.workunit.client.0.smithi076.stdout:5/863: dread d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/ffc [0,4194304] 0 2024-03-23T15:44:58.890 INFO:tasks.workunit.client.0.smithi076.stdout:5/864: truncate d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/fbd 462499 0 2024-03-23T15:44:58.890 INFO:tasks.workunit.client.0.smithi076.stdout:0/860: dwrite d2/d1b/fb5 [0,4194304] 0 2024-03-23T15:44:58.891 INFO:tasks.workunit.client.0.smithi076.stdout:5/865: getdents d1a/d33/dc3/da6 0 2024-03-23T15:44:58.892 INFO:tasks.workunit.client.0.smithi076.stdout:5/866: truncate d1a/d33/dc3/fd6 2487917 0 2024-03-23T15:44:58.899 INFO:tasks.workunit.client.0.smithi076.stdout:1/570: unlink d0/d35/d40/d47/d17/d20/l92 0 2024-03-23T15:44:58.902 INFO:tasks.workunit.client.0.smithi076.stdout:8/715: rename d8/d26/d33/dae to d8/d4f/d80/da3/df7 0 2024-03-23T15:44:58.902 INFO:tasks.workunit.client.0.smithi076.stdout:8/716: fsync d8/d83/f95 0 2024-03-23T15:44:58.905 INFO:tasks.workunit.client.0.smithi076.stdout:1/571: symlink d0/d3c/lb8 0 2024-03-23T15:44:58.905 INFO:tasks.workunit.client.0.smithi076.stdout:8/717: mkdir d8/d2e/d30/d69/df8 0 2024-03-23T15:44:58.907 INFO:tasks.workunit.client.0.smithi076.stdout:1/572: creat d0/d35/d40/d47/d55/fb9 x:0 0 0 2024-03-23T15:44:58.907 INFO:tasks.workunit.client.0.smithi076.stdout:1/573: chown d0/d2/d62/f9e 89 1 2024-03-23T15:44:58.907 INFO:tasks.workunit.client.0.smithi076.stdout:9/886: rename d1/d2f/d3a/d74/da5 to d1/d2f/d3a/d74/d102/d120 0 2024-03-23T15:44:58.907 INFO:tasks.workunit.client.0.smithi076.stdout:8/718: symlink d8/d2e/d30/d69/db9/lf9 0 2024-03-23T15:44:58.907 INFO:tasks.workunit.client.0.smithi076.stdout:8/719: stat d8/c62 0 2024-03-23T15:44:58.908 INFO:tasks.workunit.client.0.smithi076.stdout:8/720: write d8/d2e/d30/d69/dd3/fed [383805,69749] 0 2024-03-23T15:44:58.908 INFO:tasks.workunit.client.0.smithi076.stdout:8/721: dread - d8/f5a zero size 2024-03-23T15:44:58.909 INFO:tasks.workunit.client.0.smithi076.stdout:6/651: dwrite d1/dd/df/d9b/db3/d8d/f6a [8388608,4194304] 0 2024-03-23T15:44:58.909 INFO:tasks.workunit.client.0.smithi076.stdout:0/861: dread d2/de/d10/d43/d56/f60 [0,4194304] 0 2024-03-23T15:44:58.911 INFO:tasks.workunit.client.0.smithi076.stdout:1/574: read d0/d35/d40/d47/d17/d20/f46 [1692322,91033] 0 2024-03-23T15:44:58.911 INFO:tasks.workunit.client.0.smithi076.stdout:1/575: dread - d0/d2/d62/fae zero size 2024-03-23T15:44:58.911 INFO:tasks.workunit.client.0.smithi076.stdout:1/576: chown d0/d35/d40/d47/cf 0 1 2024-03-23T15:44:58.912 INFO:tasks.workunit.client.0.smithi076.stdout:1/577: mknod d0/d2/d70/d7a/d8b/cba 0 2024-03-23T15:44:58.914 INFO:tasks.workunit.client.0.smithi076.stdout:1/578: link d0/d35/d40/d47/d17/d20/cac d0/d35/d40/d47/d55/cbb 0 2024-03-23T15:44:58.915 INFO:tasks.workunit.client.0.smithi076.stdout:6/652: creat d1/dd/df/d5b/d62/dc7/fe8 x:0 0 0 2024-03-23T15:44:58.916 INFO:tasks.workunit.client.0.smithi076.stdout:9/887: write d1/f18 [2888511,77193] 0 2024-03-23T15:44:58.917 INFO:tasks.workunit.client.0.smithi076.stdout:6/653: creat d1/d4/d10/fe9 x:0 0 0 2024-03-23T15:44:58.917 INFO:tasks.workunit.client.0.smithi076.stdout:6/654: write d1/d4/d10/fdc [883125,66210] 0 2024-03-23T15:44:58.917 INFO:tasks.workunit.client.0.smithi076.stdout:6/655: write d1/d2/fb4 [654725,91018] 0 2024-03-23T15:44:58.920 INFO:tasks.workunit.client.0.smithi076.stdout:5/867: rename d1a/d33/d3a/d67/da8/dc4/f93 to d1a/f125 0 2024-03-23T15:44:58.920 INFO:tasks.workunit.client.0.smithi076.stdout:5/868: dread - d1a/d33/d3a/fda zero size 2024-03-23T15:44:58.927 INFO:tasks.workunit.client.0.smithi076.stdout:9/888: dread d1/d1f/d29/d2d/d37/f59 [0,4194304] 0 2024-03-23T15:44:58.928 INFO:tasks.workunit.client.0.smithi076.stdout:8/722: dwrite d8/f18 [0,4194304] 0 2024-03-23T15:44:58.929 INFO:tasks.workunit.client.0.smithi076.stdout:5/869: dread d1a/d33/d3a/d67/da8/dc4/d5f/f9b [0,4194304] 0 2024-03-23T15:44:58.931 INFO:tasks.workunit.client.0.smithi076.stdout:8/723: link d8/d6b/d9d/lf2 d8/d2e/d3e/deb/df1/lfa 0 2024-03-23T15:44:58.932 INFO:tasks.workunit.client.0.smithi076.stdout:5/870: dread d1a/d2c/f87 [0,4194304] 0 2024-03-23T15:44:58.932 INFO:tasks.workunit.client.0.smithi076.stdout:5/871: fsync d1a/d33/d3a/d67/d84/f11b 0 2024-03-23T15:44:58.933 INFO:tasks.workunit.client.0.smithi076.stdout:8/724: mkdir d8/d6b/dfb 0 2024-03-23T15:44:58.933 INFO:tasks.workunit.client.0.smithi076.stdout:1/579: dwrite d0/d3c/fa0 [0,4194304] 0 2024-03-23T15:44:58.937 INFO:tasks.workunit.client.0.smithi076.stdout:8/725: unlink d8/l25 0 2024-03-23T15:44:58.940 INFO:tasks.workunit.client.0.smithi076.stdout:1/580: mkdir d0/d2/d70/d7a/d8b/dbc 0 2024-03-23T15:44:58.950 INFO:tasks.workunit.client.0.smithi076.stdout:9/889: dwrite d1/dfd/f11e [0,4194304] 0 2024-03-23T15:44:58.950 INFO:tasks.workunit.client.0.smithi076.stdout:9/890: dread - d1/d2f/ff1 zero size 2024-03-23T15:44:58.951 INFO:tasks.workunit.client.0.smithi076.stdout:9/891: fsync d1/d1f/d29/d2d/d37/f59 0 2024-03-23T15:44:58.951 INFO:tasks.workunit.client.0.smithi076.stdout:5/872: dwrite d1a/f98 [0,4194304] 0 2024-03-23T15:44:58.954 INFO:tasks.workunit.client.0.smithi076.stdout:5/873: unlink d1a/d33/d3a/d67/da8/dc4/d5f/ddb/lfa 0 2024-03-23T15:44:58.957 INFO:tasks.workunit.client.0.smithi076.stdout:1/581: dwrite d0/d2/f48 [0,4194304] 0 2024-03-23T15:44:58.960 INFO:tasks.workunit.client.0.smithi076.stdout:1/582: dread d0/d2/f75 [0,4194304] 0 2024-03-23T15:44:58.962 INFO:tasks.workunit.client.0.smithi076.stdout:8/726: dwrite d8/f14 [0,4194304] 0 2024-03-23T15:44:58.963 INFO:tasks.workunit.client.0.smithi076.stdout:1/583: unlink d0/d35/d40/d47/d74/f7f 0 2024-03-23T15:44:58.963 INFO:tasks.workunit.client.0.smithi076.stdout:1/584: fdatasync d0/d3c/f6c 0 2024-03-23T15:44:58.964 INFO:tasks.workunit.client.0.smithi076.stdout:1/585: mkdir d0/d35/d40/d47/d74/d83/dbd 0 2024-03-23T15:44:58.965 INFO:tasks.workunit.client.0.smithi076.stdout:8/727: getdents d8/d4f/d80 0 2024-03-23T15:44:58.966 INFO:tasks.workunit.client.0.smithi076.stdout:1/586: creat d0/d2/d70/d7a/d8b/fbe x:0 0 0 2024-03-23T15:44:58.973 INFO:tasks.workunit.client.0.smithi076.stdout:1/587: dread d0/d35/d40/d47/d17/f1d [0,4194304] 0 2024-03-23T15:44:58.973 INFO:tasks.workunit.client.0.smithi076.stdout:1/588: read d0/d35/d40/d47/d17/d20/f46 [2475168,107257] 0 2024-03-23T15:44:58.973 INFO:tasks.workunit.client.0.smithi076.stdout:1/589: fsync d0/d2/d91/f5b 0 2024-03-23T15:44:58.974 INFO:tasks.workunit.client.0.smithi076.stdout:9/892: rename d1/d1f/d46/d4a/d5a/daa/ffc to d1/d2f/d3a/d53/f121 0 2024-03-23T15:44:58.976 INFO:tasks.workunit.client.0.smithi076.stdout:5/874: rename d1a/d33/d3a/d67/da8/l10a to d1a/d33/dc3/l126 0 2024-03-23T15:44:58.978 INFO:tasks.workunit.client.0.smithi076.stdout:8/728: rename d8/d26/d33/d8c/c9c to d8/d89/db2/dd5/cfc 0 2024-03-23T15:44:58.979 INFO:tasks.workunit.client.0.smithi076.stdout:5/875: link d1a/d33/d3a/d67/da8/dc4/f52 d1a/d33/d3a/d67/da8/dc4/d119/f127 0 2024-03-23T15:44:58.981 INFO:tasks.workunit.client.0.smithi076.stdout:1/590: rename d0/d2/d1f to d0/d2/d70/da1/dbf 0 2024-03-23T15:44:58.982 INFO:tasks.workunit.client.0.smithi076.stdout:8/729: creat d8/d26/d33/ddd/ffd x:0 0 0 2024-03-23T15:44:58.983 INFO:tasks.workunit.client.0.smithi076.stdout:1/591: getdents d0/d2/d70/d7a/d8b 0 2024-03-23T15:44:58.984 INFO:tasks.workunit.client.0.smithi076.stdout:1/592: link d0/d35/d40/d47/d55/ca6 d0/d35/d40/d47/d8c/cc0 0 2024-03-23T15:44:58.985 INFO:tasks.workunit.client.0.smithi076.stdout:1/593: mknod d0/d2/d91/d86/cc1 0 2024-03-23T15:44:58.985 INFO:tasks.workunit.client.0.smithi076.stdout:1/594: fsync d0/d35/d40/f90 0 2024-03-23T15:44:58.986 INFO:tasks.workunit.client.0.smithi076.stdout:1/595: creat d0/d35/d40/d47/d74/d83/fc2 x:0 0 0 2024-03-23T15:44:58.986 INFO:tasks.workunit.client.0.smithi076.stdout:5/876: write d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/ddf/ff9 [3137137,81338] 0 2024-03-23T15:44:58.990 INFO:tasks.workunit.client.0.smithi076.stdout:5/877: creat d1a/d33/d3a/d67/da8/dc4/d119/f128 x:0 0 0 2024-03-23T15:44:58.990 INFO:tasks.workunit.client.0.smithi076.stdout:5/878: write d1a/d33/d36/fc0 [646994,49190] 0 2024-03-23T15:44:58.991 INFO:tasks.workunit.client.0.smithi076.stdout:9/893: dwrite d1/d1f/d46/d83/dad/f118 [0,4194304] 0 2024-03-23T15:44:58.992 INFO:tasks.workunit.client.0.smithi076.stdout:9/894: getdents d1/d2f/d3f/dc0 0 2024-03-23T15:44:58.993 INFO:tasks.workunit.client.0.smithi076.stdout:1/596: dread d0/d3c/fa0 [0,4194304] 0 2024-03-23T15:44:58.993 INFO:tasks.workunit.client.0.smithi076.stdout:1/597: read d0/d2/d70/da1/dbf/f43 [1616773,23294] 0 2024-03-23T15:44:58.994 INFO:tasks.workunit.client.0.smithi076.stdout:9/895: creat d1/d2f/d3f/da0/dd5/f122 x:0 0 0 2024-03-23T15:44:58.995 INFO:tasks.workunit.client.0.smithi076.stdout:1/598: mkdir d0/dc3 0 2024-03-23T15:44:58.995 INFO:tasks.workunit.client.0.smithi076.stdout:9/896: write d1/d1f/d82/d9c/fdc [136547,108288] 0 2024-03-23T15:44:58.996 INFO:tasks.workunit.client.0.smithi076.stdout:1/599: mkdir d0/d3c/d6e/dc4 0 2024-03-23T15:44:58.996 INFO:tasks.workunit.client.0.smithi076.stdout:1/600: truncate d0/d2/d70/fb6 145577 0 2024-03-23T15:44:58.996 INFO:tasks.workunit.client.0.smithi076.stdout:5/879: unlink d1a/c28 0 2024-03-23T15:44:58.997 INFO:tasks.workunit.client.0.smithi076.stdout:1/601: creat d0/d35/d40/d47/d8c/fc5 x:0 0 0 2024-03-23T15:44:58.998 INFO:tasks.workunit.client.0.smithi076.stdout:1/602: rmdir d0/d35/d40/d47/d17/d20 39 2024-03-23T15:44:58.999 INFO:tasks.workunit.client.0.smithi076.stdout:1/603: creat d0/d35/d40/d47/d55/fc6 x:0 0 0 2024-03-23T15:44:59.000 INFO:tasks.workunit.client.0.smithi076.stdout:1/604: creat d0/d2/fc7 x:0 0 0 2024-03-23T15:44:59.001 INFO:tasks.workunit.client.0.smithi076.stdout:1/605: mknod d0/d2/d70/d7a/d8b/cc8 0 2024-03-23T15:44:59.003 INFO:tasks.workunit.client.0.smithi076.stdout:1/606: creat d0/d2/d91/d86/fc9 x:0 0 0 2024-03-23T15:44:59.003 INFO:tasks.workunit.client.0.smithi076.stdout:1/607: chown d0/d2/f9a 919051756 1 2024-03-23T15:44:59.010 INFO:tasks.workunit.client.0.smithi076.stdout:5/880: dwrite d1a/d33/d3a/f7a [0,4194304] 0 2024-03-23T15:44:59.010 INFO:tasks.workunit.client.0.smithi076.stdout:4/826: sync 2024-03-23T15:44:59.010 INFO:tasks.workunit.client.0.smithi076.stdout:4/827: chown d8/d3a/d18/d1b/l6c 229 1 2024-03-23T15:44:59.010 INFO:tasks.workunit.client.0.smithi076.stdout:4/828: truncate d8/d3a/d3b/f25 1717060 0 2024-03-23T15:44:59.010 INFO:tasks.workunit.client.0.smithi076.stdout:1/608: getdents d0/d3c/d6e 0 2024-03-23T15:44:59.011 INFO:tasks.workunit.client.0.smithi076.stdout:5/881: symlink d1a/d33/d3a/d67/l129 0 2024-03-23T15:44:59.011 INFO:tasks.workunit.client.0.smithi076.stdout:5/882: chown d1a/d33/c86 30 1 2024-03-23T15:44:59.012 INFO:tasks.workunit.client.0.smithi076.stdout:4/829: creat d8/d3a/d18/d52/f106 x:0 0 0 2024-03-23T15:44:59.012 INFO:tasks.workunit.client.0.smithi076.stdout:4/830: write d8/d41/d97/fb7 [4798631,59837] 0 2024-03-23T15:44:59.013 INFO:tasks.workunit.client.0.smithi076.stdout:4/831: truncate d8/d9/fd3 481116 0 2024-03-23T15:44:59.017 INFO:tasks.workunit.client.0.smithi076.stdout:5/883: mkdir d1a/d33/d3a/d67/d84/d12a 0 2024-03-23T15:44:59.017 INFO:tasks.workunit.client.0.smithi076.stdout:4/832: creat d8/d3a/d69/f107 x:0 0 0 2024-03-23T15:44:59.019 INFO:tasks.workunit.client.0.smithi076.stdout:5/884: mknod d1a/d33/dc3/c12b 0 2024-03-23T15:44:59.019 INFO:tasks.workunit.client.0.smithi076.stdout:4/833: mkdir d8/d41/d108 0 2024-03-23T15:44:59.021 INFO:tasks.workunit.client.0.smithi076.stdout:5/885: symlink d1a/d33/d3a/d67/da8/dc4/d5f/d8f/l12c 0 2024-03-23T15:44:59.021 INFO:tasks.workunit.client.0.smithi076.stdout:4/834: link d8/c31 d8/c109 0 2024-03-23T15:44:59.022 INFO:tasks.workunit.client.0.smithi076.stdout:5/886: write d1a/d33/d3a/d67/da8/dc4/ff6 [163731,21813] 0 2024-03-23T15:44:59.022 INFO:tasks.workunit.client.0.smithi076.stdout:5/887: read - d1a/d2c/f11e zero size 2024-03-23T15:44:59.023 INFO:tasks.workunit.client.0.smithi076.stdout:4/835: creat d8/f10a x:0 0 0 2024-03-23T15:44:59.024 INFO:tasks.workunit.client.0.smithi076.stdout:4/836: creat d8/de3/db0/de9/f10b x:0 0 0 2024-03-23T15:44:59.025 INFO:tasks.workunit.client.0.smithi076.stdout:4/837: stat l7 0 2024-03-23T15:44:59.025 INFO:tasks.workunit.client.0.smithi076.stdout:5/888: dread d1a/d33/d3a/d67/da8/dc4/d65/f6b [0,4194304] 0 2024-03-23T15:44:59.027 INFO:tasks.workunit.client.0.smithi076.stdout:5/889: dread d1a/d2c/fce [0,4194304] 0 2024-03-23T15:44:59.027 INFO:tasks.workunit.client.0.smithi076.stdout:5/890: fdatasync d1a/d33/d3a/f8a 0 2024-03-23T15:44:59.027 INFO:tasks.workunit.client.0.smithi076.stdout:1/609: dwrite d0/d2/d70/da1/dbf/fb2 [0,4194304] 0 2024-03-23T15:44:59.030 INFO:tasks.workunit.client.0.smithi076.stdout:5/891: unlink d1a/d33/d3a/d67/da8/dc4/ffe 0 2024-03-23T15:44:59.030 INFO:tasks.workunit.client.0.smithi076.stdout:1/610: mkdir d0/d3c/d6e/dca 0 2024-03-23T15:44:59.034 INFO:tasks.workunit.client.0.smithi076.stdout:1/611: mkdir d0/d2/d70/d7a/d8b/dcb 0 2024-03-23T15:44:59.034 INFO:tasks.workunit.client.0.smithi076.stdout:1/612: write d0/d2/d62/fa7 [556230,121483] 0 2024-03-23T15:44:59.037 INFO:tasks.workunit.client.0.smithi076.stdout:5/892: write d1a/f3c [2542518,16852] 0 2024-03-23T15:44:59.038 INFO:tasks.workunit.client.0.smithi076.stdout:5/893: mknod d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/c12d 0 2024-03-23T15:44:59.040 INFO:tasks.workunit.client.0.smithi076.stdout:5/894: mkdir d1a/d33/d3a/d67/da8/dc4/d65/d12e 0 2024-03-23T15:44:59.057 INFO:tasks.workunit.client.0.smithi076.stdout:1/613: dwrite d0/d35/d40/d47/d17/f38 [0,4194304] 0 2024-03-23T15:44:59.071 INFO:tasks.workunit.client.0.smithi076.stdout:1/614: dwrite d0/d35/f6f [0,4194304] 0 2024-03-23T15:44:59.082 INFO:tasks.workunit.client.0.smithi076.stdout:1/615: rename d0/d35/d40/d47/d17/f8a to d0/d2/d91/d86/fcc 0 2024-03-23T15:44:59.085 INFO:tasks.workunit.client.0.smithi076.stdout:7/995: sync 2024-03-23T15:44:59.085 INFO:tasks.workunit.client.0.smithi076.stdout:7/996: chown d8/d4e/d98/df6/d104/d40/d5d 10 1 2024-03-23T15:44:59.085 INFO:tasks.workunit.client.0.smithi076.stdout:7/997: chown d8/d4e/d98/df6/d104/d24/d91/d96/l78 248922 1 2024-03-23T15:44:59.086 INFO:tasks.workunit.client.0.smithi076.stdout:7/998: write d8/d4e/d98/df6/d104/d3a/dd2/f140 [548188,107186] 0 2024-03-23T15:44:59.092 INFO:tasks.workunit.client.0.smithi076.stdout:7/999: dread d8/f3f [0,4194304] 0 2024-03-23T15:44:59.093 INFO:tasks.workunit.client.0.smithi076.stdout:1/616: rename d0/d35/d40/l4d to d0/d35/d40/d47/d55/lcd 0 2024-03-23T15:44:59.093 INFO:tasks.workunit.client.0.smithi076.stdout:1/617: chown d0/d35/d40/d47/d55/cad 685 1 2024-03-23T15:44:59.110 INFO:tasks.workunit.client.0.smithi076.stdout:1/618: dwrite d0/d35/f60 [0,4194304] 0 2024-03-23T15:44:59.110 INFO:tasks.workunit.client.0.smithi076.stdout:1/619: dread - d0/d35/d40/d47/d8c/d99/fa4 zero size 2024-03-23T15:44:59.111 INFO:tasks.workunit.client.0.smithi076.stdout:1/620: mknod d0/d35/d40/d47/d17/cce 0 2024-03-23T15:44:59.126 INFO:tasks.workunit.client.0.smithi076.stdout:1/621: dwrite d0/d35/d40/d47/d17/f38 [0,4194304] 0 2024-03-23T15:44:59.128 INFO:tasks.workunit.client.0.smithi076.stdout:1/622: symlink d0/d2/d70/d7a/d8b/lcf 0 2024-03-23T15:44:59.128 INFO:tasks.workunit.client.0.smithi076.stdout:1/623: write d0/d35/f3a [866547,17273] 0 2024-03-23T15:44:59.129 INFO:tasks.workunit.client.0.smithi076.stdout:1/624: rename d0/d2/d70/da1/dbf/l72 to d0/d3c/d6e/dc4/ld0 0 2024-03-23T15:44:59.131 INFO:tasks.workunit.client.0.smithi076.stdout:1/625: mkdir d0/d35/d40/dd1 0 2024-03-23T15:44:59.131 INFO:tasks.workunit.client.0.smithi076.stdout:3/604: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:5/895: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:2/906: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:9/897: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:6/656: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:0/862: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:8/730: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:4/838: sync 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:5/896: chown d1a/d33/d3a/d120 29671 1 2024-03-23T15:44:59.133 INFO:tasks.workunit.client.0.smithi076.stdout:2/907: truncate d1/d4/db/d15/d29/d7d/fad 719476 0 2024-03-23T15:44:59.134 INFO:tasks.workunit.client.0.smithi076.stdout:2/908: stat d1/d4/f5f 0 2024-03-23T15:44:59.134 INFO:tasks.workunit.client.0.smithi076.stdout:2/909: read - d1/d4/db/d15/fc4 zero size 2024-03-23T15:44:59.134 INFO:tasks.workunit.client.0.smithi076.stdout:4/839: read d8/d9/d32/f5b [574368,115978] 0 2024-03-23T15:44:59.134 INFO:tasks.workunit.client.0.smithi076.stdout:0/863: write d2/de/d10/d43/fb9 [6571,2821] 0 2024-03-23T15:44:59.134 INFO:tasks.workunit.client.0.smithi076.stdout:8/731: creat d8/d26/d27/ffe x:0 0 0 2024-03-23T15:44:59.135 INFO:tasks.workunit.client.0.smithi076.stdout:2/910: mkdir d1/d2e/dd1/ddc/d139 0 2024-03-23T15:44:59.136 INFO:tasks.workunit.client.0.smithi076.stdout:9/898: creat d1/d2f/d3a/d74/f123 x:0 0 0 2024-03-23T15:44:59.136 INFO:tasks.workunit.client.0.smithi076.stdout:8/732: mkdir d8/d26/d33/d8c/dff 0 2024-03-23T15:44:59.137 INFO:tasks.workunit.client.0.smithi076.stdout:2/911: symlink d1/d2e/dd1/df6/dfd/l13a 0 2024-03-23T15:44:59.141 INFO:tasks.workunit.client.0.smithi076.stdout:3/605: write d5/f16 [3707253,24032] 0 2024-03-23T15:44:59.141 INFO:tasks.workunit.client.0.smithi076.stdout:1/626: write d0/f2e [12864478,59483] 0 2024-03-23T15:44:59.141 INFO:tasks.workunit.client.0.smithi076.stdout:6/657: rmdir d1/dd/df/d9b/db3/d8d/d60 39 2024-03-23T15:44:59.141 INFO:tasks.workunit.client.0.smithi076.stdout:9/899: mknod d1/d1f/d46/d4a/d5a/c124 0 2024-03-23T15:44:59.141 INFO:tasks.workunit.client.0.smithi076.stdout:9/900: write d1/d1f/d29/d2d/f86 [1069019,95220] 0 2024-03-23T15:44:59.142 INFO:tasks.workunit.client.0.smithi076.stdout:3/606: creat d5/d35/d4c/d5a/db4/fcb x:0 0 0 2024-03-23T15:44:59.142 INFO:tasks.workunit.client.0.smithi076.stdout:3/607: chown d5/d17/d1c/d31/d61/d96/cba 0 1 2024-03-23T15:44:59.142 INFO:tasks.workunit.client.0.smithi076.stdout:1/627: creat d0/d35/d40/d47/d55/fd2 x:0 0 0 2024-03-23T15:44:59.143 INFO:tasks.workunit.client.0.smithi076.stdout:3/608: getdents d5/d17/d1c/d32/d70/d8c 0 2024-03-23T15:44:59.143 INFO:tasks.workunit.client.0.smithi076.stdout:3/609: write d5/d35/d6d/d7f/fac [410678,276] 0 2024-03-23T15:44:59.143 INFO:tasks.workunit.client.0.smithi076.stdout:3/610: readlink d5/d17/d1c/d32/d8e/lc7 0 2024-03-23T15:44:59.143 INFO:tasks.workunit.client.0.smithi076.stdout:3/611: fdatasync d5/d35/d4c/d5a/f41 0 2024-03-23T15:44:59.144 INFO:tasks.workunit.client.0.smithi076.stdout:4/840: dread d8/d3a/d18/d52/f79 [0,4194304] 0 2024-03-23T15:44:59.145 INFO:tasks.workunit.client.0.smithi076.stdout:4/841: getdents d8/d3a/d69/de4 0 2024-03-23T15:44:59.145 INFO:tasks.workunit.client.0.smithi076.stdout:4/842: write d8/d3a/d69/f107 [255494,97056] 0 2024-03-23T15:44:59.145 INFO:tasks.workunit.client.0.smithi076.stdout:9/901: unlink d1/d1f/d29/d2d/d37/d47/le4 0 2024-03-23T15:44:59.145 INFO:tasks.workunit.client.0.smithi076.stdout:2/912: dread d1/d4/db/d15/f21 [0,4194304] 0 2024-03-23T15:44:59.146 INFO:tasks.workunit.client.0.smithi076.stdout:1/628: write d0/f26 [2310577,14169] 0 2024-03-23T15:44:59.146 INFO:tasks.workunit.client.0.smithi076.stdout:9/902: creat d1/d2f/d3a/d74/d102/f125 x:0 0 0 2024-03-23T15:44:59.146 INFO:tasks.workunit.client.0.smithi076.stdout:9/903: chown d1/d2f/d3a/d11f/fe5 99 1 2024-03-23T15:44:59.147 INFO:tasks.workunit.client.0.smithi076.stdout:9/904: chown d1/d1f/d29 3664974 1 2024-03-23T15:44:59.147 INFO:tasks.workunit.client.0.smithi076.stdout:2/913: creat d1/d4/d64/d1b/dd4/f13b x:0 0 0 2024-03-23T15:44:59.147 INFO:tasks.workunit.client.0.smithi076.stdout:2/914: chown d1/d2e/d6a/l10a 60 1 2024-03-23T15:44:59.148 INFO:tasks.workunit.client.0.smithi076.stdout:1/629: creat d0/d2/d70/da1/dbf/fd3 x:0 0 0 2024-03-23T15:44:59.148 INFO:tasks.workunit.client.0.smithi076.stdout:3/612: mkdir d5/d17/d1c/d32/d39/dcc 0 2024-03-23T15:44:59.150 INFO:tasks.workunit.client.0.smithi076.stdout:8/733: write d8/d2e/f4a [607423,93618] 0 2024-03-23T15:44:59.154 INFO:tasks.workunit.client.0.smithi076.stdout:8/734: mkdir d8/d2e/d30/d69/dd3/d100 0 2024-03-23T15:44:59.154 INFO:tasks.workunit.client.0.smithi076.stdout:8/735: read - d8/d6b/d91/f98 zero size 2024-03-23T15:44:59.155 INFO:tasks.workunit.client.0.smithi076.stdout:0/864: dwrite d2/de/d10/d43/fb6 [0,4194304] 0 2024-03-23T15:44:59.156 INFO:tasks.workunit.client.0.smithi076.stdout:5/897: dwrite d1a/d33/d3a/fa5 [0,4194304] 0 2024-03-23T15:44:59.156 INFO:tasks.workunit.client.0.smithi076.stdout:3/613: link d5/d17/d1c/d32/d8e/l98 d5/dc2/lcd 0 2024-03-23T15:44:59.156 INFO:tasks.workunit.client.0.smithi076.stdout:5/898: write d1a/d33/d36/fc0 [830788,39287] 0 2024-03-23T15:44:59.158 INFO:tasks.workunit.client.0.smithi076.stdout:9/905: read d1/d2f/d3f/da0/fc9 [2544828,79362] 0 2024-03-23T15:44:59.158 INFO:tasks.workunit.client.0.smithi076.stdout:9/906: dread - d1/d1f/d29/d2d/d37/d47/ff4 zero size 2024-03-23T15:44:59.158 INFO:tasks.workunit.client.0.smithi076.stdout:5/899: mknod d1a/d4b/c12f 0 2024-03-23T15:44:59.159 INFO:tasks.workunit.client.0.smithi076.stdout:9/907: dread - d1/d1f/d82/d9c/fa9 zero size 2024-03-23T15:44:59.159 INFO:tasks.workunit.client.0.smithi076.stdout:9/908: chown d1/d1f/d46/d83/cdb 0 1 2024-03-23T15:44:59.161 INFO:tasks.workunit.client.0.smithi076.stdout:3/614: link d5/d35/d6d/d7f/fac d5/d35/d3c/d67/fce 0 2024-03-23T15:44:59.163 INFO:tasks.workunit.client.0.smithi076.stdout:3/615: mkdir d5/d17/dcf 0 2024-03-23T15:44:59.165 INFO:tasks.workunit.client.0.smithi076.stdout:3/616: mknod d5/d35/d4c/d5a/d9d/cd0 0 2024-03-23T15:44:59.167 INFO:tasks.workunit.client.0.smithi076.stdout:8/736: dread f2 [0,4194304] 0 2024-03-23T15:44:59.167 INFO:tasks.workunit.client.0.smithi076.stdout:8/737: write d8/d2e/d30/d69/fb8 [445747,126446] 0 2024-03-23T15:44:59.167 INFO:tasks.workunit.client.0.smithi076.stdout:8/738: write d8/d26/d33/d8c/fb5 [1494506,79588] 0 2024-03-23T15:44:59.168 INFO:tasks.workunit.client.0.smithi076.stdout:3/617: symlink d5/d35/d6d/d7f/ld1 0 2024-03-23T15:44:59.172 INFO:tasks.workunit.client.0.smithi076.stdout:2/915: dwrite d1/d4/d64/d7c/f85 [0,4194304] 0 2024-03-23T15:44:59.172 INFO:tasks.workunit.client.0.smithi076.stdout:1/630: dwrite d0/d35/d40/d47/d55/fb9 [0,4194304] 0 2024-03-23T15:44:59.176 INFO:tasks.workunit.client.0.smithi076.stdout:0/865: dwrite d2/de/d10/f61 [4194304,4194304] 0 2024-03-23T15:44:59.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/618: write d5/d17/d1c/d32/d39/f3e [924321,113334] 0 2024-03-23T15:44:59.176 INFO:tasks.workunit.client.0.smithi076.stdout:0/866: write d2/d1b/fc9 [2464774,93313] 0 2024-03-23T15:44:59.180 INFO:tasks.workunit.client.0.smithi076.stdout:2/916: dread d1/d4/f54 [0,4194304] 0 2024-03-23T15:44:59.182 INFO:tasks.workunit.client.0.smithi076.stdout:2/917: creat d1/d4/db/d3e/d10c/f13c x:0 0 0 2024-03-23T15:44:59.182 INFO:tasks.workunit.client.0.smithi076.stdout:2/918: write d1/d4/db/d15/f55 [2717878,106388] 0 2024-03-23T15:44:59.188 INFO:tasks.workunit.client.0.smithi076.stdout:9/909: dwrite d1/d1f/f72 [8388608,4194304] 0 2024-03-23T15:44:59.193 INFO:tasks.workunit.client.0.smithi076.stdout:3/619: dread d5/d17/f29 [4194304,4194304] 0 2024-03-23T15:44:59.194 INFO:tasks.workunit.client.0.smithi076.stdout:0/867: dread d2/de/d10/d3a/d47/f4e [0,4194304] 0 2024-03-23T15:44:59.194 INFO:tasks.workunit.client.0.smithi076.stdout:6/658: rename d1/d81/l89 to d1/d2/lea 0 2024-03-23T15:44:59.194 INFO:tasks.workunit.client.0.smithi076.stdout:6/659: write d1/dd/d2a/d23/f68 [1148509,67085] 0 2024-03-23T15:44:59.194 INFO:tasks.workunit.client.0.smithi076.stdout:3/620: truncate d5/d17/d1c/f1e 591399 0 2024-03-23T15:44:59.196 INFO:tasks.workunit.client.0.smithi076.stdout:4/843: rename d8/d9/d1d/dc9 to d8/de3/db0/d10c 0 2024-03-23T15:44:59.196 INFO:tasks.workunit.client.0.smithi076.stdout:8/739: dwrite d8/f5a [0,4194304] 0 2024-03-23T15:44:59.198 INFO:tasks.workunit.client.0.smithi076.stdout:0/868: symlink d2/de/d10/df5/l103 0 2024-03-23T15:44:59.199 INFO:tasks.workunit.client.0.smithi076.stdout:5/900: rename d1a/d4b/ld4 to d1a/d20/l130 0 2024-03-23T15:44:59.200 INFO:tasks.workunit.client.0.smithi076.stdout:6/660: unlink d1/d4/d10/l4d 0 2024-03-23T15:44:59.202 INFO:tasks.workunit.client.0.smithi076.stdout:6/661: mknod d1/dd/dbc/ceb 0 2024-03-23T15:44:59.202 INFO:tasks.workunit.client.0.smithi076.stdout:6/662: truncate d1/d4/d10/fe6 276877 0 2024-03-23T15:44:59.205 INFO:tasks.workunit.client.0.smithi076.stdout:4/844: creat d8/de3/db0/d10c/f10d x:0 0 0 2024-03-23T15:44:59.205 INFO:tasks.workunit.client.0.smithi076.stdout:4/845: chown d8/f9f 121209 1 2024-03-23T15:44:59.206 INFO:tasks.workunit.client.0.smithi076.stdout:4/846: write d8/d3a/d3b/fb1 [626411,32288] 0 2024-03-23T15:44:59.206 INFO:tasks.workunit.client.0.smithi076.stdout:4/847: chown d8/d3a/d18/d1b/f96 0 1 2024-03-23T15:44:59.206 INFO:tasks.workunit.client.0.smithi076.stdout:4/848: readlink d8/d3a/la4 0 2024-03-23T15:44:59.208 INFO:tasks.workunit.client.0.smithi076.stdout:3/621: getdents d5/d35/d4c/d5a/d9d 0 2024-03-23T15:44:59.208 INFO:tasks.workunit.client.0.smithi076.stdout:2/919: dwrite d1/d4/db/f68 [0,4194304] 0 2024-03-23T15:44:59.208 INFO:tasks.workunit.client.0.smithi076.stdout:3/622: mknod d5/d17/d1c/cd2 0 2024-03-23T15:44:59.209 INFO:tasks.workunit.client.0.smithi076.stdout:2/920: rmdir d1/d4/d64/d82/d103/d79/dac/dd8/dea 39 2024-03-23T15:44:59.209 INFO:tasks.workunit.client.0.smithi076.stdout:2/921: fdatasync d1/d4/fb9 0 2024-03-23T15:44:59.211 INFO:tasks.workunit.client.0.smithi076.stdout:2/922: dread - d1/d4/db/dc8/f108 zero size 2024-03-23T15:44:59.213 INFO:tasks.workunit.client.0.smithi076.stdout:9/910: dwrite d1/d1f/fbb [0,4194304] 0 2024-03-23T15:44:59.213 INFO:tasks.workunit.client.0.smithi076.stdout:9/911: truncate d1/d1f/d46/d83/fb6 1723307 0 2024-03-23T15:44:59.213 INFO:tasks.workunit.client.0.smithi076.stdout:9/912: fdatasync d1/d2f/d9f/f10d 0 2024-03-23T15:44:59.213 INFO:tasks.workunit.client.0.smithi076.stdout:9/913: chown d1/d20/f25 1 1 2024-03-23T15:44:59.213 INFO:tasks.workunit.client.0.smithi076.stdout:9/914: write d1/d1f/f22 [3648741,129825] 0 2024-03-23T15:44:59.214 INFO:tasks.workunit.client.0.smithi076.stdout:2/923: link d1/d4/db/c65 d1/d2e/d8a/c13d 0 2024-03-23T15:44:59.216 INFO:tasks.workunit.client.0.smithi076.stdout:2/924: mknod d1/d4/db/d3e/d10c/c13e 0 2024-03-23T15:44:59.220 INFO:tasks.workunit.client.0.smithi076.stdout:8/740: dwrite d8/d2e/d30/fc3 [0,4194304] 0 2024-03-23T15:44:59.220 INFO:tasks.workunit.client.0.smithi076.stdout:8/741: truncate d8/d89/f5f 3120419 0 2024-03-23T15:44:59.220 INFO:tasks.workunit.client.0.smithi076.stdout:8/742: chown d8/d26/d33/d8c/l90 1918 1 2024-03-23T15:44:59.221 INFO:tasks.workunit.client.0.smithi076.stdout:2/925: dread d1/d4/f54 [0,4194304] 0 2024-03-23T15:44:59.225 INFO:tasks.workunit.client.0.smithi076.stdout:6/663: write d1/d2/f38 [1419358,44325] 0 2024-03-23T15:44:59.226 INFO:tasks.workunit.client.0.smithi076.stdout:0/869: dwrite d2/de/d10/d3a/d47/d69/ff4 [0,4194304] 0 2024-03-23T15:44:59.226 INFO:tasks.workunit.client.0.smithi076.stdout:2/926: dread d1/d4/db/d15/d29/d7d/f9d [0,4194304] 0 2024-03-23T15:44:59.226 INFO:tasks.workunit.client.0.smithi076.stdout:2/927: fsync d1/d4/f8d 0 2024-03-23T15:44:59.232 INFO:tasks.workunit.client.0.smithi076.stdout:4/849: dwrite d8/d9/f12 [4194304,4194304] 0 2024-03-23T15:44:59.232 INFO:tasks.workunit.client.0.smithi076.stdout:3/623: dwrite d5/d35/d3c/fb5 [0,4194304] 0 2024-03-23T15:44:59.233 INFO:tasks.workunit.client.0.smithi076.stdout:3/624: dread d5/d35/f6e [0,4194304] 0 2024-03-23T15:44:59.238 INFO:tasks.workunit.client.0.smithi076.stdout:8/743: mkdir d8/d26/d33/d101 0 2024-03-23T15:44:59.238 INFO:tasks.workunit.client.0.smithi076.stdout:8/744: dread - d8/d2e/d30/f70 zero size 2024-03-23T15:44:59.239 INFO:tasks.workunit.client.0.smithi076.stdout:3/625: creat d5/d35/d6d/fd3 x:0 0 0 2024-03-23T15:44:59.240 INFO:tasks.workunit.client.0.smithi076.stdout:3/626: read - d5/d35/d6d/f9f zero size 2024-03-23T15:44:59.241 INFO:tasks.workunit.client.0.smithi076.stdout:0/870: dread d2/d1b/fc4 [0,4194304] 0 2024-03-23T15:44:59.246 INFO:tasks.workunit.client.0.smithi076.stdout:9/915: dread d1/d2f/d3a/d53/f121 [0,4194304] 0 2024-03-23T15:44:59.246 INFO:tasks.workunit.client.0.smithi076.stdout:4/850: dwrite d8/d9/f5a [0,4194304] 0 2024-03-23T15:44:59.247 INFO:tasks.workunit.client.0.smithi076.stdout:8/745: write d8/d26/d27/d56/f5b [5231650,32108] 0 2024-03-23T15:44:59.248 INFO:tasks.workunit.client.0.smithi076.stdout:3/627: symlink d5/d17/d1c/d32/d39/ld4 0 2024-03-23T15:44:59.248 INFO:tasks.workunit.client.0.smithi076.stdout:3/628: chown d5/d17/d1c/d31/d47 512228419 1 2024-03-23T15:44:59.248 INFO:tasks.workunit.client.0.smithi076.stdout:3/629: stat d5/d17/d1c/d32/d39/f65 0 2024-03-23T15:44:59.249 INFO:tasks.workunit.client.0.smithi076.stdout:4/851: dread d8/d9/f2d [0,4194304] 0 2024-03-23T15:44:59.249 INFO:tasks.workunit.client.0.smithi076.stdout:9/916: creat d1/d2f/d3a/d74/d102/d120/f126 x:0 0 0 2024-03-23T15:44:59.251 INFO:tasks.workunit.client.0.smithi076.stdout:1/631: rename d0/f34 to d0/d35/d40/d47/d17/d20/fd4 0 2024-03-23T15:44:59.252 INFO:tasks.workunit.client.0.smithi076.stdout:4/852: creat d8/de3/f10e x:0 0 0 2024-03-23T15:44:59.252 INFO:tasks.workunit.client.0.smithi076.stdout:1/632: mknod d0/d35/d40/d47/d17/d20/cd5 0 2024-03-23T15:44:59.252 INFO:tasks.workunit.client.0.smithi076.stdout:1/633: write d0/d35/d40/d47/d17/faf [360853,105085] 0 2024-03-23T15:44:59.253 INFO:tasks.workunit.client.0.smithi076.stdout:1/634: mkdir d0/d35/d40/d47/d8c/dd6 0 2024-03-23T15:44:59.253 INFO:tasks.workunit.client.0.smithi076.stdout:1/635: fsync d0/d35/d40/d47/d74/d83/f9c 0 2024-03-23T15:44:59.255 INFO:tasks.workunit.client.0.smithi076.stdout:4/853: link d8/d3a/d18/d75/ldd d8/d3a/d69/d73/d76/l10f 0 2024-03-23T15:44:59.258 INFO:tasks.workunit.client.0.smithi076.stdout:0/871: rename d2/de/d10/df5/l103 to d2/de/d10/d12/l104 0 2024-03-23T15:44:59.258 INFO:tasks.workunit.client.0.smithi076.stdout:0/872: chown d2/d1b/d77/c7b 363635075 1 2024-03-23T15:44:59.258 INFO:tasks.workunit.client.0.smithi076.stdout:0/873: dread d2/de/d10/d3a/feb [0,4194304] 0 2024-03-23T15:44:59.259 INFO:tasks.workunit.client.0.smithi076.stdout:4/854: creat d8/d9/d32/f110 x:0 0 0 2024-03-23T15:44:59.260 INFO:tasks.workunit.client.0.smithi076.stdout:3/630: symlink d5/d17/d1c/d32/d39/ld5 0 2024-03-23T15:44:59.261 INFO:tasks.workunit.client.0.smithi076.stdout:4/855: creat d8/d3a/d69/d73/d76/f111 x:0 0 0 2024-03-23T15:44:59.261 INFO:tasks.workunit.client.0.smithi076.stdout:4/856: truncate d8/d9/f2d 1462488 0 2024-03-23T15:44:59.265 INFO:tasks.workunit.client.0.smithi076.stdout:9/917: rename d1/d1f/d29/l79 to d1/d1f/d29/d2d/d37/d47/d11c/l127 0 2024-03-23T15:44:59.265 INFO:tasks.workunit.client.0.smithi076.stdout:9/918: readlink d1/d1f/d29/d7f/lc4 0 2024-03-23T15:44:59.265 INFO:tasks.workunit.client.0.smithi076.stdout:0/874: write d2/de/d10/d3a/f5c [4164539,97623] 0 2024-03-23T15:44:59.265 INFO:tasks.workunit.client.0.smithi076.stdout:0/875: fdatasync d2/de/d10/f8c 0 2024-03-23T15:44:59.266 INFO:tasks.workunit.client.0.smithi076.stdout:4/857: rename d8/d3a/d18/d75/caf to d8/d41/c112 0 2024-03-23T15:44:59.268 INFO:tasks.workunit.client.0.smithi076.stdout:4/858: rename d8/f66 to d8/de3/db0/d10c/f113 0 2024-03-23T15:44:59.268 INFO:tasks.workunit.client.0.smithi076.stdout:4/859: chown d8/d80/fa6 0 1 2024-03-23T15:44:59.270 INFO:tasks.workunit.client.0.smithi076.stdout:4/860: creat d8/d3a/d69/d73/d9a/f114 x:0 0 0 2024-03-23T15:44:59.270 INFO:tasks.workunit.client.0.smithi076.stdout:4/861: write d8/d9/d1d/ff0 [590300,1958] 0 2024-03-23T15:44:59.272 INFO:tasks.workunit.client.0.smithi076.stdout:4/862: creat d8/de3/db0/de9/f115 x:0 0 0 2024-03-23T15:44:59.272 INFO:tasks.workunit.client.0.smithi076.stdout:4/863: read d8/f3f [2765943,14114] 0 2024-03-23T15:44:59.274 INFO:tasks.workunit.client.0.smithi076.stdout:1/636: read d0/f2e [1554948,44804] 0 2024-03-23T15:44:59.274 INFO:tasks.workunit.client.0.smithi076.stdout:1/637: fsync d0/d2/d70/da1/dbf/fb2 0 2024-03-23T15:44:59.274 INFO:tasks.workunit.client.0.smithi076.stdout:4/864: unlink d8/c6b 0 2024-03-23T15:44:59.274 INFO:tasks.workunit.client.0.smithi076.stdout:4/865: write d8/d3a/d18/d1b/f96 [204722,106439] 0 2024-03-23T15:44:59.274 INFO:tasks.workunit.client.0.smithi076.stdout:4/866: read d8/d3a/d18/d52/f79 [1437350,9695] 0 2024-03-23T15:44:59.275 INFO:tasks.workunit.client.0.smithi076.stdout:1/638: truncate d0/d35/d40/fa8 2075516 0 2024-03-23T15:44:59.276 INFO:tasks.workunit.client.0.smithi076.stdout:4/867: getdents d8/de3 0 2024-03-23T15:44:59.278 INFO:tasks.workunit.client.0.smithi076.stdout:4/868: unlink d8/l68 0 2024-03-23T15:44:59.278 INFO:tasks.workunit.client.0.smithi076.stdout:4/869: write d8/d41/fa0 [1021238,60011] 0 2024-03-23T15:44:59.280 INFO:tasks.workunit.client.0.smithi076.stdout:4/870: mkdir d8/d3a/d116 0 2024-03-23T15:44:59.283 INFO:tasks.workunit.client.0.smithi076.stdout:1/639: dread d0/d35/d40/d47/d17/f2c [0,4194304] 0 2024-03-23T15:44:59.284 INFO:tasks.workunit.client.0.smithi076.stdout:4/871: mknod d8/de3/db0/de9/c117 0 2024-03-23T15:44:59.284 INFO:tasks.workunit.client.0.smithi076.stdout:4/872: fsync d8/d3a/d69/fde 0 2024-03-23T15:44:59.284 INFO:tasks.workunit.client.0.smithi076.stdout:1/640: mknod d0/d35/d40/d47/d74/cd7 0 2024-03-23T15:44:59.286 INFO:tasks.workunit.client.0.smithi076.stdout:0/876: dwrite d2/de/d10/d12/da6/fd2 [0,4194304] 0 2024-03-23T15:44:59.286 INFO:tasks.workunit.client.0.smithi076.stdout:9/919: dwrite d1/d2f/d3f/da0/f106 [0,4194304] 0 2024-03-23T15:44:59.291 INFO:tasks.workunit.client.0.smithi076.stdout:9/920: dread d1/d2f/d9f/fec [8388608,4194304] 0 2024-03-23T15:44:59.295 INFO:tasks.workunit.client.0.smithi076.stdout:1/641: creat d0/d2/d91/fd8 x:0 0 0 2024-03-23T15:44:59.299 INFO:tasks.workunit.client.0.smithi076.stdout:0/877: rmdir d2/de/d10/d3a/d47/de3 39 2024-03-23T15:44:59.303 INFO:tasks.workunit.client.0.smithi076.stdout:0/878: mknod d2/de/d10/d43/d56/c105 0 2024-03-23T15:44:59.304 INFO:tasks.workunit.client.0.smithi076.stdout:9/921: creat d1/d2f/d3a/d53/f128 x:0 0 0 2024-03-23T15:44:59.304 INFO:tasks.workunit.client.0.smithi076.stdout:1/642: dread d0/d35/d40/d47/d17/d20/f3e [0,4194304] 0 2024-03-23T15:44:59.304 INFO:tasks.workunit.client.0.smithi076.stdout:1/643: readlink d0/d2/d91/l59 0 2024-03-23T15:44:59.310 INFO:tasks.workunit.client.0.smithi076.stdout:4/873: rename d8/ff to d8/d3a/d69/d73/df4/da9/f118 0 2024-03-23T15:44:59.311 INFO:tasks.workunit.client.0.smithi076.stdout:4/874: creat d8/d3a/d69/d73/d76/f119 x:0 0 0 2024-03-23T15:44:59.317 INFO:tasks.workunit.client.0.smithi076.stdout:4/875: write d8/d3a/d69/d73/d76/f77 [696232,106455] 0 2024-03-23T15:44:59.318 INFO:tasks.workunit.client.0.smithi076.stdout:0/879: dwrite d2/de/d10/d3a/d47/d69/fbe [0,4194304] 0 2024-03-23T15:44:59.319 INFO:tasks.workunit.client.0.smithi076.stdout:1/644: dwrite d0/d3c/f6c [0,4194304] 0 2024-03-23T15:44:59.320 INFO:tasks.workunit.client.0.smithi076.stdout:1/645: fsync d0/d35/f3a 0 2024-03-23T15:44:59.331 INFO:tasks.workunit.client.0.smithi076.stdout:0/880: write d2/d8/d2d/f3f [1623361,90837] 0 2024-03-23T15:44:59.332 INFO:tasks.workunit.client.0.smithi076.stdout:1/646: dwrite d0/d35/f87 [0,4194304] 0 2024-03-23T15:44:59.332 INFO:tasks.workunit.client.0.smithi076.stdout:1/647: truncate d0/f4b 426368 0 2024-03-23T15:44:59.332 INFO:tasks.workunit.client.0.smithi076.stdout:1/648: chown d0/d35/f64 0 1 2024-03-23T15:44:59.332 INFO:tasks.workunit.client.0.smithi076.stdout:4/876: dwrite d8/d3a/d3b/f4b [4194304,4194304] 0 2024-03-23T15:44:59.332 INFO:tasks.workunit.client.0.smithi076.stdout:4/877: dread d8/d9/fd3 [0,4194304] 0 2024-03-23T15:44:59.334 INFO:tasks.workunit.client.0.smithi076.stdout:0/881: dread d2/de/d10/d43/f88 [0,4194304] 0 2024-03-23T15:44:59.340 INFO:tasks.workunit.client.0.smithi076.stdout:1/649: dread d0/d2/f48 [0,4194304] 0 2024-03-23T15:44:59.344 INFO:tasks.workunit.client.0.smithi076.stdout:0/882: dread d2/d1b/f42 [4194304,4194304] 0 2024-03-23T15:44:59.346 INFO:tasks.workunit.client.0.smithi076.stdout:0/883: dread d2/fa7 [0,4194304] 0 2024-03-23T15:44:59.346 INFO:tasks.workunit.client.0.smithi076.stdout:1/650: dread d0/d35/d40/d47/d17/f30 [0,4194304] 0 2024-03-23T15:44:59.347 INFO:tasks.workunit.client.0.smithi076.stdout:0/884: mknod d2/d8/c106 0 2024-03-23T15:44:59.347 INFO:tasks.workunit.client.0.smithi076.stdout:0/885: chown d2/d8/d52/lda 9 1 2024-03-23T15:44:59.348 INFO:tasks.workunit.client.0.smithi076.stdout:1/651: mknod d0/d2/d70/d7a/cd9 0 2024-03-23T15:44:59.348 INFO:tasks.workunit.client.0.smithi076.stdout:1/652: write d0/d2/d70/d7a/d8b/fbe [623174,92204] 0 2024-03-23T15:44:59.349 INFO:tasks.workunit.client.0.smithi076.stdout:1/653: creat d0/d35/d40/d47/d17/fda x:0 0 0 2024-03-23T15:44:59.349 INFO:tasks.workunit.client.0.smithi076.stdout:1/654: stat d0/d35/d40/d47/d55 0 2024-03-23T15:44:59.350 INFO:tasks.workunit.client.0.smithi076.stdout:1/655: creat d0/d2/fdb x:0 0 0 2024-03-23T15:44:59.350 INFO:tasks.workunit.client.0.smithi076.stdout:1/656: stat d0/d2/d70/da1/dbf 0 2024-03-23T15:44:59.351 INFO:tasks.workunit.client.0.smithi076.stdout:1/657: symlink d0/d2/d62/ldc 0 2024-03-23T15:44:59.352 INFO:tasks.workunit.client.0.smithi076.stdout:1/658: getdents d0/d2/d91 0 2024-03-23T15:44:59.352 INFO:tasks.workunit.client.0.smithi076.stdout:1/659: rmdir d0/d2/d70/d7a/d8b 39 2024-03-23T15:44:59.358 INFO:tasks.workunit.client.0.smithi076.stdout:1/660: write d0/f2e [11142925,38559] 0 2024-03-23T15:44:59.358 INFO:tasks.workunit.client.0.smithi076.stdout:1/661: readlink d0/l27 0 2024-03-23T15:44:59.359 INFO:tasks.workunit.client.0.smithi076.stdout:1/662: mkdir d0/d2/d62/ddd 0 2024-03-23T15:44:59.361 INFO:tasks.workunit.client.0.smithi076.stdout:0/886: dwrite d2/de/d10/d3a/d47/d69/faa [0,4194304] 0 2024-03-23T15:44:59.364 INFO:tasks.workunit.client.0.smithi076.stdout:0/887: creat d2/d8/d52/f107 x:0 0 0 2024-03-23T15:44:59.365 INFO:tasks.workunit.client.0.smithi076.stdout:0/888: mknod d2/ded/df8/c108 0 2024-03-23T15:44:59.367 INFO:tasks.workunit.client.0.smithi076.stdout:0/889: rename d2/de/d10/fd5 to d2/d9a/f109 0 2024-03-23T15:44:59.367 INFO:tasks.workunit.client.0.smithi076.stdout:0/890: write d2/d9a/db8/fe7 [830285,32022] 0 2024-03-23T15:44:59.368 INFO:tasks.workunit.client.0.smithi076.stdout:0/891: truncate d2/de/d10/d12/da6/fd2 4033553 0 2024-03-23T15:44:59.368 INFO:tasks.workunit.client.0.smithi076.stdout:0/892: fsync d2/de/d10/ffe 0 2024-03-23T15:44:59.370 INFO:tasks.workunit.client.0.smithi076.stdout:0/893: mkdir d2/de/d10/d43/df3/df7/d10a 0 2024-03-23T15:44:59.372 INFO:tasks.workunit.client.0.smithi076.stdout:1/663: dwrite d0/d35/d40/f90 [0,4194304] 0 2024-03-23T15:44:59.374 INFO:tasks.workunit.client.0.smithi076.stdout:1/664: dread d0/f4b [0,4194304] 0 2024-03-23T15:44:59.374 INFO:tasks.workunit.client.0.smithi076.stdout:1/665: fsync d0/d35/f3a 0 2024-03-23T15:44:59.376 INFO:tasks.workunit.client.0.smithi076.stdout:1/666: rename d0/d35/d40/d47/d8c/d99/fa4 to d0/d35/d40/d47/d74/fde 0 2024-03-23T15:44:59.376 INFO:tasks.workunit.client.0.smithi076.stdout:1/667: fsync d0/d35/d40/d47/d74/d83/f9c 0 2024-03-23T15:44:59.376 INFO:tasks.workunit.client.0.smithi076.stdout:1/668: rename d0/d2/d62 to d0/d2/d62/ddd/ddf 22 2024-03-23T15:44:59.388 INFO:tasks.workunit.client.0.smithi076.stdout:0/894: dwrite d2/de/d10/d3a/feb [0,4194304] 0 2024-03-23T15:44:59.388 INFO:tasks.workunit.client.0.smithi076.stdout:0/895: chown d2/de/d10/d43/f81 1242222542 1 2024-03-23T15:44:59.390 INFO:tasks.workunit.client.0.smithi076.stdout:0/896: mkdir d2/de/d10/d3a/d10b 0 2024-03-23T15:44:59.390 INFO:tasks.workunit.client.0.smithi076.stdout:0/897: fdatasync d2/fb7 0 2024-03-23T15:44:59.394 INFO:tasks.workunit.client.0.smithi076.stdout:1/669: dwrite d0/d2/d91/fd8 [0,4194304] 0 2024-03-23T15:44:59.394 INFO:tasks.workunit.client.0.smithi076.stdout:1/670: chown d0/d2/d70/d7a/d8b 845615 1 2024-03-23T15:44:59.394 INFO:tasks.workunit.client.0.smithi076.stdout:1/671: dread - d0/d2/d91/d86/fcc zero size 2024-03-23T15:44:59.395 INFO:tasks.workunit.client.0.smithi076.stdout:1/672: write d0/d35/f3a [1401799,40708] 0 2024-03-23T15:44:59.395 INFO:tasks.workunit.client.0.smithi076.stdout:1/673: chown d0/d35/d40/d47/d17/d20/fd4 1 1 2024-03-23T15:44:59.395 INFO:tasks.workunit.client.0.smithi076.stdout:1/674: write d0/d2/d91/d86/fc9 [523040,107220] 0 2024-03-23T15:44:59.398 INFO:tasks.workunit.client.0.smithi076.stdout:1/675: mknod d0/d2/ce0 0 2024-03-23T15:44:59.400 INFO:tasks.workunit.client.0.smithi076.stdout:0/898: read d2/fa9 [5387580,21774] 0 2024-03-23T15:44:59.400 INFO:tasks.workunit.client.0.smithi076.stdout:1/676: mkdir d0/d35/d40/d47/d8c/dd6/de1 0 2024-03-23T15:44:59.402 INFO:tasks.workunit.client.0.smithi076.stdout:0/899: truncate d2/de/f13 147231 0 2024-03-23T15:44:59.402 INFO:tasks.workunit.client.0.smithi076.stdout:5/901: sync 2024-03-23T15:44:59.403 INFO:tasks.workunit.client.0.smithi076.stdout:2/928: sync 2024-03-23T15:44:59.403 INFO:tasks.workunit.client.0.smithi076.stdout:8/746: sync 2024-03-23T15:44:59.403 INFO:tasks.workunit.client.0.smithi076.stdout:6/664: sync 2024-03-23T15:44:59.403 INFO:tasks.workunit.client.0.smithi076.stdout:5/902: chown d1a/d20/le5 0 1 2024-03-23T15:44:59.403 INFO:tasks.workunit.client.0.smithi076.stdout:8/747: chown d8/d26/l34 15 1 2024-03-23T15:44:59.403 INFO:tasks.workunit.client.0.smithi076.stdout:2/929: fdatasync d1/d4/d64/d82/d103/d79/dac/f112 0 2024-03-23T15:44:59.403 INFO:tasks.workunit.client.0.smithi076.stdout:6/665: write d1/d4/fc6 [1047157,23165] 0 2024-03-23T15:44:59.404 INFO:tasks.workunit.client.0.smithi076.stdout:8/748: symlink d8/dd0/df0/l102 0 2024-03-23T15:44:59.404 INFO:tasks.workunit.client.0.smithi076.stdout:8/749: truncate d8/f5a 4590698 0 2024-03-23T15:44:59.405 INFO:tasks.workunit.client.0.smithi076.stdout:2/930: symlink d1/d2e/dd1/ddc/d139/l13f 0 2024-03-23T15:44:59.406 INFO:tasks.workunit.client.0.smithi076.stdout:6/666: mkdir d1/dd/df/dac/dec 0 2024-03-23T15:44:59.406 INFO:tasks.workunit.client.0.smithi076.stdout:6/667: chown d1/dd/cba 73547 1 2024-03-23T15:44:59.407 INFO:tasks.workunit.client.0.smithi076.stdout:8/750: mknod d8/d26/d27/d56/c103 0 2024-03-23T15:44:59.407 INFO:tasks.workunit.client.0.smithi076.stdout:8/751: dread - d8/d4f/d80/db7/fef zero size 2024-03-23T15:44:59.410 INFO:tasks.workunit.client.0.smithi076.stdout:4/878: sync 2024-03-23T15:44:59.410 INFO:tasks.workunit.client.0.smithi076.stdout:3/631: sync 2024-03-23T15:44:59.410 INFO:tasks.workunit.client.0.smithi076.stdout:9/922: sync 2024-03-23T15:44:59.410 INFO:tasks.workunit.client.0.smithi076.stdout:3/632: chown d5/d17/d1c/d32/d70 0 1 2024-03-23T15:44:59.411 INFO:tasks.workunit.client.0.smithi076.stdout:9/923: truncate d1/d2f/d3a/d74/fd4 157820 0 2024-03-23T15:44:59.411 INFO:tasks.workunit.client.0.smithi076.stdout:9/924: chown d1/d1f/d82/ld2 52936 1 2024-03-23T15:44:59.412 INFO:tasks.workunit.client.0.smithi076.stdout:3/633: creat d5/d17/d1c/d32/d39/d59/fd6 x:0 0 0 2024-03-23T15:44:59.412 INFO:tasks.workunit.client.0.smithi076.stdout:3/634: write d5/d17/d1c/d32/d4a/fa9 [17477,58985] 0 2024-03-23T15:44:59.412 INFO:tasks.workunit.client.0.smithi076.stdout:4/879: symlink d8/dd7/l11a 0 2024-03-23T15:44:59.413 INFO:tasks.workunit.client.0.smithi076.stdout:4/880: dread - d8/de3/db0/d10c/f10d zero size 2024-03-23T15:44:59.413 INFO:tasks.workunit.client.0.smithi076.stdout:4/881: dread - d8/d3a/d69/d73/d9a/f114 zero size 2024-03-23T15:44:59.413 INFO:tasks.workunit.client.0.smithi076.stdout:6/668: unlink d1/dd/df/d9b/db3/c82 0 2024-03-23T15:44:59.413 INFO:tasks.workunit.client.0.smithi076.stdout:3/635: mkdir d5/d17/d1c/d31/d6b/dd7 0 2024-03-23T15:44:59.413 INFO:tasks.workunit.client.0.smithi076.stdout:3/636: readlink d5/d35/d4c/d5a/l40 0 2024-03-23T15:44:59.416 INFO:tasks.workunit.client.0.smithi076.stdout:9/925: read d1/d2f/d3a/d11f/fe5 [3445926,78896] 0 2024-03-23T15:44:59.416 INFO:tasks.workunit.client.0.smithi076.stdout:9/926: write d1/d1f/d29/d2d/d37/d47/f52 [3763030,74235] 0 2024-03-23T15:44:59.416 INFO:tasks.workunit.client.0.smithi076.stdout:9/927: read - d1/d2f/d3a/d74/d102/f117 zero size 2024-03-23T15:44:59.416 INFO:tasks.workunit.client.0.smithi076.stdout:9/928: chown d1/c5 6 1 2024-03-23T15:44:59.416 INFO:tasks.workunit.client.0.smithi076.stdout:3/637: symlink d5/d17/d1c/d32/d70/ld8 0 2024-03-23T15:44:59.417 INFO:tasks.workunit.client.0.smithi076.stdout:6/669: mknod d1/dd/df/d9b/db3/d8d/d60/d7c/ced 0 2024-03-23T15:44:59.418 INFO:tasks.workunit.client.0.smithi076.stdout:8/752: dread d8/f1d [0,4194304] 0 2024-03-23T15:44:59.419 INFO:tasks.workunit.client.0.smithi076.stdout:3/638: creat d5/d17/d1c/d32/d39/d59/d79/fd9 x:0 0 0 2024-03-23T15:44:59.419 INFO:tasks.workunit.client.0.smithi076.stdout:4/882: mknod d8/d80/c11b 0 2024-03-23T15:44:59.421 INFO:tasks.workunit.client.0.smithi076.stdout:0/900: dwrite d2/de/d10/d43/f81 [0,4194304] 0 2024-03-23T15:44:59.422 INFO:tasks.workunit.client.0.smithi076.stdout:5/903: dwrite d1a/d33/f35 [0,4194304] 0 2024-03-23T15:44:59.422 INFO:tasks.workunit.client.0.smithi076.stdout:9/929: symlink d1/d2f/d3a/d53/l129 0 2024-03-23T15:44:59.423 INFO:tasks.workunit.client.0.smithi076.stdout:5/904: link d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/c109 d1a/d33/d3a/d67/da8/dc4/d5f/c131 0 2024-03-23T15:44:59.424 INFO:tasks.workunit.client.0.smithi076.stdout:9/930: mknod d1/d1f/d29/d2d/d37/d47/d11c/c12a 0 2024-03-23T15:44:59.425 INFO:tasks.workunit.client.0.smithi076.stdout:9/931: chown d1/l9b 3871347 1 2024-03-23T15:44:59.426 INFO:tasks.workunit.client.0.smithi076.stdout:5/905: rmdir d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf 39 2024-03-23T15:44:59.426 INFO:tasks.workunit.client.0.smithi076.stdout:4/883: write d8/d3a/d69/f86 [3111518,119103] 0 2024-03-23T15:44:59.427 INFO:tasks.workunit.client.0.smithi076.stdout:8/753: getdents d8/d4f/d80/db7 0 2024-03-23T15:44:59.427 INFO:tasks.workunit.client.0.smithi076.stdout:4/884: write d8/d41/fa0 [1536281,105049] 0 2024-03-23T15:44:59.427 INFO:tasks.workunit.client.0.smithi076.stdout:4/885: write d8/d3a/d18/d75/f104 [915493,11252] 0 2024-03-23T15:44:59.429 INFO:tasks.workunit.client.0.smithi076.stdout:3/639: creat d5/d35/d6d/fda x:0 0 0 2024-03-23T15:44:59.429 INFO:tasks.workunit.client.0.smithi076.stdout:3/640: write d5/d17/d1c/d32/f5f [2218220,90613] 0 2024-03-23T15:44:59.430 INFO:tasks.workunit.client.0.smithi076.stdout:0/901: chown d2/de/d10/d12/l26 1 1 2024-03-23T15:44:59.432 INFO:tasks.workunit.client.0.smithi076.stdout:8/754: dread d8/d26/d33/f86 [0,4194304] 0 2024-03-23T15:44:59.432 INFO:tasks.workunit.client.0.smithi076.stdout:8/755: dread - d8/d2e/d30/d69/fdb zero size 2024-03-23T15:44:59.432 INFO:tasks.workunit.client.0.smithi076.stdout:8/756: write d8/fe6 [275965,114194] 0 2024-03-23T15:44:59.432 INFO:tasks.workunit.client.0.smithi076.stdout:8/757: dread - d8/d2e/d30/d69/f71 zero size 2024-03-23T15:44:59.432 INFO:tasks.workunit.client.0.smithi076.stdout:8/758: chown d8/d2e/d3e/lb0 900344546 1 2024-03-23T15:44:59.433 INFO:tasks.workunit.client.0.smithi076.stdout:8/759: truncate d8/d2e/d30/d69/f8f 8055 0 2024-03-23T15:44:59.433 INFO:tasks.workunit.client.0.smithi076.stdout:8/760: write d8/d26/d33/d8c/fee [321171,115306] 0 2024-03-23T15:44:59.433 INFO:tasks.workunit.client.0.smithi076.stdout:3/641: dread d5/f6 [0,4194304] 0 2024-03-23T15:44:59.443 INFO:tasks.workunit.client.0.smithi076.stdout:4/886: dwrite d8/d3a/d69/d73/df4/d88/fc7 [0,4194304] 0 2024-03-23T15:44:59.449 INFO:tasks.workunit.client.0.smithi076.stdout:6/670: link d1/dd/l58 d1/lee 0 2024-03-23T15:44:59.452 INFO:tasks.workunit.client.0.smithi076.stdout:1/677: rename d0/d35/d40/d47/d55/c71 to d0/d2/d70/d7a/d8b/ce2 0 2024-03-23T15:44:59.453 INFO:tasks.workunit.client.0.smithi076.stdout:9/932: dwrite d1/d1f/d29/d2d/d37/fb0 [4194304,4194304] 0 2024-03-23T15:44:59.453 INFO:tasks.workunit.client.0.smithi076.stdout:9/933: fsync d1/d2f/d3a/d74/d102/f125 0 2024-03-23T15:44:59.453 INFO:tasks.workunit.client.0.smithi076.stdout:6/671: symlink d1/dd/df/d9b/db3/d72/lef 0 2024-03-23T15:44:59.453 INFO:tasks.workunit.client.0.smithi076.stdout:6/672: dread - d1/dd/df/d5b/f55 zero size 2024-03-23T15:44:59.454 INFO:tasks.workunit.client.0.smithi076.stdout:1/678: write d0/d35/f3a [1030693,38038] 0 2024-03-23T15:44:59.455 INFO:tasks.workunit.client.0.smithi076.stdout:9/934: creat d1/d1f/d46/f12b x:0 0 0 2024-03-23T15:44:59.458 INFO:tasks.workunit.client.0.smithi076.stdout:9/935: creat d1/d1f/d29/d7f/f12c x:0 0 0 2024-03-23T15:44:59.459 INFO:tasks.workunit.client.0.smithi076.stdout:9/936: rmdir d1/dfd 39 2024-03-23T15:44:59.459 INFO:tasks.workunit.client.0.smithi076.stdout:3/642: dwrite d5/fb1 [0,4194304] 0 2024-03-23T15:44:59.461 INFO:tasks.workunit.client.0.smithi076.stdout:4/887: dwrite d8/d3a/d3b/f39 [4194304,4194304] 0 2024-03-23T15:44:59.461 INFO:tasks.workunit.client.0.smithi076.stdout:9/937: symlink d1/d1f/l12d 0 2024-03-23T15:44:59.461 INFO:tasks.workunit.client.0.smithi076.stdout:8/761: dwrite d8/fe6 [0,4194304] 0 2024-03-23T15:44:59.461 INFO:tasks.workunit.client.0.smithi076.stdout:8/762: chown d8/f16 88 1 2024-03-23T15:44:59.462 INFO:tasks.workunit.client.0.smithi076.stdout:0/902: dwrite d2/d1b/d77/fd6 [0,4194304] 0 2024-03-23T15:44:59.463 INFO:tasks.workunit.client.0.smithi076.stdout:9/938: creat d1/d1f/d46/d83/dad/f12e x:0 0 0 2024-03-23T15:44:59.463 INFO:tasks.workunit.client.0.smithi076.stdout:9/939: chown d1/d1f/d46/d83/c104 2478147 1 2024-03-23T15:44:59.464 INFO:tasks.workunit.client.0.smithi076.stdout:9/940: write d1/d2f/d3f/da0/dd5/f122 [253566,1862] 0 2024-03-23T15:44:59.466 INFO:tasks.workunit.client.0.smithi076.stdout:1/679: dread d0/d3c/f6c [0,4194304] 0 2024-03-23T15:44:59.467 INFO:tasks.workunit.client.0.smithi076.stdout:0/903: link d2/d8/d2d/cac d2/de/d10/d3a/d47/d69/d84/c10c 0 2024-03-23T15:44:59.469 INFO:tasks.workunit.client.0.smithi076.stdout:4/888: read d8/d3a/d18/d52/f64 [2072906,87221] 0 2024-03-23T15:44:59.469 INFO:tasks.workunit.client.0.smithi076.stdout:4/889: fsync d8/d9/f9c 0 2024-03-23T15:44:59.470 INFO:tasks.workunit.client.0.smithi076.stdout:9/941: symlink d1/d2f/d9f/l12f 0 2024-03-23T15:44:59.473 INFO:tasks.workunit.client.0.smithi076.stdout:0/904: rmdir d2/de/d10/def 0 2024-03-23T15:44:59.475 INFO:tasks.workunit.client.0.smithi076.stdout:0/905: dread d2/d9a/fc8 [0,4194304] 0 2024-03-23T15:44:59.475 INFO:tasks.workunit.client.0.smithi076.stdout:0/906: chown d2/d8 1116699646 1 2024-03-23T15:44:59.476 INFO:tasks.workunit.client.0.smithi076.stdout:9/942: mknod d1/d2f/d3f/da0/d119/c130 0 2024-03-23T15:44:59.478 INFO:tasks.workunit.client.0.smithi076.stdout:5/906: rename d1a/f42 to d1a/d33/d3a/d67/d84/f132 0 2024-03-23T15:44:59.480 INFO:tasks.workunit.client.0.smithi076.stdout:9/943: symlink d1/d2f/d3a/l131 0 2024-03-23T15:44:59.481 INFO:tasks.workunit.client.0.smithi076.stdout:9/944: symlink d1/d1f/d46/d4a/l132 0 2024-03-23T15:44:59.481 INFO:tasks.workunit.client.0.smithi076.stdout:9/945: write d1/f18 [2337596,40202] 0 2024-03-23T15:44:59.485 INFO:tasks.workunit.client.0.smithi076.stdout:6/673: rename d1/dd/dbc to d1/dd/df/dce/df0 0 2024-03-23T15:44:59.486 INFO:tasks.workunit.client.0.smithi076.stdout:4/890: dwrite d8/d41/f54 [8388608,4194304] 0 2024-03-23T15:44:59.487 INFO:tasks.workunit.client.0.smithi076.stdout:5/907: write d1a/d33/d3a/f74 [1157775,19019] 0 2024-03-23T15:44:59.489 INFO:tasks.workunit.client.0.smithi076.stdout:3/643: rename d5/d17/l22 to d5/d35/d4c/d5a/ldb 0 2024-03-23T15:44:59.489 INFO:tasks.workunit.client.0.smithi076.stdout:3/644: chown d5/d17/cb7 705669199 1 2024-03-23T15:44:59.490 INFO:tasks.workunit.client.0.smithi076.stdout:6/674: truncate d1/f11 2372366 0 2024-03-23T15:44:59.491 INFO:tasks.workunit.client.0.smithi076.stdout:5/908: link d1a/d33/dc3/da6/fbc d1a/d33/dc3/da6/f133 0 2024-03-23T15:44:59.491 INFO:tasks.workunit.client.0.smithi076.stdout:4/891: mkdir d8/d3a/d69/d73/d76/d11c 0 2024-03-23T15:44:59.491 INFO:tasks.workunit.client.0.smithi076.stdout:4/892: chown d8/d3a/d18/d1b/f67 174437 1 2024-03-23T15:44:59.494 INFO:tasks.workunit.client.0.smithi076.stdout:3/645: truncate d5/d35/d3c/d67/fbc 3893027 0 2024-03-23T15:44:59.494 INFO:tasks.workunit.client.0.smithi076.stdout:3/646: stat d5/d17/d1c/d32/d39/d59/d79/fd9 0 2024-03-23T15:44:59.494 INFO:tasks.workunit.client.0.smithi076.stdout:3/647: chown d5/fd 0 1 2024-03-23T15:44:59.494 INFO:tasks.workunit.client.0.smithi076.stdout:5/909: symlink d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/l134 0 2024-03-23T15:44:59.495 INFO:tasks.workunit.client.0.smithi076.stdout:3/648: mkdir d5/d17/d1c/d32/d39/d59/d79/ddc 0 2024-03-23T15:44:59.495 INFO:tasks.workunit.client.0.smithi076.stdout:3/649: chown d5/d17/l74 3957556 1 2024-03-23T15:44:59.496 INFO:tasks.workunit.client.0.smithi076.stdout:4/893: chown d8/d80/fbc 55877 1 2024-03-23T15:44:59.496 INFO:tasks.workunit.client.0.smithi076.stdout:4/894: chown d8/d3a/d69/d73/d76/db4/fc5 114 1 2024-03-23T15:44:59.496 INFO:tasks.workunit.client.0.smithi076.stdout:4/895: dread - d8/d3a/d69/d73/d9a/fef zero size 2024-03-23T15:44:59.496 INFO:tasks.workunit.client.0.smithi076.stdout:4/896: dread - d8/d3a/dab/fb3 zero size 2024-03-23T15:44:59.498 INFO:tasks.workunit.client.0.smithi076.stdout:4/897: creat d8/d3a/d116/f11d x:0 0 0 2024-03-23T15:44:59.498 INFO:tasks.workunit.client.0.smithi076.stdout:4/898: chown d8/d9/l70 17 1 2024-03-23T15:44:59.499 INFO:tasks.workunit.client.0.smithi076.stdout:8/763: unlink d8/d26/d33/d8c/l90 0 2024-03-23T15:44:59.500 INFO:tasks.workunit.client.0.smithi076.stdout:5/910: dread d1a/d33/d3a/f8a [0,4194304] 0 2024-03-23T15:44:59.502 INFO:tasks.workunit.client.0.smithi076.stdout:4/899: link d8/d9/l2a d8/d3a/d69/d73/df4/da9/l11e 0 2024-03-23T15:44:59.504 INFO:tasks.workunit.client.0.smithi076.stdout:3/650: dread d5/f9 [0,4194304] 0 2024-03-23T15:44:59.504 INFO:tasks.workunit.client.0.smithi076.stdout:3/651: truncate d5/d17/d1c/d32/d39/d59/fd6 287039 0 2024-03-23T15:44:59.508 INFO:tasks.workunit.client.0.smithi076.stdout:8/764: write d8/d2e/d30/fc3 [366644,43789] 0 2024-03-23T15:44:59.508 INFO:tasks.workunit.client.0.smithi076.stdout:8/765: truncate d8/d89/db2/fc2 512589 0 2024-03-23T15:44:59.508 INFO:tasks.workunit.client.0.smithi076.stdout:8/766: readlink d8/d26/d27/l51 0 2024-03-23T15:44:59.509 INFO:tasks.workunit.client.0.smithi076.stdout:4/900: unlink d8/d9/f9c 0 2024-03-23T15:44:59.510 INFO:tasks.workunit.client.0.smithi076.stdout:4/901: write d8/d3a/d69/d73/df4/da9/fb6 [228938,104929] 0 2024-03-23T15:44:59.513 INFO:tasks.workunit.client.0.smithi076.stdout:3/652: mknod d5/d17/d1c/d31/d47/cdd 0 2024-03-23T15:44:59.514 INFO:tasks.workunit.client.0.smithi076.stdout:8/767: creat d8/d6b/d9d/f104 x:0 0 0 2024-03-23T15:44:59.516 INFO:tasks.workunit.client.0.smithi076.stdout:0/907: rename d2/de/d71 to d2/de/d10/d10d 0 2024-03-23T15:44:59.516 INFO:tasks.workunit.client.0.smithi076.stdout:0/908: stat d2/de/d10/d3a/d47/d69/d84/fa2 0 2024-03-23T15:44:59.518 INFO:tasks.workunit.client.0.smithi076.stdout:8/768: symlink d8/d6b/d9d/l105 0 2024-03-23T15:44:59.518 INFO:tasks.workunit.client.0.smithi076.stdout:8/769: chown d8/d89/db2/fcb 0 1 2024-03-23T15:44:59.520 INFO:tasks.workunit.client.0.smithi076.stdout:5/911: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/ffc [4194304,4194304] 0 2024-03-23T15:44:59.520 INFO:tasks.workunit.client.0.smithi076.stdout:5/912: write f19 [3138934,125343] 0 2024-03-23T15:44:59.521 INFO:tasks.workunit.client.0.smithi076.stdout:4/902: link d8/d80/c87 d8/d3a/d69/d73/df4/dd2/de0/c11f 0 2024-03-23T15:44:59.521 INFO:tasks.workunit.client.0.smithi076.stdout:4/903: dread - d8/d3a/d69/d73/d9a/ff3 zero size 2024-03-23T15:44:59.522 INFO:tasks.workunit.client.0.smithi076.stdout:5/913: rmdir d1a/d2c/d106 39 2024-03-23T15:44:59.524 INFO:tasks.workunit.client.0.smithi076.stdout:5/914: getdents d1a/d33/d3a/d67/da8/dc4 0 2024-03-23T15:44:59.527 INFO:tasks.workunit.client.0.smithi076.stdout:5/915: symlink d1a/d33/d3a/d80/l135 0 2024-03-23T15:44:59.529 INFO:tasks.workunit.client.0.smithi076.stdout:5/916: mkdir d1a/d33/d3a/d67/da8/dc4/d65/d136 0 2024-03-23T15:44:59.532 INFO:tasks.workunit.client.0.smithi076.stdout:5/917: creat d1a/d33/d36/f137 x:0 0 0 2024-03-23T15:44:59.532 INFO:tasks.workunit.client.0.smithi076.stdout:5/918: write d1a/d33/fe6 [707116,80359] 0 2024-03-23T15:44:59.532 INFO:tasks.workunit.client.0.smithi076.stdout:5/919: fdatasync d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fc1 0 2024-03-23T15:44:59.532 INFO:tasks.workunit.client.0.smithi076.stdout:5/920: dread - d1a/d4b/ff4 zero size 2024-03-23T15:44:59.534 INFO:tasks.workunit.client.0.smithi076.stdout:5/921: unlink d1a/d33/l103 0 2024-03-23T15:44:59.536 INFO:tasks.workunit.client.0.smithi076.stdout:5/922: truncate d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fd5 979732 0 2024-03-23T15:44:59.536 INFO:tasks.workunit.client.0.smithi076.stdout:5/923: stat d1a/d2c/fce 0 2024-03-23T15:44:59.537 INFO:tasks.workunit.client.0.smithi076.stdout:3/653: dwrite d5/d17/d1c/d32/d39/f75 [0,4194304] 0 2024-03-23T15:44:59.539 INFO:tasks.workunit.client.0.smithi076.stdout:6/675: link d1/dd/df/d9b/db3/fa1 d1/dd1/ff1 0 2024-03-23T15:44:59.539 INFO:tasks.workunit.client.0.smithi076.stdout:5/924: read d1a/d33/d3a/f8a [1072500,84571] 0 2024-03-23T15:44:59.542 INFO:tasks.workunit.client.0.smithi076.stdout:8/770: dwrite d8/d2e/d3e/f49 [0,4194304] 0 2024-03-23T15:44:59.542 INFO:tasks.workunit.client.0.smithi076.stdout:0/909: dwrite d2/d1b/fc9 [4194304,4194304] 0 2024-03-23T15:44:59.544 INFO:tasks.workunit.client.0.smithi076.stdout:4/904: dwrite d8/de3/db0/d10c/f10d [0,4194304] 0 2024-03-23T15:44:59.544 INFO:tasks.workunit.client.0.smithi076.stdout:4/905: read d8/d9/f12 [91393,6704] 0 2024-03-23T15:44:59.544 INFO:tasks.workunit.client.0.smithi076.stdout:4/906: truncate d8/d3a/d18/d1b/fa5 2691261 0 2024-03-23T15:44:59.546 INFO:tasks.workunit.client.0.smithi076.stdout:8/771: symlink d8/l106 0 2024-03-23T15:44:59.546 INFO:tasks.workunit.client.0.smithi076.stdout:6/676: unlink d1/d2/f1c 0 2024-03-23T15:44:59.549 INFO:tasks.workunit.client.0.smithi076.stdout:0/910: creat d2/de/d10/d3a/d47/f10e x:0 0 0 2024-03-23T15:44:59.549 INFO:tasks.workunit.client.0.smithi076.stdout:0/911: fsync d2/de/d10/d43/d56/f60 0 2024-03-23T15:44:59.550 INFO:tasks.workunit.client.0.smithi076.stdout:6/677: dread d1/dd/d2a/d23/faa [0,4194304] 0 2024-03-23T15:44:59.554 INFO:tasks.workunit.client.0.smithi076.stdout:9/946: rename d1/d1f/d46/d83/f73 to d1/d2f/d3f/da0/f133 0 2024-03-23T15:44:59.558 INFO:tasks.workunit.client.0.smithi076.stdout:8/772: rmdir d8/d4f/d80/db7 39 2024-03-23T15:44:59.558 INFO:tasks.workunit.client.0.smithi076.stdout:8/773: readlink d8/d26/d27/l29 0 2024-03-23T15:44:59.558 INFO:tasks.workunit.client.0.smithi076.stdout:5/925: dwrite d1a/d20/fad [0,4194304] 0 2024-03-23T15:44:59.563 INFO:tasks.workunit.client.0.smithi076.stdout:0/912: mkdir d2/de/d10/dc6/d10f 0 2024-03-23T15:44:59.563 INFO:tasks.workunit.client.0.smithi076.stdout:1/680: sync 2024-03-23T15:44:59.563 INFO:tasks.workunit.client.0.smithi076.stdout:2/931: sync 2024-03-23T15:44:59.563 INFO:tasks.workunit.client.0.smithi076.stdout:2/932: write d1/d4/db/d15/d29/f8f [842067,67473] 0 2024-03-23T15:44:59.564 INFO:tasks.workunit.client.0.smithi076.stdout:1/681: creat d0/d35/d40/d47/d8c/d99/fe3 x:0 0 0 2024-03-23T15:44:59.564 INFO:tasks.workunit.client.0.smithi076.stdout:5/926: unlink d1a/d33/ff0 0 2024-03-23T15:44:59.564 INFO:tasks.workunit.client.0.smithi076.stdout:1/682: dread d0/f4b [0,4194304] 0 2024-03-23T15:44:59.565 INFO:tasks.workunit.client.0.smithi076.stdout:4/907: rename d8/d3a/d3b/f25 to d8/d41/f120 0 2024-03-23T15:44:59.565 INFO:tasks.workunit.client.0.smithi076.stdout:4/908: fsync d8/d3a/d18/d52/f79 0 2024-03-23T15:44:59.565 INFO:tasks.workunit.client.0.smithi076.stdout:2/933: creat d1/d2e/d8a/f140 x:0 0 0 2024-03-23T15:44:59.565 INFO:tasks.workunit.client.0.smithi076.stdout:2/934: truncate d1/d4/db/f127 130948 0 2024-03-23T15:44:59.565 INFO:tasks.workunit.client.0.smithi076.stdout:2/935: stat d1/d4/d64/d1d/dbc/de6 0 2024-03-23T15:44:59.565 INFO:tasks.workunit.client.0.smithi076.stdout:5/927: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/d110/f138 x:0 0 0 2024-03-23T15:44:59.567 INFO:tasks.workunit.client.0.smithi076.stdout:5/928: truncate d1a/d33/d3a/f8a 1979336 0 2024-03-23T15:44:59.567 INFO:tasks.workunit.client.0.smithi076.stdout:5/929: write d1a/d2c/fce [1625501,48734] 0 2024-03-23T15:44:59.567 INFO:tasks.workunit.client.0.smithi076.stdout:5/930: rename d1a/d33/d3a/d67 to d1a/d33/d3a/d67/da8/dc4/d139 22 2024-03-23T15:44:59.569 INFO:tasks.workunit.client.0.smithi076.stdout:5/931: rename d1a/d2c/f11e to d1a/d4b/f13a 0 2024-03-23T15:44:59.569 INFO:tasks.workunit.client.0.smithi076.stdout:2/936: read d1/d4/d64/d1b/f6c [1739373,75147] 0 2024-03-23T15:44:59.569 INFO:tasks.workunit.client.0.smithi076.stdout:8/774: mknod d8/dd0/de1/c107 0 2024-03-23T15:44:59.569 INFO:tasks.workunit.client.0.smithi076.stdout:8/775: write d8/d2e/d30/d69/fdb [184960,11432] 0 2024-03-23T15:44:59.569 INFO:tasks.workunit.client.0.smithi076.stdout:8/776: chown d8/l15 7190994 1 2024-03-23T15:44:59.570 INFO:tasks.workunit.client.0.smithi076.stdout:5/932: creat d1a/d33/d3a/d67/d84/d12a/f13b x:0 0 0 2024-03-23T15:44:59.570 INFO:tasks.workunit.client.0.smithi076.stdout:5/933: stat f18 0 2024-03-23T15:44:59.570 INFO:tasks.workunit.client.0.smithi076.stdout:5/934: fsync d1a/d33/d3a/d80/fd8 0 2024-03-23T15:44:59.571 INFO:tasks.workunit.client.0.smithi076.stdout:5/935: write d1a/d33/d3a/d80/fca [136183,47194] 0 2024-03-23T15:44:59.572 INFO:tasks.workunit.client.0.smithi076.stdout:0/913: unlink d2/de/f1a 0 2024-03-23T15:44:59.572 INFO:tasks.workunit.client.0.smithi076.stdout:0/914: write d2/de/d10/d43/f50 [416699,122782] 0 2024-03-23T15:44:59.573 INFO:tasks.workunit.client.0.smithi076.stdout:4/909: write d8/d9/f5a [1956926,104432] 0 2024-03-23T15:44:59.578 INFO:tasks.workunit.client.0.smithi076.stdout:4/910: creat d8/de3/db0/f121 x:0 0 0 2024-03-23T15:44:59.578 INFO:tasks.workunit.client.0.smithi076.stdout:4/911: write d8/d3a/d69/d73/df4/da9/fb6 [1229329,2311] 0 2024-03-23T15:44:59.580 INFO:tasks.workunit.client.0.smithi076.stdout:5/936: dread d1a/f92 [0,4194304] 0 2024-03-23T15:44:59.580 INFO:tasks.workunit.client.0.smithi076.stdout:5/937: write f4 [3699750,74631] 0 2024-03-23T15:44:59.580 INFO:tasks.workunit.client.0.smithi076.stdout:5/938: write d1a/d33/d36/f10f [991007,4765] 0 2024-03-23T15:44:59.581 INFO:tasks.workunit.client.0.smithi076.stdout:4/912: rmdir d8/d3a/dab/dc3/df8 0 2024-03-23T15:44:59.581 INFO:tasks.workunit.client.0.smithi076.stdout:4/913: truncate d8/de3/db0/de9/ffc 620864 0 2024-03-23T15:44:59.582 INFO:tasks.workunit.client.0.smithi076.stdout:4/914: chown d8/d41/f54 164290113 1 2024-03-23T15:44:59.582 INFO:tasks.workunit.client.0.smithi076.stdout:5/939: fsync f4 0 2024-03-23T15:44:59.583 INFO:tasks.workunit.client.0.smithi076.stdout:8/777: write d8/fe6 [891058,24393] 0 2024-03-23T15:44:59.583 INFO:tasks.workunit.client.0.smithi076.stdout:4/915: creat d8/d3a/d69/d73/d76/d11c/f122 x:0 0 0 2024-03-23T15:44:59.585 INFO:tasks.workunit.client.0.smithi076.stdout:8/778: symlink d8/d26/d33/d8c/dff/l108 0 2024-03-23T15:44:59.585 INFO:tasks.workunit.client.0.smithi076.stdout:0/915: dread d2/de/d10/f68 [0,4194304] 0 2024-03-23T15:44:59.585 INFO:tasks.workunit.client.0.smithi076.stdout:1/683: dwrite d0/d35/d40/d47/f6b [0,4194304] 0 2024-03-23T15:44:59.585 INFO:tasks.workunit.client.0.smithi076.stdout:1/684: write d0/d2/d70/da1/dbf/f43 [1471857,3335] 0 2024-03-23T15:44:59.586 INFO:tasks.workunit.client.0.smithi076.stdout:8/779: truncate d8/d26/d33/f3a 1366243 0 2024-03-23T15:44:59.588 INFO:tasks.workunit.client.0.smithi076.stdout:1/685: link d0/f9d d0/d2/d70/fe4 0 2024-03-23T15:44:59.590 INFO:tasks.workunit.client.0.smithi076.stdout:8/780: creat d8/d26/d27/d56/f109 x:0 0 0 2024-03-23T15:44:59.591 INFO:tasks.workunit.client.0.smithi076.stdout:8/781: dread - d8/d2e/d30/d69/f71 zero size 2024-03-23T15:44:59.592 INFO:tasks.workunit.client.0.smithi076.stdout:8/782: creat d8/d26/d33/d101/f10a x:0 0 0 2024-03-23T15:44:59.596 INFO:tasks.workunit.client.0.smithi076.stdout:9/947: rmdir d1/d1f/d46 39 2024-03-23T15:44:59.599 INFO:tasks.workunit.client.0.smithi076.stdout:8/783: rename d8/fa to d8/dd0/f10b 0 2024-03-23T15:44:59.599 INFO:tasks.workunit.client.0.smithi076.stdout:8/784: write d8/d2e/d30/d69/f82 [1148764,11299] 0 2024-03-23T15:44:59.600 INFO:tasks.workunit.client.0.smithi076.stdout:8/785: mknod d8/d89/db2/dd5/c10c 0 2024-03-23T15:44:59.602 INFO:tasks.workunit.client.0.smithi076.stdout:8/786: symlink d8/d26/d27/d56/l10d 0 2024-03-23T15:44:59.603 INFO:tasks.workunit.client.0.smithi076.stdout:8/787: symlink d8/d89/db2/l10e 0 2024-03-23T15:44:59.603 INFO:tasks.workunit.client.0.smithi076.stdout:8/788: fsync d8/d26/d33/d8c/fee 0 2024-03-23T15:44:59.612 INFO:tasks.workunit.client.0.smithi076.stdout:0/916: dwrite d2/de/d10/f61 [0,4194304] 0 2024-03-23T15:44:59.612 INFO:tasks.workunit.client.0.smithi076.stdout:0/917: write d2/d8/d52/f107 [542339,18712] 0 2024-03-23T15:44:59.614 INFO:tasks.workunit.client.0.smithi076.stdout:0/918: stat d2/de/d10/d12/c51 0 2024-03-23T15:44:59.614 INFO:tasks.workunit.client.0.smithi076.stdout:0/919: write d2/de/d10/d43/fb6 [5109138,12679] 0 2024-03-23T15:44:59.615 INFO:tasks.workunit.client.0.smithi076.stdout:4/916: dwrite d8/d41/d97/fb9 [0,4194304] 0 2024-03-23T15:44:59.615 INFO:tasks.workunit.client.0.smithi076.stdout:4/917: dread - d8/d3a/d18/d75/fee zero size 2024-03-23T15:44:59.616 INFO:tasks.workunit.client.0.smithi076.stdout:5/940: dwrite d1a/d33/f54 [4194304,4194304] 0 2024-03-23T15:44:59.623 INFO:tasks.workunit.client.0.smithi076.stdout:5/941: dread d1a/d33/dc3/da6/fea [0,4194304] 0 2024-03-23T15:44:59.624 INFO:tasks.workunit.client.0.smithi076.stdout:5/942: rmdir d1a/d33/dc3/d105 39 2024-03-23T15:44:59.627 INFO:tasks.workunit.client.0.smithi076.stdout:0/920: dwrite d2/de/f95 [0,4194304] 0 2024-03-23T15:44:59.628 INFO:tasks.workunit.client.0.smithi076.stdout:4/918: mkdir d8/d3a/d3b/d123 0 2024-03-23T15:44:59.629 INFO:tasks.workunit.client.0.smithi076.stdout:4/919: dread - d8/f10a zero size 2024-03-23T15:44:59.629 INFO:tasks.workunit.client.0.smithi076.stdout:4/920: read - d8/d3a/d69/d73/d9a/ff3 zero size 2024-03-23T15:44:59.629 INFO:tasks.workunit.client.0.smithi076.stdout:4/921: fdatasync d8/d9/d32/f72 0 2024-03-23T15:44:59.631 INFO:tasks.workunit.client.0.smithi076.stdout:8/789: rename d8/d26/l34 to d8/d26/d33/d8c/dff/l10f 0 2024-03-23T15:44:59.631 INFO:tasks.workunit.client.0.smithi076.stdout:0/921: rmdir d2/ded 39 2024-03-23T15:44:59.632 INFO:tasks.workunit.client.0.smithi076.stdout:9/948: link d1/d1f/d46/d4a/l99 d1/d1f/d82/l134 0 2024-03-23T15:44:59.634 INFO:tasks.workunit.client.0.smithi076.stdout:8/790: mkdir d8/d2e/d30/d69/dd3/d110 0 2024-03-23T15:44:59.641 INFO:tasks.workunit.client.0.smithi076.stdout:0/922: dread d2/fa9 [0,4194304] 0 2024-03-23T15:44:59.641 INFO:tasks.workunit.client.0.smithi076.stdout:0/923: truncate d2/d9a/fbc 698289 0 2024-03-23T15:44:59.641 INFO:tasks.workunit.client.0.smithi076.stdout:0/924: getdents d2/de/d10/dc6/d10f 0 2024-03-23T15:44:59.641 INFO:tasks.workunit.client.0.smithi076.stdout:9/949: write d1/d2f/d3a/d53/f61 [11744519,54793] 0 2024-03-23T15:44:59.641 INFO:tasks.workunit.client.0.smithi076.stdout:9/950: fdatasync d1/d2f/d3f/da0/dd5/fdd 0 2024-03-23T15:44:59.642 INFO:tasks.workunit.client.0.smithi076.stdout:8/791: dread d8/d2e/d3e/f49 [0,4194304] 0 2024-03-23T15:44:59.642 INFO:tasks.workunit.client.0.smithi076.stdout:9/951: mknod d1/d1f/c135 0 2024-03-23T15:44:59.642 INFO:tasks.workunit.client.0.smithi076.stdout:9/952: rmdir d1/dfd 39 2024-03-23T15:44:59.647 INFO:tasks.workunit.client.0.smithi076.stdout:9/953: creat d1/d2f/d3f/f136 x:0 0 0 2024-03-23T15:44:59.647 INFO:tasks.workunit.client.0.smithi076.stdout:9/954: stat d1/d2f/d3f/da0/cc8 0 2024-03-23T15:44:59.647 INFO:tasks.workunit.client.0.smithi076.stdout:9/955: write d1/d1f/d29/d2d/f86 [2090216,111837] 0 2024-03-23T15:44:59.647 INFO:tasks.workunit.client.0.smithi076.stdout:9/956: truncate d1/d2f/d3f/da0/fa6 444761 0 2024-03-23T15:44:59.647 INFO:tasks.workunit.client.0.smithi076.stdout:9/957: write d1/d2f/d3a/d74/f123 [358248,27264] 0 2024-03-23T15:44:59.647 INFO:tasks.workunit.client.0.smithi076.stdout:9/958: creat d1/d2f/d3a/f137 x:0 0 0 2024-03-23T15:44:59.648 INFO:tasks.workunit.client.0.smithi076.stdout:9/959: write d1/d1f/d46/d4a/f10a [599077,83525] 0 2024-03-23T15:44:59.648 INFO:tasks.workunit.client.0.smithi076.stdout:9/960: write d1/d1f/d29/d7f/f111 [430899,46713] 0 2024-03-23T15:44:59.648 INFO:tasks.workunit.client.0.smithi076.stdout:8/792: dread d8/d89/f5f [0,4194304] 0 2024-03-23T15:44:59.648 INFO:tasks.workunit.client.0.smithi076.stdout:4/922: dwrite d8/d9/f12 [4194304,4194304] 0 2024-03-23T15:44:59.648 INFO:tasks.workunit.client.0.smithi076.stdout:4/923: readlink d8/d3a/d18/ldc 0 2024-03-23T15:44:59.648 INFO:tasks.workunit.client.0.smithi076.stdout:4/924: symlink d8/dd7/l124 0 2024-03-23T15:44:59.650 INFO:tasks.workunit.client.0.smithi076.stdout:4/925: symlink d8/de3/l125 0 2024-03-23T15:44:59.651 INFO:tasks.workunit.client.0.smithi076.stdout:4/926: dread d8/d9/f5c [0,4194304] 0 2024-03-23T15:44:59.651 INFO:tasks.workunit.client.0.smithi076.stdout:4/927: fsync d8/f28 0 2024-03-23T15:44:59.653 INFO:tasks.workunit.client.0.smithi076.stdout:4/928: mkdir d8/d3a/d69/d73/df4/dd2/de0/d126 0 2024-03-23T15:44:59.655 INFO:tasks.workunit.client.0.smithi076.stdout:4/929: getdents d8/d3a/d69 0 2024-03-23T15:44:59.657 INFO:tasks.workunit.client.0.smithi076.stdout:4/930: link d8/de3/fec d8/d3a/d69/de4/f127 0 2024-03-23T15:44:59.670 INFO:tasks.workunit.client.0.smithi076.stdout:3/654: sync 2024-03-23T15:44:59.670 INFO:tasks.workunit.client.0.smithi076.stdout:9/961: rename d1/d2f/d3a/d74/dd7/c115 to d1/d1f/d46/d83/dad/c138 0 2024-03-23T15:44:59.670 INFO:tasks.workunit.client.0.smithi076.stdout:9/962: fdatasync d1/d2f/d3a/d64/fea 0 2024-03-23T15:44:59.672 INFO:tasks.workunit.client.0.smithi076.stdout:8/793: rename d8/d2e/d30/d69/dd3/fed to d8/d2e/d30/d69/dd3/d100/f111 0 2024-03-23T15:44:59.675 INFO:tasks.workunit.client.0.smithi076.stdout:8/794: creat d8/dd0/df0/f112 x:0 0 0 2024-03-23T15:44:59.681 INFO:tasks.workunit.client.0.smithi076.stdout:8/795: dread d8/d2e/d30/f61 [4194304,4194304] 0 2024-03-23T15:44:59.681 INFO:tasks.workunit.client.0.smithi076.stdout:8/796: stat d8/d26/d27/d56/la4 0 2024-03-23T15:44:59.681 INFO:tasks.workunit.client.0.smithi076.stdout:8/797: fsync d8/d26/d33/f3a 0 2024-03-23T15:44:59.682 INFO:tasks.workunit.client.0.smithi076.stdout:8/798: mknod d8/d26/d33/d8c/c113 0 2024-03-23T15:44:59.682 INFO:tasks.workunit.client.0.smithi076.stdout:9/963: dwrite d1/d2f/d3a/d74/fd4 [0,4194304] 0 2024-03-23T15:44:59.683 INFO:tasks.workunit.client.0.smithi076.stdout:8/799: dread d8/d89/db2/fc2 [0,4194304] 0 2024-03-23T15:44:59.683 INFO:tasks.workunit.client.0.smithi076.stdout:9/964: truncate d1/d1f/d29/d2d/d37/f3c 515300 0 2024-03-23T15:44:59.684 INFO:tasks.workunit.client.0.smithi076.stdout:9/965: truncate d1/d1f/d46/d83/dad/f118 4430748 0 2024-03-23T15:44:59.686 INFO:tasks.workunit.client.0.smithi076.stdout:8/800: creat d8/f114 x:0 0 0 2024-03-23T15:44:59.686 INFO:tasks.workunit.client.0.smithi076.stdout:8/801: truncate d8/f1a 4799330 0 2024-03-23T15:44:59.686 INFO:tasks.workunit.client.0.smithi076.stdout:8/802: write d8/d2e/d3e/f44 [4735346,45455] 0 2024-03-23T15:44:59.688 INFO:tasks.workunit.client.0.smithi076.stdout:9/966: unlink d1/d1f/d46/d83/dad/cf2 0 2024-03-23T15:44:59.688 INFO:tasks.workunit.client.0.smithi076.stdout:9/967: chown d1/d2f/d9f/l12f 135 1 2024-03-23T15:44:59.692 INFO:tasks.workunit.client.0.smithi076.stdout:3/655: rename d5/d17/d1c/d32/d39/d59/d79/fd9 to d5/d17/d1c/d32/d70/fde 0 2024-03-23T15:44:59.693 INFO:tasks.workunit.client.0.smithi076.stdout:8/803: write d8/f48 [3181008,29108] 0 2024-03-23T15:44:59.703 INFO:tasks.workunit.client.0.smithi076.stdout:9/968: dwrite d1/d1f/d29/d7f/f111 [0,4194304] 0 2024-03-23T15:44:59.703 INFO:tasks.workunit.client.0.smithi076.stdout:9/969: fdatasync d1/d1f/d29/d2d/d37/fb0 0 2024-03-23T15:44:59.704 INFO:tasks.workunit.client.0.smithi076.stdout:8/804: link d8/d26/d27/d56/f8b d8/d2e/d30/d64/f115 0 2024-03-23T15:44:59.706 INFO:tasks.workunit.client.0.smithi076.stdout:8/805: mknod d8/d2e/d3e/c116 0 2024-03-23T15:44:59.706 INFO:tasks.workunit.client.0.smithi076.stdout:8/806: stat d8/d2e/d30/d69/f71 0 2024-03-23T15:44:59.707 INFO:tasks.workunit.client.0.smithi076.stdout:8/807: readlink d8/l12 0 2024-03-23T15:44:59.716 INFO:tasks.workunit.client.0.smithi076.stdout:3/656: dwrite d5/f43 [0,4194304] 0 2024-03-23T15:44:59.716 INFO:tasks.workunit.client.0.smithi076.stdout:9/970: read d1/d20/f97 [531957,28491] 0 2024-03-23T15:44:59.717 INFO:tasks.workunit.client.0.smithi076.stdout:9/971: truncate d1/d20/f45 4574523 0 2024-03-23T15:44:59.717 INFO:tasks.workunit.client.0.smithi076.stdout:9/972: creat d1/d20/f139 x:0 0 0 2024-03-23T15:44:59.717 INFO:tasks.workunit.client.0.smithi076.stdout:8/808: rmdir d8/d89 39 2024-03-23T15:44:59.717 INFO:tasks.workunit.client.0.smithi076.stdout:8/809: dread - d8/d83/fce zero size 2024-03-23T15:44:59.723 INFO:tasks.workunit.client.0.smithi076.stdout:3/657: dread d5/d17/d1c/f2b [0,4194304] 0 2024-03-23T15:44:59.723 INFO:tasks.workunit.client.0.smithi076.stdout:3/658: write d5/d17/d1c/d32/d70/fc1 [261022,82730] 0 2024-03-23T15:44:59.723 INFO:tasks.workunit.client.0.smithi076.stdout:3/659: chown d5/d35/d4c/d5a/l3f 0 1 2024-03-23T15:44:59.725 INFO:tasks.workunit.client.0.smithi076.stdout:3/660: symlink d5/d35/d6d/d7f/ldf 0 2024-03-23T15:44:59.727 INFO:tasks.workunit.client.0.smithi076.stdout:3/661: mknod d5/d17/d1c/d32/d5d/ce0 0 2024-03-23T15:44:59.728 INFO:tasks.workunit.client.0.smithi076.stdout:3/662: mkdir d5/d35/d4c/de1 0 2024-03-23T15:44:59.728 INFO:tasks.workunit.client.0.smithi076.stdout:3/663: write d5/d35/d4c/d5a/fb0 [2590605,122716] 0 2024-03-23T15:44:59.729 INFO:tasks.workunit.client.0.smithi076.stdout:8/810: dwrite f3 [0,4194304] 0 2024-03-23T15:44:59.729 INFO:tasks.workunit.client.0.smithi076.stdout:8/811: chown d8/d26/d33/cc4 1 1 2024-03-23T15:44:59.729 INFO:tasks.workunit.client.0.smithi076.stdout:8/812: chown d8/d89/db2/fcb 276 1 2024-03-23T15:44:59.730 INFO:tasks.workunit.client.0.smithi076.stdout:8/813: dread - d8/d26/d33/fe8 zero size 2024-03-23T15:44:59.730 INFO:tasks.workunit.client.0.smithi076.stdout:8/814: chown d8/le2 0 1 2024-03-23T15:44:59.730 INFO:tasks.workunit.client.0.smithi076.stdout:8/815: truncate d8/d89/db2/fc2 883386 0 2024-03-23T15:44:59.730 INFO:tasks.workunit.client.0.smithi076.stdout:8/816: write d8/d26/d27/d56/f63 [763802,86785] 0 2024-03-23T15:44:59.730 INFO:tasks.workunit.client.0.smithi076.stdout:8/817: dread - d8/d6b/d91/f98 zero size 2024-03-23T15:44:59.730 INFO:tasks.workunit.client.0.smithi076.stdout:3/664: rename d5/d17/d1c/d31/d6b to d5/d17/dcf/de2 0 2024-03-23T15:44:59.731 INFO:tasks.workunit.client.0.smithi076.stdout:3/665: rmdir d5/d17/d1c/d32/d5d 39 2024-03-23T15:44:59.731 INFO:tasks.workunit.client.0.smithi076.stdout:3/666: write d5/d35/d6d/d7f/fae [813216,113688] 0 2024-03-23T15:44:59.731 INFO:tasks.workunit.client.0.smithi076.stdout:3/667: write d5/f9 [4991783,30218] 0 2024-03-23T15:44:59.732 INFO:tasks.workunit.client.0.smithi076.stdout:3/668: chown d5/d17/d1c/d32/d5d/l80 339252021 1 2024-03-23T15:44:59.732 INFO:tasks.workunit.client.0.smithi076.stdout:3/669: creat d5/d17/fe3 x:0 0 0 2024-03-23T15:44:59.738 INFO:tasks.workunit.client.0.smithi076.stdout:9/973: dwrite d1/d2f/d3a/d53/f121 [0,4194304] 0 2024-03-23T15:44:59.738 INFO:tasks.workunit.client.0.smithi076.stdout:9/974: dread - d1/d1f/d29/d7f/ff6 zero size 2024-03-23T15:44:59.739 INFO:tasks.workunit.client.0.smithi076.stdout:9/975: mknod d1/d1f/d29/d2d/d37/d47/d11c/c13a 0 2024-03-23T15:44:59.739 INFO:tasks.workunit.client.0.smithi076.stdout:9/976: write d1/d1f/f30 [7262672,3918] 0 2024-03-23T15:44:59.744 INFO:tasks.workunit.client.0.smithi076.stdout:9/977: rmdir d1/d1f/d46/d83/dad 39 2024-03-23T15:44:59.749 INFO:tasks.workunit.client.0.smithi076.stdout:8/818: dwrite d8/d26/f4d [0,4194304] 0 2024-03-23T15:44:59.752 INFO:tasks.workunit.client.0.smithi076.stdout:9/978: write d1/f1a [13050014,77798] 0 2024-03-23T15:44:59.753 INFO:tasks.workunit.client.0.smithi076.stdout:9/979: creat d1/d1f/d29/f13b x:0 0 0 2024-03-23T15:44:59.755 INFO:tasks.workunit.client.0.smithi076.stdout:3/670: dwrite d5/d35/f36 [0,4194304] 0 2024-03-23T15:44:59.755 INFO:tasks.workunit.client.0.smithi076.stdout:3/671: truncate d5/d35/d4c/d5a/f49 4401306 0 2024-03-23T15:44:59.755 INFO:tasks.workunit.client.0.smithi076.stdout:3/672: fsync d5/d35/d3c/f94 0 2024-03-23T15:44:59.756 INFO:tasks.workunit.client.0.smithi076.stdout:8/819: write d8/d26/d33/f3d [1869385,127351] 0 2024-03-23T15:44:59.757 INFO:tasks.workunit.client.0.smithi076.stdout:3/673: rmdir d5/d17/d1c/d32/d39/dcc 0 2024-03-23T15:44:59.771 INFO:tasks.workunit.client.0.smithi076.stdout:9/980: dwrite d1/f12 [0,4194304] 0 2024-03-23T15:44:59.771 INFO:tasks.workunit.client.0.smithi076.stdout:9/981: fdatasync d1/d20/f45 0 2024-03-23T15:44:59.772 INFO:tasks.workunit.client.0.smithi076.stdout:9/982: write d1/d1f/d29/d7f/ff6 [1034225,72394] 0 2024-03-23T15:44:59.774 INFO:tasks.workunit.client.0.smithi076.stdout:6/678: sync 2024-03-23T15:44:59.775 INFO:tasks.workunit.client.0.smithi076.stdout:8/820: rename d8/d2e/f4a to d8/d89/f117 0 2024-03-23T15:44:59.781 INFO:tasks.workunit.client.0.smithi076.stdout:6/679: readlink d1/dd/df/d5b/dcd/lc5 0 2024-03-23T15:44:59.784 INFO:tasks.workunit.client.0.smithi076.stdout:3/674: rename d5/d35/d6d/d7f/fac to d5/d17/d1c/d32/d8e/fe4 0 2024-03-23T15:44:59.788 INFO:tasks.workunit.client.0.smithi076.stdout:8/821: rename d8/d89/db2/dd5/cfc to d8/d89/db2/c118 0 2024-03-23T15:44:59.788 INFO:tasks.workunit.client.0.smithi076.stdout:8/822: fdatasync d8/d2e/d30/d69/f71 0 2024-03-23T15:44:59.788 INFO:tasks.workunit.client.0.smithi076.stdout:8/823: stat d8/c54 0 2024-03-23T15:44:59.788 INFO:tasks.workunit.client.0.smithi076.stdout:8/824: stat d8/d26/d33/cc4 0 2024-03-23T15:44:59.788 INFO:tasks.workunit.client.0.smithi076.stdout:3/675: creat d5/d17/d1c/d32/d70/fe5 x:0 0 0 2024-03-23T15:44:59.788 INFO:tasks.workunit.client.0.smithi076.stdout:3/676: chown d5/d17/d1c/f2b 1044824 1 2024-03-23T15:44:59.791 INFO:tasks.workunit.client.0.smithi076.stdout:2/937: sync 2024-03-23T15:44:59.793 INFO:tasks.workunit.client.0.smithi076.stdout:2/938: dread d1/d4/d64/d82/d103/d79/dac/dd8/f11f [0,4194304] 0 2024-03-23T15:44:59.793 INFO:tasks.workunit.client.0.smithi076.stdout:2/939: chown d1/d4/d64/c57 209 1 2024-03-23T15:44:59.796 INFO:tasks.workunit.client.0.smithi076.stdout:2/940: symlink d1/d4/d52/ded/l141 0 2024-03-23T15:44:59.797 INFO:tasks.workunit.client.0.smithi076.stdout:8/825: getdents d8/d2e/d30 0 2024-03-23T15:44:59.799 INFO:tasks.workunit.client.0.smithi076.stdout:2/941: rename d1/d4/d64/d82/d103/d79/dac/dd8/dea/fef to d1/dbf/f142 0 2024-03-23T15:44:59.799 INFO:tasks.workunit.client.0.smithi076.stdout:2/942: chown d1/d4/d52/ccd 4 1 2024-03-23T15:44:59.800 INFO:tasks.workunit.client.0.smithi076.stdout:8/826: mknod d8/d2e/d30/c119 0 2024-03-23T15:44:59.802 INFO:tasks.workunit.client.0.smithi076.stdout:6/680: dwrite d1/dd/df/d5b/d62/f8b [0,4194304] 0 2024-03-23T15:44:59.802 INFO:tasks.workunit.client.0.smithi076.stdout:6/681: stat d1/d4/d10/d39/f76 0 2024-03-23T15:44:59.802 INFO:tasks.workunit.client.0.smithi076.stdout:9/983: link d1/d2f/d3f/da0/f133 d1/d1f/d29/d7f/f13c 0 2024-03-23T15:44:59.802 INFO:tasks.workunit.client.0.smithi076.stdout:9/984: chown d1/d2f/d3f/da0/fd9 4888441 1 2024-03-23T15:44:59.805 INFO:tasks.workunit.client.0.smithi076.stdout:3/677: dwrite d5/d35/d4c/d5a/fbd [0,4194304] 0 2024-03-23T15:44:59.810 INFO:tasks.workunit.client.0.smithi076.stdout:3/678: symlink d5/d17/d1c/d31/d61/d96/le6 0 2024-03-23T15:44:59.812 INFO:tasks.workunit.client.0.smithi076.stdout:3/679: mkdir d5/d17/dcf/de2/de7 0 2024-03-23T15:44:59.815 INFO:tasks.workunit.client.0.smithi076.stdout:3/680: symlink d5/d17/d1c/le8 0 2024-03-23T15:44:59.815 INFO:tasks.workunit.client.0.smithi076.stdout:3/681: chown d5/d35/d3c/d67/fbc 7392 1 2024-03-23T15:44:59.815 INFO:tasks.workunit.client.0.smithi076.stdout:3/682: write d5/d35/d4c/d5a/f81 [901144,85368] 0 2024-03-23T15:44:59.815 INFO:tasks.workunit.client.0.smithi076.stdout:2/943: dread d1/d4/d64/fc7 [0,4194304] 0 2024-03-23T15:44:59.816 INFO:tasks.workunit.client.0.smithi076.stdout:2/944: write d1/d4/d64/fc7 [4199199,85716] 0 2024-03-23T15:44:59.816 INFO:tasks.workunit.client.0.smithi076.stdout:2/945: write d1/d4/d64/d82/ffa [5319555,66056] 0 2024-03-23T15:44:59.816 INFO:tasks.workunit.client.0.smithi076.stdout:9/985: rename d1/d2f/d3f/da0/f133 to d1/dfd/f13d 0 2024-03-23T15:44:59.816 INFO:tasks.workunit.client.0.smithi076.stdout:2/946: creat d1/d4/f143 x:0 0 0 2024-03-23T15:44:59.818 INFO:tasks.workunit.client.0.smithi076.stdout:2/947: creat d1/d4/d52/f144 x:0 0 0 2024-03-23T15:44:59.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:59 smithi192 ceph-mon[36653]: scrub summary: idle 2024-03-23T15:44:59.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:44:59 smithi192 ceph-mon[36653]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 149 MiB/s rd, 193 MiB/s wr, 1.82k op/s 2024-03-23T15:44:59.821 INFO:tasks.workunit.client.0.smithi076.stdout:9/986: creat d1/f13e x:0 0 0 2024-03-23T15:44:59.824 INFO:tasks.workunit.client.0.smithi076.stdout:3/683: dread d5/f28 [0,4194304] 0 2024-03-23T15:44:59.824 INFO:tasks.workunit.client.0.smithi076.stdout:3/684: write d5/d35/d4c/fa6 [741563,39191] 0 2024-03-23T15:44:59.824 INFO:tasks.workunit.client.0.smithi076.stdout:6/682: write d1/d81/f9a [2948672,36474] 0 2024-03-23T15:44:59.824 INFO:tasks.workunit.client.0.smithi076.stdout:6/683: write d1/d4/f35 [974948,50526] 0 2024-03-23T15:44:59.824 INFO:tasks.workunit.client.0.smithi076.stdout:3/685: link d5/d35/d4c/d5a/d9d/cd0 d5/d35/ce9 0 2024-03-23T15:44:59.824 INFO:tasks.workunit.client.0.smithi076.stdout:3/686: mknod d5/d17/cea 0 2024-03-23T15:44:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:59 smithi076 ceph-mon[32176]: scrub summary: idle 2024-03-23T15:44:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:44:59 smithi076 ceph-mon[32176]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 149 MiB/s rd, 193 MiB/s wr, 1.82k op/s 2024-03-23T15:44:59.826 INFO:tasks.workunit.client.0.smithi076.stdout:2/948: dread d1/d4/d64/d82/d103/d79/fb5 [0,4194304] 0 2024-03-23T15:44:59.828 INFO:tasks.workunit.client.0.smithi076.stdout:8/827: dwrite d8/d4f/d80/da3/fdf [0,4194304] 0 2024-03-23T15:44:59.837 INFO:tasks.workunit.client.0.smithi076.stdout:3/687: creat d5/d17/d1c/d32/d4a/feb x:0 0 0 2024-03-23T15:44:59.837 INFO:tasks.workunit.client.0.smithi076.stdout:3/688: write d5/d35/d3c/f94 [55624,3088] 0 2024-03-23T15:44:59.840 INFO:tasks.workunit.client.0.smithi076.stdout:3/689: mknod d5/dc2/cec 0 2024-03-23T15:44:59.840 INFO:tasks.workunit.client.0.smithi076.stdout:6/684: dread d1/d81/f9a [0,4194304] 0 2024-03-23T15:44:59.841 INFO:tasks.workunit.client.0.smithi076.stdout:9/987: dwrite d1/d1f/d46/d83/dad/f12e [0,4194304] 0 2024-03-23T15:44:59.842 INFO:tasks.workunit.client.0.smithi076.stdout:8/828: dread d8/d26/f4d [0,4194304] 0 2024-03-23T15:44:59.843 INFO:tasks.workunit.client.0.smithi076.stdout:8/829: mkdir d8/d2e/d30/d69/dd3/d110/d11a 0 2024-03-23T15:44:59.845 INFO:tasks.workunit.client.0.smithi076.stdout:3/690: rmdir d5/d35/d4c 39 2024-03-23T15:44:59.845 INFO:tasks.workunit.client.0.smithi076.stdout:3/691: truncate d5/f56 527486 0 2024-03-23T15:44:59.850 INFO:tasks.workunit.client.0.smithi076.stdout:9/988: dread d1/d2f/d3a/d74/fd4 [0,4194304] 0 2024-03-23T15:44:59.852 INFO:tasks.workunit.client.0.smithi076.stdout:9/989: creat d1/d2f/d9f/f13f x:0 0 0 2024-03-23T15:44:59.852 INFO:tasks.workunit.client.0.smithi076.stdout:9/990: chown d1/d1f/l24 290940 1 2024-03-23T15:44:59.855 INFO:tasks.workunit.client.0.smithi076.stdout:6/685: mkdir d1/d2/df2 0 2024-03-23T15:44:59.855 INFO:tasks.workunit.client.0.smithi076.stdout:8/830: mknod d8/d4f/d80/da3/df7/c11b 0 2024-03-23T15:44:59.857 INFO:tasks.workunit.client.0.smithi076.stdout:6/686: creat d1/d4/d10/ff3 x:0 0 0 2024-03-23T15:44:59.859 INFO:tasks.workunit.client.0.smithi076.stdout:6/687: rmdir d1/d4/d10 39 2024-03-23T15:44:59.859 INFO:tasks.workunit.client.0.smithi076.stdout:6/688: dread - d1/dd/df/fb2 zero size 2024-03-23T15:44:59.860 INFO:tasks.workunit.client.0.smithi076.stdout:6/689: getdents d1/dd/df/d9b/db3/d8d/d60/d7c 0 2024-03-23T15:44:59.868 INFO:tasks.workunit.client.0.smithi076.stdout:8/831: dwrite d8/f114 [0,4194304] 0 2024-03-23T15:44:59.871 INFO:tasks.workunit.client.0.smithi076.stdout:2/949: link d1/d4/d64/d82/d7f/le7 d1/d4/d64/d1d/dbc/l145 0 2024-03-23T15:44:59.871 INFO:tasks.workunit.client.0.smithi076.stdout:2/950: dread - d1/d4/db/dc8/f108 zero size 2024-03-23T15:44:59.872 INFO:tasks.workunit.client.0.smithi076.stdout:0/925: sync 2024-03-23T15:44:59.872 INFO:tasks.workunit.client.0.smithi076.stdout:4/931: sync 2024-03-23T15:44:59.872 INFO:tasks.workunit.client.0.smithi076.stdout:1/686: sync 2024-03-23T15:44:59.872 INFO:tasks.workunit.client.0.smithi076.stdout:5/943: sync 2024-03-23T15:44:59.872 INFO:tasks.workunit.client.0.smithi076.stdout:0/926: fsync d2/de/d10/d12/f9e 0 2024-03-23T15:44:59.872 INFO:tasks.workunit.client.0.smithi076.stdout:5/944: fsync d1a/f3c 0 2024-03-23T15:44:59.873 INFO:tasks.workunit.client.0.smithi076.stdout:1/687: dread d0/d2/d70/fb6 [0,4194304] 0 2024-03-23T15:44:59.873 INFO:tasks.workunit.client.0.smithi076.stdout:1/688: readlink d0/d35/d40/d47/d17/l5f 0 2024-03-23T15:44:59.873 INFO:tasks.workunit.client.0.smithi076.stdout:3/692: link d5/d17/d1c/d32/d39/f6c d5/d35/d4c/d5a/d9d/fed 0 2024-03-23T15:44:59.874 INFO:tasks.workunit.client.0.smithi076.stdout:5/945: getdents d1a/d33/d3a/d67/da8 0 2024-03-23T15:44:59.874 INFO:tasks.workunit.client.0.smithi076.stdout:5/946: write d1a/f68 [183477,50596] 0 2024-03-23T15:44:59.875 INFO:tasks.workunit.client.0.smithi076.stdout:5/947: dread - d1a/d7e/fe7 zero size 2024-03-23T15:44:59.875 INFO:tasks.workunit.client.0.smithi076.stdout:3/693: chown d5/d17/d1c/d32/d39/c3b 165 1 2024-03-23T15:44:59.875 INFO:tasks.workunit.client.0.smithi076.stdout:0/927: creat d2/d8/f110 x:0 0 0 2024-03-23T15:44:59.876 INFO:tasks.workunit.client.0.smithi076.stdout:5/948: mkdir d1a/d33/d13c 0 2024-03-23T15:44:59.876 INFO:tasks.workunit.client.0.smithi076.stdout:5/949: readlink d1a/d33/dc3/da6/l11a 0 2024-03-23T15:44:59.877 INFO:tasks.workunit.client.0.smithi076.stdout:5/950: chown d1a/d7e 6493086 1 2024-03-23T15:44:59.877 INFO:tasks.workunit.client.0.smithi076.stdout:6/690: link d1/dd/df/d5b/d62/f6e d1/d4/d10/d39/ff4 0 2024-03-23T15:44:59.878 INFO:tasks.workunit.client.0.smithi076.stdout:0/928: creat d2/de/d10/f111 x:0 0 0 2024-03-23T15:44:59.879 INFO:tasks.workunit.client.0.smithi076.stdout:6/691: creat d1/dd1/ff5 x:0 0 0 2024-03-23T15:44:59.879 INFO:tasks.workunit.client.0.smithi076.stdout:6/692: readlink d1/dd/df/l4b 0 2024-03-23T15:44:59.879 INFO:tasks.workunit.client.0.smithi076.stdout:5/951: mknod d1a/d33/d3a/d67/da8/dc4/d65/c13d 0 2024-03-23T15:44:59.879 INFO:tasks.workunit.client.0.smithi076.stdout:5/952: getdents d1a/d33/d3a/d120 0 2024-03-23T15:44:59.882 INFO:tasks.workunit.client.0.smithi076.stdout:6/693: mkdir d1/d81/df6 0 2024-03-23T15:44:59.882 INFO:tasks.workunit.client.0.smithi076.stdout:5/953: link d1a/d2c/ff7 d1a/d33/d3a/d80/f13e 0 2024-03-23T15:44:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:59 smithi183 ceph-mon[36296]: scrub summary: idle 2024-03-23T15:44:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:44:59 smithi183 ceph-mon[36296]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 149 MiB/s rd, 193 MiB/s wr, 1.82k op/s 2024-03-23T15:44:59.884 INFO:tasks.workunit.client.0.smithi076.stdout:8/832: dwrite f7 [8388608,4194304] 0 2024-03-23T15:44:59.885 INFO:tasks.workunit.client.0.smithi076.stdout:9/991: rename d1/d1f/d46/d4a/l9a to d1/d20/l140 0 2024-03-23T15:44:59.887 INFO:tasks.workunit.client.0.smithi076.stdout:9/992: creat d1/d2f/d9f/f141 x:0 0 0 2024-03-23T15:44:59.887 INFO:tasks.workunit.client.0.smithi076.stdout:4/932: dread d8/d9/d1d/f4f [0,4194304] 0 2024-03-23T15:44:59.888 INFO:tasks.workunit.client.0.smithi076.stdout:0/929: dread d2/de/d10/d43/f63 [0,4194304] 0 2024-03-23T15:44:59.888 INFO:tasks.workunit.client.0.smithi076.stdout:2/951: rename d1/d4/d64/fc7 to d1/d4/f146 0 2024-03-23T15:44:59.889 INFO:tasks.workunit.client.0.smithi076.stdout:1/689: dread d0/d2/d91/f61 [0,4194304] 0 2024-03-23T15:44:59.889 INFO:tasks.workunit.client.0.smithi076.stdout:4/933: unlink d8/d9/d1d/f4f 0 2024-03-23T15:44:59.889 INFO:tasks.workunit.client.0.smithi076.stdout:4/934: write d8/d3a/d18/d1b/f96 [1173869,94151] 0 2024-03-23T15:44:59.891 INFO:tasks.workunit.client.0.smithi076.stdout:5/954: rename d1a/d33/c6a to d1a/d33/d3a/d67/da8/dc4/d5f/c13f 0 2024-03-23T15:44:59.891 INFO:tasks.workunit.client.0.smithi076.stdout:1/690: truncate d0/d2/f75 1929217 0 2024-03-23T15:44:59.891 INFO:tasks.workunit.client.0.smithi076.stdout:4/935: symlink d8/d3a/dab/l128 0 2024-03-23T15:44:59.891 INFO:tasks.workunit.client.0.smithi076.stdout:4/936: chown d8/d9/l2a 21 1 2024-03-23T15:44:59.891 INFO:tasks.workunit.client.0.smithi076.stdout:2/952: write d1/d4/d64/d1b/f44 [1286037,66479] 0 2024-03-23T15:44:59.892 INFO:tasks.workunit.client.0.smithi076.stdout:5/955: getdents d1a/d33/d3a/d67/da8/dc4/d119 0 2024-03-23T15:44:59.893 INFO:tasks.workunit.client.0.smithi076.stdout:0/930: unlink d2/de/d10/d12/c4d 0 2024-03-23T15:44:59.894 INFO:tasks.workunit.client.0.smithi076.stdout:5/956: fdatasync d1a/f125 0 2024-03-23T15:44:59.894 INFO:tasks.workunit.client.0.smithi076.stdout:1/691: getdents d0/d2/d70/d7a 0 2024-03-23T15:44:59.894 INFO:tasks.workunit.client.0.smithi076.stdout:1/692: write d0/d35/d40/d47/d17/f1d [3545889,68956] 0 2024-03-23T15:44:59.894 INFO:tasks.workunit.client.0.smithi076.stdout:1/693: chown d0/d2/d70/da1/dbf/l24 1 1 2024-03-23T15:44:59.896 INFO:tasks.workunit.client.0.smithi076.stdout:5/957: unlink d1a/d33/d3a/d67/da8/dc4/le9 0 2024-03-23T15:44:59.899 INFO:tasks.workunit.client.0.smithi076.stdout:4/937: truncate d8/f71 4946737 0 2024-03-23T15:44:59.902 INFO:tasks.workunit.client.0.smithi076.stdout:0/931: mkdir d2/d1b/d112 0 2024-03-23T15:44:59.902 INFO:tasks.workunit.client.0.smithi076.stdout:0/932: stat d2/d8/d2d/l5a 0 2024-03-23T15:44:59.904 INFO:tasks.workunit.client.0.smithi076.stdout:1/694: creat d0/d3c/d6e/fe5 x:0 0 0 2024-03-23T15:44:59.904 INFO:tasks.workunit.client.0.smithi076.stdout:1/695: chown d0/d35/d40/d47/d74/d83/ca3 2399872 1 2024-03-23T15:44:59.904 INFO:tasks.workunit.client.0.smithi076.stdout:5/958: unlink d1a/d33/d3a/d67/da8/dc4/d65/f70 0 2024-03-23T15:44:59.907 INFO:tasks.workunit.client.0.smithi076.stdout:4/938: link d8/d3a/f48 d8/de3/f129 0 2024-03-23T15:44:59.907 INFO:tasks.workunit.client.0.smithi076.stdout:5/959: unlink d1a/d33/c46 0 2024-03-23T15:44:59.909 INFO:tasks.workunit.client.0.smithi076.stdout:0/933: truncate d2/de/d10/d3a/f5c 2073551 0 2024-03-23T15:44:59.910 INFO:tasks.workunit.client.0.smithi076.stdout:5/960: dread d1a/d33/d36/f3d [0,4194304] 0 2024-03-23T15:44:59.911 INFO:tasks.workunit.client.0.smithi076.stdout:0/934: dread d2/d8/d32/f4a [0,4194304] 0 2024-03-23T15:44:59.912 INFO:tasks.workunit.client.0.smithi076.stdout:3/694: dwrite d5/d35/d6d/d7f/fae [0,4194304] 0 2024-03-23T15:44:59.912 INFO:tasks.workunit.client.0.smithi076.stdout:3/695: chown d5/d17/d1c/d32/d39/f3e 63043 1 2024-03-23T15:44:59.912 INFO:tasks.workunit.client.0.smithi076.stdout:5/961: dread d1a/d33/fe6 [0,4194304] 0 2024-03-23T15:44:59.912 INFO:tasks.workunit.client.0.smithi076.stdout:1/696: creat d0/d2/d62/fe6 x:0 0 0 2024-03-23T15:44:59.913 INFO:tasks.workunit.client.0.smithi076.stdout:1/697: readlink d0/d35/d40/d47/l93 0 2024-03-23T15:44:59.914 INFO:tasks.workunit.client.0.smithi076.stdout:3/696: creat d5/d17/d1c/d32/d39/d59/d79/ddc/fee x:0 0 0 2024-03-23T15:44:59.914 INFO:tasks.workunit.client.0.smithi076.stdout:5/962: creat d1a/d2c/d106/f140 x:0 0 0 2024-03-23T15:44:59.914 INFO:tasks.workunit.client.0.smithi076.stdout:5/963: stat d1a/d33/d3a/d80/fd8 0 2024-03-23T15:44:59.915 INFO:tasks.workunit.client.0.smithi076.stdout:4/939: rename d8/d9/d1d/lea to d8/d41/d97/l12a 0 2024-03-23T15:44:59.919 INFO:tasks.workunit.client.0.smithi076.stdout:9/993: dread d1/d1f/f10b [0,4194304] 0 2024-03-23T15:44:59.920 INFO:tasks.workunit.client.0.smithi076.stdout:8/833: dwrite f7 [4194304,4194304] 0 2024-03-23T15:44:59.920 INFO:tasks.workunit.client.0.smithi076.stdout:8/834: truncate d8/f5a 5125127 0 2024-03-23T15:44:59.921 INFO:tasks.workunit.client.0.smithi076.stdout:3/697: creat d5/d35/d6d/d7f/fef x:0 0 0 2024-03-23T15:44:59.925 INFO:tasks.workunit.client.0.smithi076.stdout:2/953: link d1/d4/d52/d6e/lae d1/d2e/dd1/ddc/d139/l147 0 2024-03-23T15:44:59.925 INFO:tasks.workunit.client.0.smithi076.stdout:6/694: dwrite d1/d4/d10/f4a [0,4194304] 0 2024-03-23T15:44:59.927 INFO:tasks.workunit.client.0.smithi076.stdout:2/954: dread d1/fd9 [0,4194304] 0 2024-03-23T15:44:59.927 INFO:tasks.workunit.client.0.smithi076.stdout:0/935: mkdir d2/de/d113 0 2024-03-23T15:44:59.927 INFO:tasks.workunit.client.0.smithi076.stdout:0/936: fsync d2/de/d10/d3a/d47/d69/da3/fe4 0 2024-03-23T15:44:59.927 INFO:tasks.workunit.client.0.smithi076.stdout:2/955: write d1/d4/d52/f144 [347551,104460] 0 2024-03-23T15:44:59.928 INFO:tasks.workunit.client.0.smithi076.stdout:2/956: symlink d1/d4/db/d3e/d10c/l148 0 2024-03-23T15:44:59.931 INFO:tasks.workunit.client.0.smithi076.stdout:3/698: creat d5/dc2/ff0 x:0 0 0 2024-03-23T15:44:59.932 INFO:tasks.workunit.client.0.smithi076.stdout:2/957: link d1/d4/d64/d82/d103/cdd d1/d4/d52/ded/c149 0 2024-03-23T15:44:59.933 INFO:tasks.workunit.client.0.smithi076.stdout:5/964: dwrite d1a/d33/f51 [0,4194304] 0 2024-03-23T15:44:59.933 INFO:tasks.workunit.client.0.smithi076.stdout:4/940: link d8/d3a/d3b/c2f d8/d3a/d69/d73/df4/dd2/c12b 0 2024-03-23T15:44:59.935 INFO:tasks.workunit.client.0.smithi076.stdout:5/965: mknod d1a/d33/dc3/c141 0 2024-03-23T15:44:59.935 INFO:tasks.workunit.client.0.smithi076.stdout:5/966: chown d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/f107 1 1 2024-03-23T15:44:59.935 INFO:tasks.workunit.client.0.smithi076.stdout:5/967: read - d1a/d33/d3a/d67/d84/d12a/f13b zero size 2024-03-23T15:44:59.935 INFO:tasks.workunit.client.0.smithi076.stdout:0/937: rename d2/de/d10/l66 to d2/d1b/d77/l114 0 2024-03-23T15:44:59.938 INFO:tasks.workunit.client.0.smithi076.stdout:1/698: dwrite d0/d35/d82/f89 [0,4194304] 0 2024-03-23T15:44:59.938 INFO:tasks.workunit.client.0.smithi076.stdout:6/695: dread d1/d4/d10/f30 [0,4194304] 0 2024-03-23T15:44:59.938 INFO:tasks.workunit.client.0.smithi076.stdout:1/699: chown d0/d2/d70/d7a/d8b/fbe 2044003179 1 2024-03-23T15:44:59.938 INFO:tasks.workunit.client.0.smithi076.stdout:6/696: write d1/dd/df/d9b/db3/d72/fa6 [657315,63375] 0 2024-03-23T15:44:59.938 INFO:tasks.workunit.client.0.smithi076.stdout:8/835: dwrite d8/d4f/d80/da3/fb4 [0,4194304] 0 2024-03-23T15:44:59.938 INFO:tasks.workunit.client.0.smithi076.stdout:5/968: mkdir d1a/d33/d3a/d67/d84/d142 0 2024-03-23T15:44:59.942 INFO:tasks.workunit.client.0.smithi076.stdout:2/958: write d1/d4/d64/d1d/fe5 [928712,80102] 0 2024-03-23T15:44:59.942 INFO:tasks.workunit.client.0.smithi076.stdout:0/938: mkdir d2/d8/d2d/d115 0 2024-03-23T15:44:59.943 INFO:tasks.workunit.client.0.smithi076.stdout:1/700: creat d0/d2/d62/fe7 x:0 0 0 2024-03-23T15:44:59.944 INFO:tasks.workunit.client.0.smithi076.stdout:4/941: dread d8/d3a/d18/d52/f79 [0,4194304] 0 2024-03-23T15:44:59.944 INFO:tasks.workunit.client.0.smithi076.stdout:3/699: truncate d5/d35/d3c/d67/fc9 1918525 0 2024-03-23T15:44:59.944 INFO:tasks.workunit.client.0.smithi076.stdout:2/959: mkdir d1/d2e/d8a/d14a 0 2024-03-23T15:44:59.946 INFO:tasks.workunit.client.0.smithi076.stdout:2/960: link d1/d4/db/f119 d1/d4/d64/d82/d103/d79/dac/dd8/f14b 0 2024-03-23T15:44:59.947 INFO:tasks.workunit.client.0.smithi076.stdout:9/994: truncate d1/f1a 11875560 0 2024-03-23T15:44:59.948 INFO:tasks.workunit.client.0.smithi076.stdout:2/961: creat d1/d4/db/d15/d29/d7d/dbd/f14c x:0 0 0 2024-03-23T15:44:59.950 INFO:tasks.workunit.client.0.smithi076.stdout:2/962: creat d1/d4/db/d3e/f14d x:0 0 0 2024-03-23T15:44:59.952 INFO:tasks.workunit.client.0.smithi076.stdout:3/700: symlink d5/d17/d1c/d32/d8e/lf1 0 2024-03-23T15:44:59.953 INFO:tasks.workunit.client.0.smithi076.stdout:2/963: creat d1/d4/d64/d1b/dd4/f14e x:0 0 0 2024-03-23T15:44:59.953 INFO:tasks.workunit.client.0.smithi076.stdout:2/964: dread - d1/d4/d64/d7c/f107 zero size 2024-03-23T15:44:59.953 INFO:tasks.workunit.client.0.smithi076.stdout:5/969: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/fba [0,4194304] 0 2024-03-23T15:44:59.954 INFO:tasks.workunit.client.0.smithi076.stdout:5/970: dread - d1a/d4b/ff4 zero size 2024-03-23T15:44:59.958 INFO:tasks.workunit.client.0.smithi076.stdout:5/971: link d1a/d33/d3a/d67/da8/dc4/d5f/d8f/l12c d1a/d33/d3a/d67/l143 0 2024-03-23T15:44:59.958 INFO:tasks.workunit.client.0.smithi076.stdout:5/972: getdents d1a/d33/d3a/d67/da8 0 2024-03-23T15:44:59.958 INFO:tasks.workunit.client.0.smithi076.stdout:5/973: write d1a/d33/dc3/f114 [628520,76341] 0 2024-03-23T15:44:59.959 INFO:tasks.workunit.client.0.smithi076.stdout:2/965: dread d1/d4/d64/f23 [0,4194304] 0 2024-03-23T15:44:59.964 INFO:tasks.workunit.client.0.smithi076.stdout:4/942: dwrite d8/d9/f5a [0,4194304] 0 2024-03-23T15:44:59.964 INFO:tasks.workunit.client.0.smithi076.stdout:4/943: truncate d8/d3a/d69/d73/d9a/fef 239955 0 2024-03-23T15:44:59.967 INFO:tasks.workunit.client.0.smithi076.stdout:2/966: getdents d1/d4/db/d3e 0 2024-03-23T15:44:59.967 INFO:tasks.workunit.client.0.smithi076.stdout:4/944: unlink d8/d9/fc 0 2024-03-23T15:44:59.967 INFO:tasks.workunit.client.0.smithi076.stdout:4/945: write d8/d3a/d69/f107 [283999,86268] 0 2024-03-23T15:44:59.968 INFO:tasks.workunit.client.0.smithi076.stdout:2/967: symlink d1/d2e/dd1/ddc/d139/l14f 0 2024-03-23T15:44:59.973 INFO:tasks.workunit.client.0.smithi076.stdout:5/974: dwrite d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/fec [0,4194304] 0 2024-03-23T15:44:59.974 INFO:tasks.workunit.client.0.smithi076.stdout:2/968: dread d1/d4/d64/d82/ffa [0,4194304] 0 2024-03-23T15:44:59.974 INFO:tasks.workunit.client.0.smithi076.stdout:2/969: chown d1/d2e/dd1/ddc/c132 238143185 1 2024-03-23T15:44:59.974 INFO:tasks.workunit.client.0.smithi076.stdout:0/939: link d2/lc d2/d8/d32/l116 0 2024-03-23T15:44:59.974 INFO:tasks.workunit.client.0.smithi076.stdout:0/940: truncate d2/de/d10/fb0 349015 0 2024-03-23T15:44:59.975 INFO:tasks.workunit.client.0.smithi076.stdout:2/970: truncate d1/d2e/f42 1975667 0 2024-03-23T15:44:59.976 INFO:tasks.workunit.client.0.smithi076.stdout:6/697: rename d1/dd/df/d9b/db3/d72 to d1/dd/df/d5b/d62/df7 0 2024-03-23T15:44:59.977 INFO:tasks.workunit.client.0.smithi076.stdout:0/941: mkdir d2/de/d10/d3a/d47/de3/d117 0 2024-03-23T15:44:59.977 INFO:tasks.workunit.client.0.smithi076.stdout:0/942: stat d2/d1b/fc9 0 2024-03-23T15:44:59.977 INFO:tasks.workunit.client.0.smithi076.stdout:0/943: chown d2/d1b/fc9 62678 1 2024-03-23T15:44:59.989 INFO:tasks.workunit.client.0.smithi076.stdout:5/975: write d1a/d33/f35 [3756782,3042] 0 2024-03-23T15:44:59.989 INFO:tasks.workunit.client.0.smithi076.stdout:5/976: truncate d1a/d33/d3a/d67/da8/dc4/d65/f8c 942837 0 2024-03-23T15:44:59.989 INFO:tasks.workunit.client.0.smithi076.stdout:4/946: dwrite d8/d3a/d69/d73/d76/f78 [0,4194304] 0 2024-03-23T15:44:59.989 INFO:tasks.workunit.client.0.smithi076.stdout:4/947: write d8/d3a/d18/d1b/fa5 [1163175,100029] 0 2024-03-23T15:44:59.994 INFO:tasks.workunit.client.0.smithi076.stdout:2/971: dread d1/d4/f1e [0,4194304] 0 2024-03-23T15:44:59.994 INFO:tasks.workunit.client.0.smithi076.stdout:2/972: chown d1/d4/d64/d1d/fa8 1042639 1 2024-03-23T15:44:59.994 INFO:tasks.workunit.client.0.smithi076.stdout:2/973: mkdir d1/d2e/d150 0 2024-03-23T15:44:59.994 INFO:tasks.workunit.client.0.smithi076.stdout:6/698: read d1/d2/f87 [2772460,109911] 0 2024-03-23T15:44:59.995 INFO:tasks.workunit.client.0.smithi076.stdout:5/977: write d1a/d4b/fb8 [3915992,128585] 0 2024-03-23T15:44:59.996 INFO:tasks.workunit.client.0.smithi076.stdout:6/699: creat d1/dd/df/dac/ff8 x:0 0 0 2024-03-23T15:44:59.996 INFO:tasks.workunit.client.0.smithi076.stdout:6/700: write d1/d81/f9a [751939,5643] 0 2024-03-23T15:44:59.996 INFO:tasks.workunit.client.0.smithi076.stdout:2/974: dread d1/d4/d64/f48 [0,4194304] 0 2024-03-23T15:44:59.998 INFO:tasks.workunit.client.0.smithi076.stdout:5/978: dread d1a/d2c/fce [0,4194304] 0 2024-03-23T15:44:59.999 INFO:tasks.workunit.client.0.smithi076.stdout:5/979: mkdir d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d144 0 2024-03-23T15:45:00.001 INFO:tasks.workunit.client.0.smithi076.stdout:0/944: dwrite d2/de/d10/d3a/d47/d69/faa [0,4194304] 0 2024-03-23T15:45:00.001 INFO:tasks.workunit.client.0.smithi076.stdout:5/980: unlink d1a/d33/d3a/d67/da8/dc4/f52 0 2024-03-23T15:45:00.002 INFO:tasks.workunit.client.0.smithi076.stdout:2/975: rmdir d1/d2e 39 2024-03-23T15:45:00.004 INFO:tasks.workunit.client.0.smithi076.stdout:4/948: dwrite d8/d3a/f74 [0,4194304] 0 2024-03-23T15:45:00.006 INFO:tasks.workunit.client.0.smithi076.stdout:0/945: creat d2/de/d113/f118 x:0 0 0 2024-03-23T15:45:00.007 INFO:tasks.workunit.client.0.smithi076.stdout:1/701: rename d0/d35/d40/d47/d74/fde to d0/d2/d91/d86/fe8 0 2024-03-23T15:45:00.009 INFO:tasks.workunit.client.0.smithi076.stdout:0/946: symlink d2/de/d10/d43/d56/l119 0 2024-03-23T15:45:00.011 INFO:tasks.workunit.client.0.smithi076.stdout:0/947: symlink d2/de/d10/d3a/d10b/l11a 0 2024-03-23T15:45:00.011 INFO:tasks.workunit.client.0.smithi076.stdout:0/948: dread - d2/ff2 zero size 2024-03-23T15:45:00.011 INFO:tasks.workunit.client.0.smithi076.stdout:0/949: creat d2/d8/d52/f11b x:0 0 0 2024-03-23T15:45:00.018 INFO:tasks.workunit.client.0.smithi076.stdout:2/976: dwrite d1/d4/db/d15/d29/d7d/f12d [0,4194304] 0 2024-03-23T15:45:00.019 INFO:tasks.workunit.client.0.smithi076.stdout:1/702: dwrite d0/d35/d40/d47/d17/f1d [0,4194304] 0 2024-03-23T15:45:00.019 INFO:tasks.workunit.client.0.smithi076.stdout:1/703: stat d0/d35/d40/d47/d17/cce 0 2024-03-23T15:45:00.019 INFO:tasks.workunit.client.0.smithi076.stdout:1/704: truncate d0/d2/d91/f81 1016286 0 2024-03-23T15:45:00.023 INFO:tasks.workunit.client.0.smithi076.stdout:2/977: unlink d1/d4/db/d15/d29/f8f 0 2024-03-23T15:45:00.027 INFO:tasks.workunit.client.0.smithi076.stdout:0/950: link d2/d8/d32/f48 d2/d1b/d77/f11c 0 2024-03-23T15:45:00.029 INFO:tasks.workunit.client.0.smithi076.stdout:0/951: creat d2/d9a/f11d x:0 0 0 2024-03-23T15:45:00.030 INFO:tasks.workunit.client.0.smithi076.stdout:0/952: creat d2/de/d10/d3a/f11e x:0 0 0 2024-03-23T15:45:00.031 INFO:tasks.workunit.client.0.smithi076.stdout:0/953: dread d2/d8/f62 [0,4194304] 0 2024-03-23T15:45:00.033 INFO:tasks.workunit.client.0.smithi076.stdout:2/978: write d1/d2e/dd1/f102 [791020,57544] 0 2024-03-23T15:45:00.035 INFO:tasks.workunit.client.0.smithi076.stdout:9/995: rename d1/d20/c8c to d1/d2f/d3a/d11f/c142 0 2024-03-23T15:45:00.035 INFO:tasks.workunit.client.0.smithi076.stdout:9/996: stat d1/d20/f43 0 2024-03-23T15:45:00.035 INFO:tasks.workunit.client.0.smithi076.stdout:9/997: truncate d1/d1f/f30 9149211 0 2024-03-23T15:45:00.035 INFO:tasks.workunit.client.0.smithi076.stdout:9/998: stat d1/d1f/d29/d7f 0 2024-03-23T15:45:00.035 INFO:tasks.workunit.client.0.smithi076.stdout:9/999: truncate d1/d1f/d29/d7f/f111 5015136 0 2024-03-23T15:45:00.037 INFO:tasks.workunit.client.0.smithi076.stdout:3/701: rename d5/d17/d1c/d31/d47/l4d to d5/d35/d4c/d5a/db4/lf2 0 2024-03-23T15:45:00.038 INFO:tasks.workunit.client.0.smithi076.stdout:0/954: dread d2/d1b/d77/f11c [0,4194304] 0 2024-03-23T15:45:00.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/955: stat d2/d8/d2d/f9f 0 2024-03-23T15:45:00.039 INFO:tasks.workunit.client.0.smithi076.stdout:3/702: truncate d5/d17/d1c/d32/d39/f3a 1209274 0 2024-03-23T15:45:00.039 INFO:tasks.workunit.client.0.smithi076.stdout:3/703: write d5/f16 [264884,96815] 0 2024-03-23T15:45:00.039 INFO:tasks.workunit.client.0.smithi076.stdout:3/704: write d5/d17/d1c/f1e [688620,95355] 0 2024-03-23T15:45:00.039 INFO:tasks.workunit.client.0.smithi076.stdout:0/956: mknod d2/de/d10/d43/df3/c11f 0 2024-03-23T15:45:00.041 INFO:tasks.workunit.client.0.smithi076.stdout:0/957: mknod d2/de/d10/d3a/d47/d69/d84/c120 0 2024-03-23T15:45:00.041 INFO:tasks.workunit.client.0.smithi076.stdout:2/979: write d1/d4/d64/d1b/f71 [1384712,127784] 0 2024-03-23T15:45:00.042 INFO:tasks.workunit.client.0.smithi076.stdout:2/980: mknod d1/d4/d52/d6e/c151 0 2024-03-23T15:45:00.043 INFO:tasks.workunit.client.0.smithi076.stdout:2/981: symlink d1/d2e/l152 0 2024-03-23T15:45:00.043 INFO:tasks.workunit.client.0.smithi076.stdout:2/982: write d1/d4/d64/d82/ffa [2799092,10933] 0 2024-03-23T15:45:00.044 INFO:tasks.workunit.client.0.smithi076.stdout:2/983: rmdir d1/d4/dfb 39 2024-03-23T15:45:00.045 INFO:tasks.workunit.client.0.smithi076.stdout:2/984: mkdir d1/d4/d64/d1d/dbc/de6/d153 0 2024-03-23T15:45:00.046 INFO:tasks.workunit.client.0.smithi076.stdout:2/985: mknod d1/d4/d64/d82/d103/d79/c154 0 2024-03-23T15:45:00.047 INFO:tasks.workunit.client.0.smithi076.stdout:2/986: creat d1/d4/d64/f155 x:0 0 0 2024-03-23T15:45:00.047 INFO:tasks.workunit.client.0.smithi076.stdout:2/987: fdatasync d1/d4/d64/d1b/fe1 0 2024-03-23T15:45:00.047 INFO:tasks.workunit.client.0.smithi076.stdout:2/988: chown d1/d2e/dd1/ddc/d139/l13f 42157552 1 2024-03-23T15:45:00.047 INFO:tasks.workunit.client.0.smithi076.stdout:2/989: write d1/d4/f54 [8360580,49067] 0 2024-03-23T15:45:00.048 INFO:tasks.workunit.client.0.smithi076.stdout:2/990: mkdir d1/d4/d64/d82/d7f/d156 0 2024-03-23T15:45:00.062 INFO:tasks.workunit.client.0.smithi076.stdout:6/701: rename d1/dd/df/f13 to d1/dd/df/d5b/d62/df7/ff9 0 2024-03-23T15:45:00.064 INFO:tasks.workunit.client.0.smithi076.stdout:5/981: rename d1a/d2c/cae to d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/df1/d110/c145 0 2024-03-23T15:45:00.064 INFO:tasks.workunit.client.0.smithi076.stdout:6/702: mknod d1/dd/df/d5b/dcd/dd0/cfa 0 2024-03-23T15:45:00.065 INFO:tasks.workunit.client.0.smithi076.stdout:5/982: chown d1a/d33/d3a/d67/d84/f8e 5393 1 2024-03-23T15:45:00.065 INFO:tasks.workunit.client.0.smithi076.stdout:5/983: fdatasync d1a/d4b/f13a 0 2024-03-23T15:45:00.066 INFO:tasks.workunit.client.0.smithi076.stdout:4/949: rename d8/d9/d32/c57 to d8/d3a/d69/de4/c12c 0 2024-03-23T15:45:00.066 INFO:tasks.workunit.client.0.smithi076.stdout:4/950: chown d8/d3a/dab/fb3 1772618 1 2024-03-23T15:45:00.067 INFO:tasks.workunit.client.0.smithi076.stdout:4/951: readlink d8/d3a/d18/d52/lc6 0 2024-03-23T15:45:00.067 INFO:tasks.workunit.client.0.smithi076.stdout:4/952: chown d8/d9/d32/c62 1350 1 2024-03-23T15:45:00.067 INFO:tasks.workunit.client.0.smithi076.stdout:6/703: symlink d1/dd/df/d5b/lfb 0 2024-03-23T15:45:00.069 INFO:tasks.workunit.client.0.smithi076.stdout:3/705: rename d5/d35/f6e to d5/d35/d3c/d67/ff3 0 2024-03-23T15:45:00.070 INFO:tasks.workunit.client.0.smithi076.stdout:5/984: read d1a/d33/d3a/f43 [3094704,87782] 0 2024-03-23T15:45:00.072 INFO:tasks.workunit.client.0.smithi076.stdout:3/706: symlink d5/d35/d4c/d5a/db4/lf4 0 2024-03-23T15:45:00.072 INFO:tasks.workunit.client.0.smithi076.stdout:3/707: read - d5/dc2/ff0 zero size 2024-03-23T15:45:00.072 INFO:tasks.workunit.client.0.smithi076.stdout:5/985: truncate d1a/d33/f73 433203 0 2024-03-23T15:45:00.073 INFO:tasks.workunit.client.0.smithi076.stdout:5/986: symlink d1a/d33/d3a/d67/da8/dc4/d65/d12e/l146 0 2024-03-23T15:45:00.074 INFO:tasks.workunit.client.0.smithi076.stdout:3/708: symlink d5/d17/dcf/lf5 0 2024-03-23T15:45:00.076 INFO:tasks.workunit.client.0.smithi076.stdout:3/709: mkdir d5/d35/d3c/d67/df6 0 2024-03-23T15:45:00.090 INFO:tasks.workunit.client.0.smithi076.stdout:5/987: dwrite d1a/f102 [0,4194304] 0 2024-03-23T15:45:00.090 INFO:tasks.workunit.client.0.smithi076.stdout:5/988: chown d1a/d33/d3a/d80/fd8 443913 1 2024-03-23T15:45:00.092 INFO:tasks.workunit.client.0.smithi076.stdout:0/958: rename d2/de/d10/d43/d56/fd7 to d2/d8/d2d/f121 0 2024-03-23T15:45:00.092 INFO:tasks.workunit.client.0.smithi076.stdout:5/989: creat d1a/d33/d3a/d67/da8/dc4/d5f/d8f/d96/dcf/f147 x:0 0 0 2024-03-23T15:45:00.094 INFO:tasks.workunit.client.0.smithi076.stdout:5/990: mkdir d1a/d33/d3a/d67/d148 0 2024-03-23T15:45:00.095 INFO:tasks.workunit.client.0.smithi076.stdout:5/991: mknod d1a/d33/d3a/d120/c149 0 2024-03-23T15:45:00.095 INFO:tasks.workunit.client.0.smithi076.stdout:5/992: truncate d1a/d2c/d106/f140 546284 0 2024-03-23T15:45:00.100 INFO:tasks.workunit.client.0.smithi076.stdout:3/710: dwrite d5/f56 [0,4194304] 0 2024-03-23T15:45:00.100 INFO:tasks.workunit.client.0.smithi076.stdout:0/959: write d2/de/f14 [1435715,47302] 0 2024-03-23T15:45:00.109 INFO:tasks.workunit.client.0.smithi076.stdout:2/991: rename d1/d4/db/d15/fc4 to d1/d4/d64/d1d/dbc/de6/f157 0 2024-03-23T15:45:00.110 INFO:tasks.workunit.client.0.smithi076.stdout:3/711: dread d5/d17/d1c/d32/d39/f65 [0,4194304] 0 2024-03-23T15:45:00.112 INFO:tasks.workunit.client.0.smithi076.stdout:2/992: dread d1/d4/d64/d1b/f44 [0,4194304] 0 2024-03-23T15:45:00.113 INFO:tasks.workunit.client.0.smithi076.stdout:4/953: rename d8/de3/fcb to d8/d3a/d69/d73/df4/da9/df5/f12d 0 2024-03-23T15:45:00.113 INFO:tasks.workunit.client.0.smithi076.stdout:4/954: chown d8/d3a/d69/d73/d76/db4/le2 9836109 1 2024-03-23T15:45:00.113 INFO:tasks.workunit.client.0.smithi076.stdout:2/993: mknod d1/d4/d52/ded/c158 0 2024-03-23T15:45:00.115 INFO:tasks.workunit.client.0.smithi076.stdout:6/704: rename d1/d4/cb7 to d1/d81/cfc 0 2024-03-23T15:45:00.115 INFO:tasks.workunit.client.0.smithi076.stdout:6/705: readlink d1/d2/laf 0 2024-03-23T15:45:00.115 INFO:tasks.workunit.client.0.smithi076.stdout:6/706: fsync d1/dd/df/d9b/db3/d8d/f9f 0 2024-03-23T15:45:00.116 INFO:tasks.workunit.client.0.smithi076.stdout:6/707: chown d1/dd/d2a/d23/f2d 590 1 2024-03-23T15:45:00.116 INFO:tasks.workunit.client.0.smithi076.stdout:4/955: dread d8/d9/d32/f5b [0,4194304] 0 2024-03-23T15:45:00.116 INFO:tasks.workunit.client.0.smithi076.stdout:4/956: mknod d8/d3a/d18/c12e 0 2024-03-23T15:45:00.117 INFO:tasks.workunit.client.0.smithi076.stdout:0/960: dwrite d2/d9a/fbc [0,4194304] 0 2024-03-23T15:45:00.117 INFO:tasks.workunit.client.0.smithi076.stdout:4/957: mkdir d8/d3a/d18/d12f 0 2024-03-23T15:45:00.128 INFO:tasks.workunit.client.0.smithi076.stdout:2/994: write d1/d4/f146 [2188501,43774] 0 2024-03-23T15:45:00.129 INFO:tasks.workunit.client.0.smithi076.stdout:2/995: creat d1/d2e/d8a/f159 x:0 0 0 2024-03-23T15:45:00.133 INFO:tasks.workunit.client.0.smithi076.stdout:4/958: write d8/de3/db0/d10c/fd6 [4199002,19686] 0 2024-03-23T15:45:00.133 INFO:tasks.workunit.client.0.smithi076.stdout:0/961: dwrite d2/d8/d52/f5f [0,4194304] 0 2024-03-23T15:45:00.135 INFO:tasks.workunit.client.0.smithi076.stdout:6/708: dwrite d1/f21 [0,4194304] 0 2024-03-23T15:45:00.135 INFO:tasks.workunit.client.0.smithi076.stdout:6/709: truncate d1/dd/df/fa4 1911620 0 2024-03-23T15:45:00.136 INFO:tasks.workunit.client.0.smithi076.stdout:6/710: mkdir d1/dd/d2a/dfd 0 2024-03-23T15:45:00.136 INFO:tasks.workunit.client.0.smithi076.stdout:0/962: rename d2/de/d10/d3a/d47/d69/fbe to d2/de/d10/dc6/d10f/f122 0 2024-03-23T15:45:00.138 INFO:tasks.workunit.client.0.smithi076.stdout:6/711: creat d1/dd/d2a/ffe x:0 0 0 2024-03-23T15:45:00.145 INFO:tasks.workunit.client.0.smithi076.stdout:6/712: write d1/dd/df/d5b/d62/df7/ff9 [6831444,33912] 0 2024-03-23T15:45:00.147 INFO:tasks.workunit.client.0.smithi076.stdout:4/959: dwrite d8/d3a/d69/d73/df4/f103 [0,4194304] 0 2024-03-23T15:45:00.149 INFO:tasks.workunit.client.0.smithi076.stdout:0/963: dwrite d2/de/f13 [0,4194304] 0 2024-03-23T15:45:00.151 INFO:tasks.workunit.client.0.smithi076.stdout:4/960: symlink d8/d3a/d18/d12f/l130 0 2024-03-23T15:45:00.153 INFO:tasks.workunit.client.0.smithi076.stdout:0/964: symlink d2/d1b/d112/l123 0 2024-03-23T15:45:00.153 INFO:tasks.workunit.client.0.smithi076.stdout:0/965: chown d2/de/d10/d3a/d47/d69 8813338 1 2024-03-23T15:45:00.153 INFO:tasks.workunit.client.0.smithi076.stdout:0/966: write d2/de/d10/d43/f79 [1805862,80513] 0 2024-03-23T15:45:00.156 INFO:tasks.workunit.client.0.smithi076.stdout:4/961: rename d8/d41/c9b to d8/d41/d97/c131 0 2024-03-23T15:45:00.156 INFO:tasks.workunit.client.0.smithi076.stdout:4/962: write d8/f3d [443133,24087] 0 2024-03-23T15:45:00.158 INFO:tasks.workunit.client.0.smithi076.stdout:6/713: dwrite d1/dd/df/fa4 [0,4194304] 0 2024-03-23T15:45:00.167 INFO:tasks.workunit.client.0.smithi076.stdout:4/963: write d8/d3a/f74 [1249340,14982] 0 2024-03-23T15:45:00.167 INFO:tasks.workunit.client.0.smithi076.stdout:4/964: chown d8/f6a 0 1 2024-03-23T15:45:00.169 INFO:tasks.workunit.client.0.smithi076.stdout:0/967: dwrite d2/de/d10/d3a/fce [0,4194304] 0 2024-03-23T15:45:00.174 INFO:tasks.workunit.client.0.smithi076.stdout:6/714: dread d1/dd/df/d9b/db3/fa1 [8388608,4194304] 0 2024-03-23T15:45:00.174 INFO:tasks.workunit.client.0.smithi076.stdout:6/715: chown d1/dd/d2a/d23/f3a 96613 1 2024-03-23T15:45:00.174 INFO:tasks.workunit.client.0.smithi076.stdout:0/968: rename d2/d8/d2d/d115 to d2/de/d10/df5/d124 0 2024-03-23T15:45:00.175 INFO:tasks.workunit.client.0.smithi076.stdout:0/969: chown d2/de/d10/d3a/d47/d69/le5 1000 0 2024-03-23T15:45:00.175 INFO:tasks.workunit.client.0.smithi076.stdout:8/836: sync 2024-03-23T15:45:00.175 INFO:tasks.workunit.client.0.smithi076.stdout:1/705: sync 2024-03-23T15:45:00.175 INFO:tasks.workunit.client.0.smithi076.stdout:1/706: chown d0/d35/d40/l73 175 1 2024-03-23T15:45:00.177 INFO:tasks.workunit.client.0.smithi076.stdout:8/837: creat d8/d2e/d30/d64/f11c x:0 0 0 2024-03-23T15:45:00.177 INFO:tasks.workunit.client.0.smithi076.stdout:8/838: fdatasync d8/d2e/d30/d69/f81 0 2024-03-23T15:45:00.179 INFO:tasks.workunit.client.0.smithi076.stdout:8/839: write d8/d2e/d30/fa2 [626538,7104] 0 2024-03-23T15:45:00.183 INFO:tasks.workunit.client.0.smithi076.stdout:1/707: dread d0/d35/f87 [0,4194304] 0 2024-03-23T15:45:00.183 INFO:tasks.workunit.client.0.smithi076.stdout:1/708: truncate d0/d2/d91/f81 1068157 0 2024-03-23T15:45:00.186 INFO:tasks.workunit.client.0.smithi076.stdout:1/709: unlink d0/d35/d40/c67 0 2024-03-23T15:45:00.187 INFO:tasks.workunit.client.0.smithi076.stdout:1/710: read d0/d35/d40/d47/d17/d20/fd4 [3007961,82258] 0 2024-03-23T15:45:00.187 INFO:tasks.workunit.client.0.smithi076.stdout:1/711: chown d0/d35/f60 30721 1 2024-03-23T15:45:00.188 INFO:tasks.workunit.client.0.smithi076.stdout:4/965: dwrite d8/d9/d32/f110 [0,4194304] 0 2024-03-23T15:45:00.188 INFO:tasks.workunit.client.0.smithi076.stdout:4/966: read d8/d3a/f34 [363861,40090] 0 2024-03-23T15:45:00.191 INFO:tasks.workunit.client.0.smithi076.stdout:8/840: rmdir d8/d2e/d30/d69 39 2024-03-23T15:45:00.191 INFO:tasks.workunit.client.0.smithi076.stdout:8/841: chown d8/dd0/df0/f112 38 1 2024-03-23T15:45:00.191 INFO:tasks.workunit.client.0.smithi076.stdout:6/716: dwrite d1/dd/df/dac/ff8 [0,4194304] 0 2024-03-23T15:45:00.192 INFO:tasks.workunit.client.0.smithi076.stdout:6/717: write d1/dd/df/f66 [2595927,124304] 0 2024-03-23T15:45:00.192 INFO:tasks.workunit.client.0.smithi076.stdout:6/718: chown d1/d2/dc8 2363 1 2024-03-23T15:45:00.195 INFO:tasks.workunit.client.0.smithi076.stdout:0/970: rmdir d2/d8/d32 39 2024-03-23T15:45:00.197 INFO:tasks.workunit.client.0.smithi076.stdout:8/842: getdents d8/d83 0 2024-03-23T15:45:00.198 INFO:tasks.workunit.client.0.smithi076.stdout:0/971: symlink d2/de/d10/dc6/d10f/l125 0 2024-03-23T15:45:00.198 INFO:tasks.workunit.client.0.smithi076.stdout:4/967: symlink d8/d3a/d3b/l132 0 2024-03-23T15:45:00.200 INFO:tasks.workunit.client.0.smithi076.stdout:4/968: unlink d8/d9/f12 0 2024-03-23T15:45:00.201 INFO:tasks.workunit.client.0.smithi076.stdout:0/972: mknod d2/de/d10/d12/da6/c126 0 2024-03-23T15:45:00.205 INFO:tasks.workunit.client.0.smithi076.stdout:0/973: rename d2/de/d10/ffe to d2/d1b/d112/f127 0 2024-03-23T15:45:00.205 INFO:tasks.workunit.client.0.smithi076.stdout:1/712: dwrite d0/d35/d40/d47/d74/d83/d8f/fb1 [0,4194304] 0 2024-03-23T15:45:00.205 INFO:tasks.workunit.client.0.smithi076.stdout:4/969: getdents d8/d80 0 2024-03-23T15:45:00.206 INFO:tasks.workunit.client.0.smithi076.stdout:0/974: rename d2/de/d10/d12/da6/c126 to d2/ded/c128 0 2024-03-23T15:45:00.207 INFO:tasks.workunit.client.0.smithi076.stdout:4/970: mknod d8/d3a/d69/de4/c133 0 2024-03-23T15:45:00.209 INFO:tasks.workunit.client.0.smithi076.stdout:4/971: mknod d8/d3a/dab/c134 0 2024-03-23T15:45:00.211 INFO:tasks.workunit.client.0.smithi076.stdout:0/975: link d2/de/d10/d3a/d47/l90 d2/de/d10/d3a/d47/d69/l129 0 2024-03-23T15:45:00.216 INFO:tasks.workunit.client.0.smithi076.stdout:0/976: link d2/de/d10/d12/f85 d2/de/d10/d3a/d47/d69/f12a 0 2024-03-23T15:45:00.218 INFO:tasks.workunit.client.0.smithi076.stdout:1/713: write d0/d35/f64 [3526831,118430] 0 2024-03-23T15:45:00.219 INFO:tasks.workunit.client.0.smithi076.stdout:1/714: truncate d0/d35/d40/d47/d8c/fc5 82719 0 2024-03-23T15:45:00.219 INFO:tasks.workunit.client.0.smithi076.stdout:1/715: creat d0/d35/d40/d47/d74/fe9 x:0 0 0 2024-03-23T15:45:00.219 INFO:tasks.workunit.client.0.smithi076.stdout:1/716: fsync d0/d35/f68 0 2024-03-23T15:45:00.219 INFO:tasks.workunit.client.0.smithi076.stdout:1/717: creat d0/d2/d70/d7a/fea x:0 0 0 2024-03-23T15:45:00.220 INFO:tasks.workunit.client.0.smithi076.stdout:1/718: getdents d0/d3c/d6e/dc4 0 2024-03-23T15:45:00.221 INFO:tasks.workunit.client.0.smithi076.stdout:1/719: write d0/d35/d40/d47/d17/d20/f58 [583523,92282] 0 2024-03-23T15:45:00.224 INFO:tasks.workunit.client.0.smithi076.stdout:6/719: dwrite d1/dd/df/d9b/db3/fa1 [4194304,4194304] 0 2024-03-23T15:45:00.224 INFO:tasks.workunit.client.0.smithi076.stdout:6/720: chown d1/dd/df/d5b/dcd/cd7 151 1 2024-03-23T15:45:00.224 INFO:tasks.workunit.client.0.smithi076.stdout:6/721: dread - d1/dd/df/d5b/d62/dc7/fd6 zero size 2024-03-23T15:45:00.224 INFO:tasks.workunit.client.0.smithi076.stdout:6/722: dread - d1/dd/df/d5b/f55 zero size 2024-03-23T15:45:00.224 INFO:tasks.workunit.client.0.smithi076.stdout:6/723: write d1/d4/d10/fe6 [857531,1643] 0 2024-03-23T15:45:00.225 INFO:tasks.workunit.client.0.smithi076.stdout:8/843: dwrite d8/d26/d27/f9e [4194304,4194304] 0 2024-03-23T15:45:00.225 INFO:tasks.workunit.client.0.smithi076.stdout:1/720: read d0/d35/f87 [2421902,58039] 0 2024-03-23T15:45:00.227 INFO:tasks.workunit.client.0.smithi076.stdout:8/844: symlink d8/d26/d27/l11d 0 2024-03-23T15:45:00.230 INFO:tasks.workunit.client.0.smithi076.stdout:1/721: symlink d0/d35/d40/d47/d8c/d99/leb 0 2024-03-23T15:45:00.233 INFO:tasks.workunit.client.0.smithi076.stdout:1/722: link d0/d35/f60 d0/d2/d70/da1/fec 0 2024-03-23T15:45:00.238 INFO:tasks.workunit.client.0.smithi076.stdout:1/723: unlink d0/d35/l63 0 2024-03-23T15:45:00.240 INFO:tasks.workunit.client.0.smithi076.stdout:8/845: dread d8/d4f/d80/da3/fdf [0,4194304] 0 2024-03-23T15:45:00.242 INFO:tasks.workunit.client.0.smithi076.stdout:1/724: symlink d0/d2/led 0 2024-03-23T15:45:00.243 INFO:tasks.workunit.client.0.smithi076.stdout:1/725: write d0/d35/d82/f89 [4318127,17490] 0 2024-03-23T15:45:00.244 INFO:tasks.workunit.client.0.smithi076.stdout:6/724: dwrite d1/d2/f71 [0,4194304] 0 2024-03-23T15:45:00.244 INFO:tasks.workunit.client.0.smithi076.stdout:6/725: readlink d1/d2/laf 0 2024-03-23T15:45:00.244 INFO:tasks.workunit.client.0.smithi076.stdout:6/726: truncate d1/d4/fc6 1087582 0 2024-03-23T15:45:00.245 INFO:tasks.workunit.client.0.smithi076.stdout:1/726: rename d0/d2/d70/d7a/cb0 to d0/d3c/d6e/dca/cee 0 2024-03-23T15:45:00.246 INFO:tasks.workunit.client.0.smithi076.stdout:6/727: link d1/dd/df/d9b/db3/d8d/f46 d1/d2/dc8/fff 0 2024-03-23T15:45:00.247 INFO:tasks.workunit.client.0.smithi076.stdout:8/846: dread d8/d4f/d80/da3/fb4 [0,4194304] 0 2024-03-23T15:45:00.247 INFO:tasks.workunit.client.0.smithi076.stdout:5/993: sync 2024-03-23T15:45:00.247 INFO:tasks.workunit.client.0.smithi076.stdout:5/994: fdatasync d1a/d33/d3a/f41 0 2024-03-23T15:45:00.249 INFO:tasks.workunit.client.0.smithi076.stdout:5/995: symlink d1a/d33/l14a 0 2024-03-23T15:45:00.250 INFO:tasks.workunit.client.0.smithi076.stdout:5/996: unlink d1a/d33/dc3/da6/fea 0 2024-03-23T15:45:00.250 INFO:tasks.workunit.client.0.smithi076.stdout:5/997: mknod d1a/d33/d3a/d67/da8/dc4/d65/d136/c14b 0 2024-03-23T15:45:00.251 INFO:tasks.workunit.client.0.smithi076.stdout:5/998: fdatasync d1a/f1b 0 2024-03-23T15:45:00.252 INFO:tasks.workunit.client.0.smithi076.stdout:5/999: creat d1a/d33/d3a/d67/da8/dc4/d65/d136/f14c x:0 0 0 2024-03-23T15:45:00.265 INFO:tasks.workunit.client.0.smithi076.stdout:8/847: dwrite d8/d26/fc5 [0,4194304] 0 2024-03-23T15:45:00.265 INFO:tasks.workunit.client.0.smithi076.stdout:8/848: read d8/d26/d27/d56/fba [582789,122144] 0 2024-03-23T15:45:00.268 INFO:tasks.workunit.client.0.smithi076.stdout:8/849: dread d8/d26/d33/d8c/fb5 [0,4194304] 0 2024-03-23T15:45:00.268 INFO:tasks.workunit.client.0.smithi076.stdout:8/850: chown d8/d26/d27/d56/fd6 789477 1 2024-03-23T15:45:00.269 INFO:tasks.workunit.client.0.smithi076.stdout:8/851: dread - d8/d2e/d30/d69/f81 zero size 2024-03-23T15:45:00.269 INFO:tasks.workunit.client.0.smithi076.stdout:8/852: stat d8/d6b/ff4 0 2024-03-23T15:45:00.269 INFO:tasks.workunit.client.0.smithi076.stdout:8/853: chown d8/f1d 2 1 2024-03-23T15:45:00.269 INFO:tasks.workunit.client.0.smithi076.stdout:8/854: chown d8/d26 50079 1 2024-03-23T15:45:00.285 INFO:tasks.workunit.client.0.smithi076.stdout:8/855: dwrite d8/fe [0,4194304] 0 2024-03-23T15:45:00.287 INFO:tasks.workunit.client.0.smithi076.stdout:8/856: symlink d8/d89/l11e 0 2024-03-23T15:45:00.291 INFO:tasks.workunit.client.0.smithi076.stdout:8/857: mkdir d8/d26/d27/d11f 0 2024-03-23T15:45:00.291 INFO:tasks.workunit.client.0.smithi076.stdout:8/858: fdatasync d8/f1a 0 2024-03-23T15:45:00.303 INFO:tasks.workunit.client.0.smithi076.stdout:8/859: dwrite d8/d4f/d80/db7/fbe [0,4194304] 0 2024-03-23T15:45:00.305 INFO:tasks.workunit.client.0.smithi076.stdout:8/860: rename d8/d6b/fdc to d8/dd0/de1/f120 0 2024-03-23T15:45:00.305 INFO:tasks.workunit.client.0.smithi076.stdout:8/861: readlink d8/d26/d33/l42 0 2024-03-23T15:45:00.309 INFO:tasks.workunit.client.0.smithi076.stdout:3/712: sync 2024-03-23T15:45:00.309 INFO:tasks.workunit.client.0.smithi076.stdout:2/996: sync 2024-03-23T15:45:00.312 INFO:tasks.workunit.client.0.smithi076.stdout:3/713: dread d5/d35/d4c/d5a/d9d/fed [0,4194304] 0 2024-03-23T15:45:00.318 INFO:tasks.workunit.client.0.smithi076.stdout:8/862: dwrite d8/f23 [0,4194304] 0 2024-03-23T15:45:00.322 INFO:tasks.workunit.client.0.smithi076.stdout:8/863: rmdir d8/d26/d33/d101 39 2024-03-23T15:45:00.324 INFO:tasks.workunit.client.0.smithi076.stdout:2/997: rename d1/d2e/dd1/ddc to d1/dbf/d15a 0 2024-03-23T15:45:00.327 INFO:tasks.workunit.client.0.smithi076.stdout:3/714: dwrite d5/d17/d1c/d32/d4a/feb [0,4194304] 0 2024-03-23T15:45:00.327 INFO:tasks.workunit.client.0.smithi076.stdout:3/715: write d5/d35/d6d/d7f/fef [225069,73022] 0 2024-03-23T15:45:00.329 INFO:tasks.workunit.client.0.smithi076.stdout:3/716: unlink d5/d17/d1c/d32/d70/fc1 0 2024-03-23T15:45:00.330 INFO:tasks.workunit.client.0.smithi076.stdout:3/717: chown d5/d17/d1c/d32/l50 69862 1 2024-03-23T15:45:00.330 INFO:tasks.workunit.client.0.smithi076.stdout:3/718: chown d5/d17/d1c/d32/d8e/lf1 3 1 2024-03-23T15:45:00.334 INFO:tasks.workunit.client.0.smithi076.stdout:2/998: write d1/d4/f98 [2303843,120572] 0 2024-03-23T15:45:00.334 INFO:tasks.workunit.client.0.smithi076.stdout:2/999: readlink d1/d4/db/d15/la2 0 2024-03-23T15:45:00.336 INFO:tasks.workunit.client.0.smithi076.stdout:8/864: dwrite d8/d26/d33/d8c/fee [0,4194304] 0 2024-03-23T15:45:00.340 INFO:tasks.workunit.client.0.smithi076.stdout:8/865: mkdir d8/d4f/d80/da3/df7/d121 0 2024-03-23T15:45:00.340 INFO:tasks.workunit.client.0.smithi076.stdout:8/866: write d8/d2e/d30/d64/f11c [260856,101565] 0 2024-03-23T15:45:00.349 INFO:tasks.workunit.client.0.smithi076.stdout:3/719: dwrite d5/d17/d1c/d32/d39/f44 [0,4194304] 0 2024-03-23T15:45:00.349 INFO:tasks.workunit.client.0.smithi076.stdout:3/720: chown d5/d17/d1c/d32/d8e/la8 33252952 1 2024-03-23T15:45:00.349 INFO:tasks.workunit.client.0.smithi076.stdout:3/721: symlink d5/d17/dcf/de2/lf7 0 2024-03-23T15:45:00.352 INFO:tasks.workunit.client.0.smithi076.stdout:4/972: sync 2024-03-23T15:45:00.352 INFO:tasks.workunit.client.0.smithi076.stdout:4/973: chown d8/f42 28948 1 2024-03-23T15:45:00.354 INFO:tasks.workunit.client.0.smithi076.stdout:4/974: mknod d8/d3a/d69/d73/df4/d88/c135 0 2024-03-23T15:45:00.355 INFO:tasks.workunit.client.0.smithi076.stdout:4/975: mknod d8/de3/db0/c136 0 2024-03-23T15:45:00.359 INFO:tasks.workunit.client.0.smithi076.stdout:4/976: symlink d8/l137 0 2024-03-23T15:45:00.359 INFO:tasks.workunit.client.0.smithi076.stdout:4/977: creat d8/d3a/d18/d1b/f138 x:0 0 0 2024-03-23T15:45:00.360 INFO:tasks.workunit.client.0.smithi076.stdout:4/978: unlink d8/de3/db0/d10c/f10d 0 2024-03-23T15:45:00.360 INFO:tasks.workunit.client.0.smithi076.stdout:4/979: fsync d8/d3a/d69/d73/d76/f7a 0 2024-03-23T15:45:00.362 INFO:tasks.workunit.client.0.smithi076.stdout:4/980: mkdir d8/d3a/d3b/d139 0 2024-03-23T15:45:00.363 INFO:tasks.workunit.client.0.smithi076.stdout:4/981: dread d8/fae [0,4194304] 0 2024-03-23T15:45:00.365 INFO:tasks.workunit.client.0.smithi076.stdout:4/982: truncate d8/d3a/d3b/f39 5044645 0 2024-03-23T15:45:00.370 INFO:tasks.workunit.client.0.smithi076.stdout:3/722: link d5/d35/f36 d5/d17/d1c/d32/d4a/ff8 0 2024-03-23T15:45:00.382 INFO:tasks.workunit.client.0.smithi076.stdout:4/983: dwrite d8/de3/f129 [0,4194304] 0 2024-03-23T15:45:00.384 INFO:tasks.workunit.client.0.smithi076.stdout:4/984: creat d8/d9/d1d/f13a x:0 0 0 2024-03-23T15:45:00.384 INFO:tasks.workunit.client.0.smithi076.stdout:3/723: rename d5/d35/d3c/l4e to d5/d17/d1c/d32/d39/d59/lf9 0 2024-03-23T15:45:00.386 INFO:tasks.workunit.client.0.smithi076.stdout:4/985: dread d8/d3a/d18/f4d [0,4194304] 0 2024-03-23T15:45:00.386 INFO:tasks.workunit.client.0.smithi076.stdout:4/986: chown d8/d3a/d18/d1b 531851 1 2024-03-23T15:45:00.386 INFO:tasks.workunit.client.0.smithi076.stdout:3/724: mkdir d5/d17/d1c/d32/d70/dfa 0 2024-03-23T15:45:00.386 INFO:tasks.workunit.client.0.smithi076.stdout:4/987: getdents d8/d3a/d69/d73/df4/da9 0 2024-03-23T15:45:00.399 INFO:tasks.workunit.client.0.smithi076.stdout:4/988: dwrite d8/de3/fbd [0,4194304] 0 2024-03-23T15:45:00.400 INFO:tasks.workunit.client.0.smithi076.stdout:4/989: getdents d8/d3a 0 2024-03-23T15:45:00.401 INFO:tasks.workunit.client.0.smithi076.stdout:4/990: unlink d8/d3a/d18/d12f/l130 0 2024-03-23T15:45:00.402 INFO:tasks.workunit.client.0.smithi076.stdout:4/991: mknod d8/d3a/dab/c13b 0 2024-03-23T15:45:00.403 INFO:tasks.workunit.client.0.smithi076.stdout:4/992: dread - d8/d3a/d18/d75/fd8 zero size 2024-03-23T15:45:00.403 INFO:tasks.workunit.client.0.smithi076.stdout:4/993: write d8/de3/db0/de9/f115 [290682,26359] 0 2024-03-23T15:45:00.403 INFO:tasks.workunit.client.0.smithi076.stdout:4/994: creat d8/d3a/dab/dc3/f13c x:0 0 0 2024-03-23T15:45:00.403 INFO:tasks.workunit.client.0.smithi076.stdout:3/725: dwrite d5/f6 [0,4194304] 0 2024-03-23T15:45:00.404 INFO:tasks.workunit.client.0.smithi076.stdout:4/995: dread d8/d9/f5c [0,4194304] 0 2024-03-23T15:45:00.406 INFO:tasks.workunit.client.0.smithi076.stdout:3/726: creat d5/ffb x:0 0 0 2024-03-23T15:45:00.407 INFO:tasks.workunit.client.0.smithi076.stdout:4/996: symlink d8/d3a/d69/de4/l13d 0 2024-03-23T15:45:00.409 INFO:tasks.workunit.client.0.smithi076.stdout:3/727: creat d5/d35/d3c/d67/df6/ffc x:0 0 0 2024-03-23T15:45:00.409 INFO:tasks.workunit.client.0.smithi076.stdout:3/728: chown d5/d17/d1c/d32/d70/d8c 51899657 1 2024-03-23T15:45:00.412 INFO:tasks.workunit.client.0.smithi076.stdout:4/997: rename d8/de3/db0/de9/f10b to d8/d3a/d69/d73/d76/d11c/f13e 0 2024-03-23T15:45:00.413 INFO:tasks.workunit.client.0.smithi076.stdout:1/727: sync 2024-03-23T15:45:00.414 INFO:tasks.workunit.client.0.smithi076.stdout:6/728: sync 2024-03-23T15:45:00.414 INFO:tasks.workunit.client.0.smithi076.stdout:0/977: sync 2024-03-23T15:45:00.414 INFO:tasks.workunit.client.0.smithi076.stdout:1/728: dread - d0/d35/d40/d47/d55/f56 zero size 2024-03-23T15:45:00.414 INFO:tasks.workunit.client.0.smithi076.stdout:1/729: read d0/d2/d70/fb6 [18472,111475] 0 2024-03-23T15:45:00.414 INFO:tasks.workunit.client.0.smithi076.stdout:1/730: readlink d0/d35/d40/d47/d17/d20/l41 0 2024-03-23T15:45:00.415 INFO:tasks.workunit.client.0.smithi076.stdout:0/978: mkdir d2/d12b 0 2024-03-23T15:45:00.417 INFO:tasks.workunit.client.0.smithi076.stdout:0/979: mkdir d2/d1b/d112/d12c 0 2024-03-23T15:45:00.417 INFO:tasks.workunit.client.0.smithi076.stdout:0/980: chown d2/de/d10/d43/df3 22 1 2024-03-23T15:45:00.417 INFO:tasks.workunit.client.0.smithi076.stdout:0/981: chown d2/d8/d32/ce9 388 1 2024-03-23T15:45:00.418 INFO:tasks.workunit.client.0.smithi076.stdout:0/982: symlink d2/ded/df8/l12d 0 2024-03-23T15:45:00.423 INFO:tasks.workunit.client.0.smithi076.stdout:6/729: dread d1/dd/d2a/d23/f3a [4194304,4194304] 0 2024-03-23T15:45:00.423 INFO:tasks.workunit.client.0.smithi076.stdout:4/998: write d8/de3/f129 [3955499,67290] 0 2024-03-23T15:45:00.423 INFO:tasks.workunit.client.0.smithi076.stdout:4/999: dread - d8/d3a/dab/fb3 zero size 2024-03-23T15:45:00.426 INFO:tasks.workunit.client.0.smithi076.stdout:6/730: mknod d1/dd/df/d5b/d62/df7/c100 0 2024-03-23T15:45:00.431 INFO:tasks.workunit.client.0.smithi076.stdout:1/731: dwrite d0/d3c/d6e/fe5 [0,4194304] 0 2024-03-23T15:45:00.435 INFO:tasks.workunit.client.0.smithi076.stdout:1/732: truncate d0/d35/f87 3633990 0 2024-03-23T15:45:00.437 INFO:tasks.workunit.client.0.smithi076.stdout:0/983: dwrite d2/de/d10/d43/f63 [0,4194304] 0 2024-03-23T15:45:00.438 INFO:tasks.workunit.client.0.smithi076.stdout:0/984: creat d2/de/d10/d12/f12e x:0 0 0 2024-03-23T15:45:00.440 INFO:tasks.workunit.client.0.smithi076.stdout:0/985: creat d2/ded/f12f x:0 0 0 2024-03-23T15:45:00.442 INFO:tasks.workunit.client.0.smithi076.stdout:0/986: mkdir d2/de/d10/d130 0 2024-03-23T15:45:00.443 INFO:tasks.workunit.client.0.smithi076.stdout:1/733: write d0/d35/f68 [1194898,70403] 0 2024-03-23T15:45:00.444 INFO:tasks.workunit.client.0.smithi076.stdout:0/987: creat d2/de/d10/d3a/d47/d69/f131 x:0 0 0 2024-03-23T15:45:00.444 INFO:tasks.workunit.client.0.smithi076.stdout:0/988: chown d2/de/d10/d3a/d47/d69/ff4 1912 1 2024-03-23T15:45:00.444 INFO:tasks.workunit.client.0.smithi076.stdout:0/989: chown d2/d8/f28 2 1 2024-03-23T15:45:00.444 INFO:tasks.workunit.client.0.smithi076.stdout:0/990: write d2/de/d10/d12/da6/fd2 [4669316,121588] 0 2024-03-23T15:45:00.446 INFO:tasks.workunit.client.0.smithi076.stdout:0/991: rename d2/de/d10/c35 to d2/de/d10/c132 0 2024-03-23T15:45:00.448 INFO:tasks.workunit.client.0.smithi076.stdout:0/992: mkdir d2/de/d133 0 2024-03-23T15:45:00.448 INFO:tasks.workunit.client.0.smithi076.stdout:0/993: write d2/d8/f62 [951529,97069] 0 2024-03-23T15:45:00.449 INFO:tasks.workunit.client.0.smithi076.stdout:0/994: link d2/de/d10/d3a/fca d2/d1b/d112/f134 0 2024-03-23T15:45:00.450 INFO:tasks.workunit.client.0.smithi076.stdout:0/995: creat d2/de/d10/d3a/d47/d69/f135 x:0 0 0 2024-03-23T15:45:00.454 INFO:tasks.workunit.client.0.smithi076.stdout:0/996: rename d2/de/d10/d12/c21 to d2/de/d10/d43/df3/df7/d10a/c136 0 2024-03-23T15:45:00.454 INFO:tasks.workunit.client.0.smithi076.stdout:0/997: fsync d2/de/d10/f61 0 2024-03-23T15:45:00.455 INFO:tasks.workunit.client.0.smithi076.stdout:0/998: truncate d2/d1b/f9d 3437636 0 2024-03-23T15:45:00.455 INFO:tasks.workunit.client.0.smithi076.stdout:0/999: read d2/de/d10/d43/fb9 [212405,62673] 0 2024-03-23T15:45:00.457 INFO:tasks.workunit.client.0.smithi076.stdout:1/734: dwrite d0/d35/d40/f90 [0,4194304] 0 2024-03-23T15:45:00.457 INFO:tasks.workunit.client.0.smithi076.stdout:1/735: truncate d0/d3c/fa0 4840110 0 2024-03-23T15:45:00.457 INFO:tasks.workunit.client.0.smithi076.stdout:1/736: getdents d0/d2/d70/da1 0 2024-03-23T15:45:00.469 INFO:tasks.workunit.client.0.smithi076.stdout:1/737: dwrite d0/d35/d40/d47/d17/faf [0,4194304] 0 2024-03-23T15:45:00.471 INFO:tasks.workunit.client.0.smithi076.stdout:1/738: unlink d0/d3c/f57 0 2024-03-23T15:45:00.475 INFO:tasks.workunit.client.0.smithi076.stdout:1/739: dread d0/d3c/d6e/f80 [0,4194304] 0 2024-03-23T15:45:00.476 INFO:tasks.workunit.client.0.smithi076.stdout:1/740: write d0/d2/d62/fa7 [444821,124836] 0 2024-03-23T15:45:00.476 INFO:tasks.workunit.client.0.smithi076.stdout:1/741: readlink d0/d35/d40/d47/l5a 0 2024-03-23T15:45:00.477 INFO:tasks.workunit.client.0.smithi076.stdout:1/742: creat d0/d2/d70/d7a/d8b/dcb/fef x:0 0 0 2024-03-23T15:45:00.477 INFO:tasks.workunit.client.0.smithi076.stdout:1/743: dread - d0/d35/d40/d47/d55/fc6 zero size 2024-03-23T15:45:00.489 INFO:tasks.workunit.client.0.smithi076.stdout:1/744: dwrite d0/d35/d40/d47/d55/fd2 [0,4194304] 0 2024-03-23T15:45:00.491 INFO:tasks.workunit.client.0.smithi076.stdout:1/745: mknod d0/d3c/d6e/cf0 0 2024-03-23T15:45:00.497 INFO:tasks.workunit.client.0.smithi076.stdout:1/746: dread d0/d35/d40/d47/d17/f1d [4194304,4194304] 0 2024-03-23T15:45:00.513 INFO:tasks.workunit.client.0.smithi076.stdout:1/747: dwrite d0/d2/d70/da1/dbf/f43 [0,4194304] 0 2024-03-23T15:45:00.944 INFO:tasks.workunit.client.0.smithi076.stdout:3/729: sync 2024-03-23T15:45:00.944 INFO:tasks.workunit.client.0.smithi076.stdout:6/731: sync 2024-03-23T15:45:00.945 INFO:tasks.workunit.client.0.smithi076.stdout:1/748: sync 2024-03-23T15:45:00.945 INFO:tasks.workunit.client.0.smithi076.stdout:8/867: sync 2024-03-23T15:45:00.945 INFO:tasks.workunit.client.0.smithi076.stdout:8/868: fsync d8/d26/d33/d8c/fee 0 2024-03-23T15:45:00.945 INFO:tasks.workunit.client.0.smithi076.stdout:6/732: write d1/d4/d10/fe9 [478014,105042] 0 2024-03-23T15:45:00.945 INFO:tasks.workunit.client.0.smithi076.stdout:3/730: mknod d5/d35/d4c/cfd 0 2024-03-23T15:45:00.947 INFO:tasks.workunit.client.0.smithi076.stdout:8/869: rename d8/d26/c6d to d8/d26/c122 0 2024-03-23T15:45:00.949 INFO:tasks.workunit.client.0.smithi076.stdout:6/733: mknod d1/dd/df/d9b/db3/d8d/c101 0 2024-03-23T15:45:00.949 INFO:tasks.workunit.client.0.smithi076.stdout:6/734: chown d1/dd/df/fa7 7403 1 2024-03-23T15:45:00.950 INFO:tasks.workunit.client.0.smithi076.stdout:3/731: creat d5/d35/d4c/d5a/d9d/ffe x:0 0 0 2024-03-23T15:45:00.950 INFO:tasks.workunit.client.0.smithi076.stdout:3/732: truncate d5/d17/d1c/f1e 1280066 0 2024-03-23T15:45:00.951 INFO:tasks.workunit.client.0.smithi076.stdout:6/735: creat d1/dd/d2a/d23/f102 x:0 0 0 2024-03-23T15:45:00.953 INFO:tasks.workunit.client.0.smithi076.stdout:6/736: mkdir d1/d4/d103 0 2024-03-23T15:45:00.968 INFO:tasks.workunit.client.0.smithi076.stdout:1/749: dwrite d0/d2/d70/da1/dbf/fb2 [0,4194304] 0 2024-03-23T15:45:00.968 INFO:tasks.workunit.client.0.smithi076.stdout:8/870: dwrite d8/f1d [0,4194304] 0 2024-03-23T15:45:00.971 INFO:tasks.workunit.client.0.smithi076.stdout:8/871: mknod d8/c123 0 2024-03-23T15:45:00.971 INFO:tasks.workunit.client.0.smithi076.stdout:8/872: truncate d8/d6b/d9d/f104 860819 0 2024-03-23T15:45:00.973 INFO:tasks.workunit.client.0.smithi076.stdout:8/873: creat d8/d2e/d30/d69/dd3/d110/f124 x:0 0 0 2024-03-23T15:45:00.974 INFO:tasks.workunit.client.0.smithi076.stdout:3/733: dwrite d5/f6 [0,4194304] 0 2024-03-23T15:45:00.974 INFO:tasks.workunit.client.0.smithi076.stdout:8/874: dread d8/d89/db2/fc2 [0,4194304] 0 2024-03-23T15:45:00.978 INFO:tasks.workunit.client.0.smithi076.stdout:8/875: mkdir d8/d2e/d3e/deb/df1/d125 0 2024-03-23T15:45:00.978 INFO:tasks.workunit.client.0.smithi076.stdout:8/876: read - d8/d26/d33/fe8 zero size 2024-03-23T15:45:00.978 INFO:tasks.workunit.client.0.smithi076.stdout:1/750: write d0/d3c/f6c [587575,119628] 0 2024-03-23T15:45:00.979 INFO:tasks.workunit.client.0.smithi076.stdout:8/877: creat d8/d26/d33/ddd/f126 x:0 0 0 2024-03-23T15:45:00.980 INFO:tasks.workunit.client.0.smithi076.stdout:3/734: symlink d5/d35/d6d/lff 0 2024-03-23T15:45:00.980 INFO:tasks.workunit.client.0.smithi076.stdout:3/735: chown d5/d35/d4c/d5a/db4/lf2 111284948 1 2024-03-23T15:45:00.980 INFO:tasks.workunit.client.0.smithi076.stdout:3/736: fsync d5/d17/d1c/d32/d39/f3a 0 2024-03-23T15:45:00.981 INFO:tasks.workunit.client.0.smithi076.stdout:3/737: mkdir d5/d17/d1c/d32/d70/d100 0 2024-03-23T15:45:00.982 INFO:tasks.workunit.client.0.smithi076.stdout:8/878: rmdir d8/d2e/d30/d69/dd3/d110/d11a 0 2024-03-23T15:45:00.982 INFO:tasks.workunit.client.0.smithi076.stdout:8/879: chown d8/d2e/d3e/deb/df1 14471794 1 2024-03-23T15:45:00.983 INFO:tasks.workunit.client.0.smithi076.stdout:3/738: mknod d5/d17/d1c/d31/d47/c101 0 2024-03-23T15:45:00.983 INFO:tasks.workunit.client.0.smithi076.stdout:3/739: fdatasync d5/d17/d1c/d32/d70/d8c/fb3 0 2024-03-23T15:45:00.984 INFO:tasks.workunit.client.0.smithi076.stdout:1/751: write d0/d2/f48 [3160833,101311] 0 2024-03-23T15:45:00.998 INFO:tasks.workunit.client.0.smithi076.stdout:8/880: dwrite d8/d2e/d30/d69/faa [0,4194304] 0 2024-03-23T15:45:00.998 INFO:tasks.workunit.client.0.smithi076.stdout:8/881: chown d8/d26/d33/d8c/c113 3751 1 2024-03-23T15:45:00.998 INFO:tasks.workunit.client.0.smithi076.stdout:6/737: creat d1/dd/df/d9b/f104 x:0 0 0 2024-03-23T15:45:00.999 INFO:tasks.workunit.client.0.smithi076.stdout:3/740: dwrite d5/d17/d1c/d31/d61/faa [0,4194304] 0 2024-03-23T15:45:00.999 INFO:tasks.workunit.client.0.smithi076.stdout:3/741: dread - d5/d17/d1c/d32/d39/d59/d79/ddc/fee zero size 2024-03-23T15:45:01.000 INFO:tasks.workunit.client.0.smithi076.stdout:8/882: link d8/dd0/ce5 d8/d6b/d91/c127 0 2024-03-23T15:45:01.002 INFO:tasks.workunit.client.0.smithi076.stdout:8/883: dread - d8/d2e/d30/d64/f115 zero size 2024-03-23T15:45:01.002 INFO:tasks.workunit.client.0.smithi076.stdout:8/884: fdatasync d8/d26/d27/d56/f63 0 2024-03-23T15:45:01.004 INFO:tasks.workunit.client.0.smithi076.stdout:8/885: creat d8/d4f/d80/da3/df7/d121/f128 x:0 0 0 2024-03-23T15:45:01.009 INFO:tasks.workunit.client.0.smithi076.stdout:8/886: write d8/d2e/d3e/f49 [716349,5511] 0 2024-03-23T15:45:01.009 INFO:tasks.workunit.client.0.smithi076.stdout:6/738: unlink d1/dd/df/d5b/d62/df7/ff9 0 2024-03-23T15:45:01.010 INFO:tasks.workunit.client.0.smithi076.stdout:6/739: write d1/dd/df/d5b/f55 [889686,78600] 0 2024-03-23T15:45:01.010 INFO:tasks.workunit.client.0.smithi076.stdout:6/740: fdatasync d1/dd1/ff5 0 2024-03-23T15:45:01.010 INFO:tasks.workunit.client.0.smithi076.stdout:1/752: dwrite d0/d2/d70/da1/fec [0,4194304] 0 2024-03-23T15:45:01.012 INFO:tasks.workunit.client.0.smithi076.stdout:6/741: mknod d1/dd/df/d5b/c105 0 2024-03-23T15:45:01.012 INFO:tasks.workunit.client.0.smithi076.stdout:1/753: creat d0/d35/d40/d47/d8c/d99/ff1 x:0 0 0 2024-03-23T15:45:01.012 INFO:tasks.workunit.client.0.smithi076.stdout:1/754: chown d0/d35/d40/d47/d74/d83/d8f/lb4 2 1 2024-03-23T15:45:01.012 INFO:tasks.workunit.client.0.smithi076.stdout:1/755: dread - d0/d35/d40/d47/d74/d83/f98 zero size 2024-03-23T15:45:01.012 INFO:tasks.workunit.client.0.smithi076.stdout:8/887: mknod d8/d2e/c129 0 2024-03-23T15:45:01.012 INFO:tasks.workunit.client.0.smithi076.stdout:3/742: dwrite d5/d35/d4c/d5a/f82 [0,4194304] 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:6/742: mknod d1/dd/d2a/d23/c106 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:6/743: fsync d1/dd/df/d5b/d62/dc7/fe8 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:1/756: unlink d0/d2/d91/d86/cc1 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:1/757: write d0/d35/d40/d47/f4e [112948,81881] 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:1/758: truncate d0/d35/d40/d47/d8c/d99/ff1 365852 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:1/759: stat d0/f9d 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:1/760: dread - d0/d2/d70/d7a/d8b/fb7 zero size 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:6/744: creat d1/dd/df/d9b/db3/d8d/d60/f107 x:0 0 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:6/745: readlink d1/d81/la5 0 2024-03-23T15:45:01.015 INFO:tasks.workunit.client.0.smithi076.stdout:8/888: write d8/d26/d27/da9/fc9 [544708,6343] 0 2024-03-23T15:45:01.016 INFO:tasks.workunit.client.0.smithi076.stdout:3/743: creat d5/f102 x:0 0 0 2024-03-23T15:45:01.018 INFO:tasks.workunit.client.0.smithi076.stdout:3/744: chown d5/d35/d6d/d7f/c9a 9 1 2024-03-23T15:45:01.018 INFO:tasks.workunit.client.0.smithi076.stdout:3/745: chown d5/d17/d1c/d32/d39/d59/lf9 201598 1 2024-03-23T15:45:01.021 INFO:tasks.workunit.client.0.smithi076.stdout:8/889: write d8/d4f/d80/db7/fbe [3447778,34917] 0 2024-03-23T15:45:01.021 INFO:tasks.workunit.client.0.smithi076.stdout:8/890: write d8/d26/fc5 [4947569,71312] 0 2024-03-23T15:45:01.024 INFO:tasks.workunit.client.0.smithi076.stdout:8/891: symlink d8/d89/db2/dd9/l12a 0 2024-03-23T15:45:01.026 INFO:tasks.workunit.client.0.smithi076.stdout:8/892: symlink d8/d2e/l12b 0 2024-03-23T15:45:01.031 INFO:tasks.workunit.client.0.smithi076.stdout:3/746: dread d5/d17/d1c/d32/d4a/ff8 [0,4194304] 0 2024-03-23T15:45:01.031 INFO:tasks.workunit.client.0.smithi076.stdout:3/747: chown d5/d17/d1c/d31/d47/ca2 308 1 2024-03-23T15:45:01.031 INFO:tasks.workunit.client.0.smithi076.stdout:3/748: stat d5/d35/d4c 0 2024-03-23T15:45:01.036 INFO:tasks.workunit.client.0.smithi076.stdout:1/761: link d0/d35/d40/d47/d17/d20/l41 d0/d2/d70/d7a/d8b/dcb/lf2 0 2024-03-23T15:45:01.037 INFO:tasks.workunit.client.0.smithi076.stdout:6/746: dwrite d1/d2/f6d [0,4194304] 0 2024-03-23T15:45:01.037 INFO:tasks.workunit.client.0.smithi076.stdout:6/747: dread - d1/dd/d2a/ffe zero size 2024-03-23T15:45:01.038 INFO:tasks.workunit.client.0.smithi076.stdout:1/762: mknod d0/dc3/cf3 0 2024-03-23T15:45:01.038 INFO:tasks.workunit.client.0.smithi076.stdout:1/763: chown d0/d35/d40/d47/l5a 32828549 1 2024-03-23T15:45:01.039 INFO:tasks.workunit.client.0.smithi076.stdout:6/748: symlink d1/dd/df/dce/l108 0 2024-03-23T15:45:01.039 INFO:tasks.workunit.client.0.smithi076.stdout:1/764: creat d0/d3c/d6e/dc4/ff4 x:0 0 0 2024-03-23T15:45:01.040 INFO:tasks.workunit.client.0.smithi076.stdout:1/765: truncate d0/d3c/d6e/fe5 4721672 0 2024-03-23T15:45:01.041 INFO:tasks.workunit.client.0.smithi076.stdout:1/766: creat d0/d2/d70/ff5 x:0 0 0 2024-03-23T15:45:01.044 INFO:tasks.workunit.client.0.smithi076.stdout:1/767: unlink d0/d2/d91/l4f 0 2024-03-23T15:45:01.054 INFO:tasks.workunit.client.0.smithi076.stdout:8/893: rename d8/c9 to d8/d2e/d30/d69/c12c 0 2024-03-23T15:45:01.055 INFO:tasks.workunit.client.0.smithi076.stdout:3/749: link d5/d17/d1c/d31/d61/l77 d5/d17/d1c/d32/d39/d59/l103 0 2024-03-23T15:45:01.057 INFO:tasks.workunit.client.0.smithi076.stdout:1/768: dwrite d0/d35/f60 [4194304,4194304] 0 2024-03-23T15:45:01.058 INFO:tasks.workunit.client.0.smithi076.stdout:1/769: mknod d0/d35/d40/d47/d17/cf6 0 2024-03-23T15:45:01.059 INFO:tasks.workunit.client.0.smithi076.stdout:1/770: mknod d0/d35/d40/d47/d74/cf7 0 2024-03-23T15:45:01.063 INFO:tasks.workunit.client.0.smithi076.stdout:3/750: link d5/d17/d1c/d32/d4a/ff8 d5/d17/f104 0 2024-03-23T15:45:01.065 INFO:tasks.workunit.client.0.smithi076.stdout:1/771: dread d0/d3c/fa0 [0,4194304] 0 2024-03-23T15:45:01.067 INFO:tasks.workunit.client.0.smithi076.stdout:1/772: creat d0/d3c/d6e/dc4/ff8 x:0 0 0 2024-03-23T15:45:01.069 INFO:tasks.workunit.client.0.smithi076.stdout:1/773: unlink d0/d2/d62/f9e 0 2024-03-23T15:45:01.079 INFO:tasks.workunit.client.0.smithi076.stdout:8/894: dwrite d8/d26/d27/d56/f5b [4194304,4194304] 0 2024-03-23T15:45:01.083 INFO:tasks.workunit.client.0.smithi076.stdout:8/895: rmdir d8/dd0/df0 39 2024-03-23T15:45:01.083 INFO:tasks.workunit.client.0.smithi076.stdout:8/896: readlink d8/d26/d27/l8a 0 2024-03-23T15:45:01.083 INFO:tasks.workunit.client.0.smithi076.stdout:8/897: write d8/d2e/d3e/deb/fec [387465,13493] 0 2024-03-23T15:45:01.083 INFO:tasks.workunit.client.0.smithi076.stdout:8/898: write d8/d2e/d3e/f44 [5246933,10703] 0 2024-03-23T15:45:01.083 INFO:tasks.workunit.client.0.smithi076.stdout:8/899: chown d8/d26/c55 93 1 2024-03-23T15:45:01.084 INFO:tasks.workunit.client.0.smithi076.stdout:3/751: dwrite d5/d17/d1c/d32/d39/f44 [0,4194304] 0 2024-03-23T15:45:01.084 INFO:tasks.workunit.client.0.smithi076.stdout:3/752: fsync d5/d35/d3c/d67/ff3 0 2024-03-23T15:45:01.084 INFO:tasks.workunit.client.0.smithi076.stdout:8/900: read d8/f7a [199288,24038] 0 2024-03-23T15:45:01.086 INFO:tasks.workunit.client.0.smithi076.stdout:6/749: rename d1/dd/d2a/d23/f96 to d1/d4/f109 0 2024-03-23T15:45:01.088 INFO:tasks.workunit.client.0.smithi076.stdout:3/753: mknod d5/d17/dcf/c105 0 2024-03-23T15:45:01.088 INFO:tasks.workunit.client.0.smithi076.stdout:3/754: readlink d5/l23 0 2024-03-23T15:45:01.090 INFO:tasks.workunit.client.0.smithi076.stdout:6/750: mknod d1/d4/de2/c10a 0 2024-03-23T15:45:01.091 INFO:tasks.workunit.client.0.smithi076.stdout:3/755: rmdir d5/d35/d4c/d5a/db4 39 2024-03-23T15:45:01.091 INFO:tasks.workunit.client.0.smithi076.stdout:6/751: truncate d1/dd/df/fa7 2861874 0 2024-03-23T15:45:01.092 INFO:tasks.workunit.client.0.smithi076.stdout:6/752: getdents d1/dd/df/d9b/db3 0 2024-03-23T15:45:01.093 INFO:tasks.workunit.client.0.smithi076.stdout:3/756: symlink d5/d35/d4c/d5a/d9d/l106 0 2024-03-23T15:45:01.094 INFO:tasks.workunit.client.0.smithi076.stdout:3/757: dread - d5/d35/d6d/fd3 zero size 2024-03-23T15:45:01.098 INFO:tasks.workunit.client.0.smithi076.stdout:3/758: link d5/d17/d1c/d31/d61/l77 d5/d35/d4c/l107 0 2024-03-23T15:45:01.101 INFO:tasks.workunit.client.0.smithi076.stdout:3/759: unlink d5/d17/d1c/d32/d39/d59/lf9 0 2024-03-23T15:45:01.101 INFO:tasks.workunit.client.0.smithi076.stdout:3/760: chown d5/d17/d1c/d32/d8e/lc7 141915 1 2024-03-23T15:45:01.101 INFO:tasks.workunit.client.0.smithi076.stdout:3/761: stat d5/d17/dcf/de2 0 2024-03-23T15:45:01.101 INFO:tasks.workunit.client.0.smithi076.stdout:3/762: chown d5/d17/dcf/de2/de7 3 1 2024-03-23T15:45:01.106 INFO:tasks.workunit.client.0.smithi076.stdout:3/763: link d5/d17/d1c/f2b d5/d17/d1c/d32/d5d/f108 0 2024-03-23T15:45:01.107 INFO:tasks.workunit.client.0.smithi076.stdout:3/764: readlink d5/d17/d1c/d31/d61/d96/le6 0 2024-03-23T15:45:01.107 INFO:tasks.workunit.client.0.smithi076.stdout:3/765: truncate d5/f1f 871898 0 2024-03-23T15:45:01.107 INFO:tasks.workunit.client.0.smithi076.stdout:3/766: rmdir d5/d35/d6d 39 2024-03-23T15:45:01.109 INFO:tasks.workunit.client.0.smithi076.stdout:3/767: rename d5/f102 to d5/f109 0 2024-03-23T15:45:01.110 INFO:tasks.workunit.client.0.smithi076.stdout:3/768: truncate d5/f56 4840141 0 2024-03-23T15:45:01.126 INFO:tasks.workunit.client.0.smithi076.stdout:3/769: dwrite d5/d17/d1c/d32/d39/f44 [0,4194304] 0 2024-03-23T15:45:01.139 INFO:tasks.workunit.client.0.smithi076.stdout:8/901: rename d8/d26/c55 to d8/d89/db2/dd9/c12d 0 2024-03-23T15:45:01.142 INFO:tasks.workunit.client.0.smithi076.stdout:3/770: rename d5/d35/d4c/d5a/db4 to d5/d17/d1c/d31/d61/d92/d10a 0 2024-03-23T15:45:01.142 INFO:tasks.workunit.client.0.smithi076.stdout:3/771: readlink d5/d17/dcf/lf5 0 2024-03-23T15:45:01.145 INFO:tasks.workunit.client.0.smithi076.stdout:3/772: stat d5/d17/d1c/d32/d4a/ff8 0 2024-03-23T15:45:01.146 INFO:tasks.workunit.client.0.smithi076.stdout:3/773: fsync d5/d35/d6d/fbe 0 2024-03-23T15:45:01.148 INFO:tasks.workunit.client.0.smithi076.stdout:3/774: unlink d5/d17/d1c/d32/d39/f65 0 2024-03-23T15:45:01.150 INFO:tasks.workunit.client.0.smithi076.stdout:3/775: mknod d5/d17/d1c/d31/d61/d96/c10b 0 2024-03-23T15:45:01.150 INFO:tasks.workunit.client.0.smithi076.stdout:3/776: read - d5/d35/d6d/fda zero size 2024-03-23T15:45:01.159 INFO:tasks.workunit.client.0.smithi076.stdout:8/902: rename d8/d89/db2/dd9/l12a to d8/dd0/df0/l12e 0 2024-03-23T15:45:01.162 INFO:tasks.workunit.client.0.smithi076.stdout:3/777: link d5/d17/f29 d5/d17/d1c/d32/d39/d59/d79/ddc/f10c 0 2024-03-23T15:45:01.162 INFO:tasks.workunit.client.0.smithi076.stdout:3/778: mkdir d5/d17/d1c/d32/d70/d10d 0 2024-03-23T15:45:01.167 INFO:tasks.workunit.client.0.smithi076.stdout:8/903: write d8/dd0/f10b [1506254,6413] 0 2024-03-23T15:45:01.176 INFO:tasks.workunit.client.0.smithi076.stdout:8/904: dread d8/d4f/d80/da3/fdf [0,4194304] 0 2024-03-23T15:45:01.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/779: write d5/d17/d1c/d32/d4a/feb [2901863,113230] 0 2024-03-23T15:45:01.177 INFO:tasks.workunit.client.0.smithi076.stdout:8/905: mkdir d8/dd0/d12f 0 2024-03-23T15:45:01.177 INFO:tasks.workunit.client.0.smithi076.stdout:3/780: mkdir d5/d35/d10e 0 2024-03-23T15:45:01.178 INFO:tasks.workunit.client.0.smithi076.stdout:8/906: mknod d8/d2e/d3e/deb/df1/c130 0 2024-03-23T15:45:01.179 INFO:tasks.workunit.client.0.smithi076.stdout:8/907: dread d8/d26/d27/fa5 [0,4194304] 0 2024-03-23T15:45:01.180 INFO:tasks.workunit.client.0.smithi076.stdout:3/781: unlink d5/f109 0 2024-03-23T15:45:01.183 INFO:tasks.workunit.client.0.smithi076.stdout:8/908: getdents d8/d2e/d30/d69/dd3 0 2024-03-23T15:45:01.183 INFO:tasks.workunit.client.0.smithi076.stdout:8/909: truncate d8/d26/d33/fe8 386536 0 2024-03-23T15:45:01.184 INFO:tasks.workunit.client.0.smithi076.stdout:8/910: creat d8/d26/d33/f131 x:0 0 0 2024-03-23T15:45:01.187 INFO:tasks.workunit.client.0.smithi076.stdout:3/782: dread d5/f6 [0,4194304] 0 2024-03-23T15:45:01.198 INFO:tasks.workunit.client.0.smithi076.stdout:8/911: rename d8/d26/d33/d8c/fb5 to d8/d4f/d80/da3/f132 0 2024-03-23T15:45:01.199 INFO:tasks.workunit.client.0.smithi076.stdout:8/912: write d8/d26/d33/f131 [4849,55418] 0 2024-03-23T15:45:01.204 INFO:tasks.workunit.client.0.smithi076.stdout:3/783: dwrite d5/d17/d1c/f2b [0,4194304] 0 2024-03-23T15:45:01.221 INFO:tasks.workunit.client.0.smithi076.stdout:3/784: dwrite d5/d17/d1c/d32/d39/d59/d79/f8d [4194304,4194304] 0 2024-03-23T15:45:01.221 INFO:tasks.workunit.client.0.smithi076.stdout:3/785: fdatasync d5/d35/d4c/d5a/f41 0 2024-03-23T15:45:01.222 INFO:tasks.workunit.client.0.smithi076.stdout:3/786: write d5/d17/d1c/f1e [941649,105508] 0 2024-03-23T15:45:01.223 INFO:tasks.workunit.client.0.smithi076.stdout:3/787: creat d5/d17/dcf/f10f x:0 0 0 2024-03-23T15:45:01.223 INFO:tasks.workunit.client.0.smithi076.stdout:3/788: readlink d5/d17/d1c/d32/d8e/lc7 0 2024-03-23T15:45:01.227 INFO:tasks.workunit.client.0.smithi076.stdout:3/789: symlink d5/d17/d1c/d32/d39/l110 0 2024-03-23T15:45:01.247 INFO:tasks.workunit.client.0.smithi076.stdout:3/790: dwrite d5/d35/d3c/d67/fbc [0,4194304] 0 2024-03-23T15:45:01.248 INFO:tasks.workunit.client.0.smithi076.stdout:3/791: symlink d5/d35/d3c/d67/df6/l111 0 2024-03-23T15:45:01.248 INFO:tasks.workunit.client.0.smithi076.stdout:3/792: fdatasync d5/d17/d1c/d32/d4a/ff8 0 2024-03-23T15:45:01.249 INFO:tasks.workunit.client.0.smithi076.stdout:3/793: unlink d5/f7 0 2024-03-23T15:45:01.258 INFO:tasks.workunit.client.0.smithi076.stdout:3/794: link d5/d17/d1c/d32/d4a/f55 d5/d17/d1c/d32/d70/d100/f112 0 2024-03-23T15:45:01.258 INFO:tasks.workunit.client.0.smithi076.stdout:3/795: symlink d5/d17/dcf/de2/de7/l113 0 2024-03-23T15:45:01.261 INFO:tasks.workunit.client.0.smithi076.stdout:1/774: sync 2024-03-23T15:45:01.262 INFO:tasks.workunit.client.0.smithi076.stdout:6/753: sync 2024-03-23T15:45:01.262 INFO:tasks.workunit.client.0.smithi076.stdout:1/775: chown d0/d2/c44 0 1 2024-03-23T15:45:01.263 INFO:tasks.workunit.client.0.smithi076.stdout:1/776: rmdir d0/d35/d40/d47/d17/d20 39 2024-03-23T15:45:01.264 INFO:tasks.workunit.client.0.smithi076.stdout:1/777: rmdir d0 39 2024-03-23T15:45:01.264 INFO:tasks.workunit.client.0.smithi076.stdout:1/778: chown d0/d35/d40/d47/d8c/dd6 14 1 2024-03-23T15:45:01.265 INFO:tasks.workunit.client.0.smithi076.stdout:1/779: getdents d0/d35/d40/d47/d74 0 2024-03-23T15:45:01.266 INFO:tasks.workunit.client.0.smithi076.stdout:1/780: rmdir d0 39 2024-03-23T15:45:01.267 INFO:tasks.workunit.client.0.smithi076.stdout:1/781: creat d0/d2/d91/ff9 x:0 0 0 2024-03-23T15:45:01.268 INFO:tasks.workunit.client.0.smithi076.stdout:1/782: rmdir d0/dc3 39 2024-03-23T15:45:01.269 INFO:tasks.workunit.client.0.smithi076.stdout:1/783: creat d0/d2/d70/ffa x:0 0 0 2024-03-23T15:45:01.269 INFO:tasks.workunit.client.0.smithi076.stdout:1/784: read - d0/d35/d40/d47/d55/fc6 zero size 2024-03-23T15:45:01.269 INFO:tasks.workunit.client.0.smithi076.stdout:1/785: readlink d0/d2/d91/l10 0 2024-03-23T15:45:01.279 INFO:tasks.workunit.client.0.smithi076.stdout:6/754: dwrite d1/dd/df/d5b/dcd/dd0/fd9 [0,4194304] 0 2024-03-23T15:45:01.282 INFO:tasks.workunit.client.0.smithi076.stdout:6/755: creat d1/dd/df/d5b/d62/df7/f10b x:0 0 0 2024-03-23T15:45:01.283 INFO:tasks.workunit.client.0.smithi076.stdout:3/796: dwrite d5/d17/d1c/d31/fa1 [0,4194304] 0 2024-03-23T15:45:01.285 INFO:tasks.workunit.client.0.smithi076.stdout:3/797: unlink d5/d17/l74 0 2024-03-23T15:45:01.285 INFO:tasks.workunit.client.0.smithi076.stdout:1/786: dwrite d0/f4b [0,4194304] 0 2024-03-23T15:45:01.286 INFO:tasks.workunit.client.0.smithi076.stdout:3/798: mkdir d5/d17/d1c/d32/d39/d59/d79/d114 0 2024-03-23T15:45:01.286 INFO:tasks.workunit.client.0.smithi076.stdout:3/799: stat d5/d35/d6d/d7f/fef 0 2024-03-23T15:45:01.300 INFO:tasks.workunit.client.0.smithi076.stdout:6/756: link d1/l3f d1/d2/dc8/l10c 0 2024-03-23T15:45:01.300 INFO:tasks.workunit.client.0.smithi076.stdout:6/757: write d1/dd/d2a/f8c [118460,31683] 0 2024-03-23T15:45:01.301 INFO:tasks.workunit.client.0.smithi076.stdout:8/913: sync 2024-03-23T15:45:01.301 INFO:tasks.workunit.client.0.smithi076.stdout:6/758: mknod d1/dd/d2a/dfd/c10d 0 2024-03-23T15:45:01.301 INFO:tasks.workunit.client.0.smithi076.stdout:6/759: chown d1/d2/f22 6220160 1 2024-03-23T15:45:01.301 INFO:tasks.workunit.client.0.smithi076.stdout:3/800: dwrite d5/d35/d3c/f94 [0,4194304] 0 2024-03-23T15:45:01.304 INFO:tasks.workunit.client.0.smithi076.stdout:1/787: dwrite d0/d35/d40/d47/d17/f54 [0,4194304] 0 2024-03-23T15:45:01.304 INFO:tasks.workunit.client.0.smithi076.stdout:8/914: creat d8/d89/f133 x:0 0 0 2024-03-23T15:45:01.306 INFO:tasks.workunit.client.0.smithi076.stdout:8/915: creat d8/d2e/d30/d69/dd3/f134 x:0 0 0 2024-03-23T15:45:01.306 INFO:tasks.workunit.client.0.smithi076.stdout:8/916: dread - d8/d26/d33/ddd/ffd zero size 2024-03-23T15:45:01.308 INFO:tasks.workunit.client.0.smithi076.stdout:3/801: dread d5/d17/d1c/f1e [0,4194304] 0 2024-03-23T15:45:01.313 INFO:tasks.workunit.client.0.smithi076.stdout:6/760: link d1/dd/df/d9b/db3/d8d/c101 d1/d4/de2/c10e 0 2024-03-23T15:45:01.313 INFO:tasks.workunit.client.0.smithi076.stdout:1/788: dread d0/d2/d91/f5b [0,4194304] 0 2024-03-23T15:45:01.314 INFO:tasks.workunit.client.0.smithi076.stdout:8/917: dread f7 [0,4194304] 0 2024-03-23T15:45:01.315 INFO:tasks.workunit.client.0.smithi076.stdout:1/789: rename d0/d35/d40/d47/l5a to d0/d35/d40/d47/d17/d20/lfb 0 2024-03-23T15:45:01.315 INFO:tasks.workunit.client.0.smithi076.stdout:8/918: truncate d8/d89/f5f 1358693 0 2024-03-23T15:45:01.316 INFO:tasks.workunit.client.0.smithi076.stdout:8/919: readlink d8/d6b/d9d/l105 0 2024-03-23T15:45:01.316 INFO:tasks.workunit.client.0.smithi076.stdout:6/761: mkdir d1/dd/df/d5b/dcd/d10f 0 2024-03-23T15:45:01.322 INFO:tasks.workunit.client.0.smithi076.stdout:1/790: dread d0/d35/d40/d47/d17/f38 [0,4194304] 0 2024-03-23T15:45:01.323 INFO:tasks.workunit.client.0.smithi076.stdout:6/762: dread d1/d2/f87 [0,4194304] 0 2024-03-23T15:45:01.323 INFO:tasks.workunit.client.0.smithi076.stdout:6/763: chown d1/dd/c93 1 1 2024-03-23T15:45:01.325 INFO:tasks.workunit.client.0.smithi076.stdout:6/764: rmdir d1/dd4 0 2024-03-23T15:45:01.325 INFO:tasks.workunit.client.0.smithi076.stdout:3/802: dwrite d5/d35/d4c/d5a/f81 [0,4194304] 0 2024-03-23T15:45:01.328 INFO:tasks.workunit.client.0.smithi076.stdout:3/803: creat d5/d35/d6d/f115 x:0 0 0 2024-03-23T15:45:01.328 INFO:tasks.workunit.client.0.smithi076.stdout:3/804: chown d5/d17/d1c/d32/d39/d59/d79/c90 15572 1 2024-03-23T15:45:01.328 INFO:tasks.workunit.client.0.smithi076.stdout:1/791: dread d0/f4b [0,4194304] 0 2024-03-23T15:45:01.339 INFO:tasks.workunit.client.0.smithi076.stdout:3/805: dwrite d5/d17/d1c/d31/d61/faa [4194304,4194304] 0 2024-03-23T15:45:01.440 INFO:tasks.workunit.client.0.smithi076.stdout:1/792: mkdir d0/d2/dfc 0 2024-03-23T15:45:01.442 INFO:tasks.workunit.client.0.smithi076.stdout:1/793: dread d0/d35/d40/d47/d8c/d99/ff1 [0,4194304] 0 2024-03-23T15:45:01.442 INFO:tasks.workunit.client.0.smithi076.stdout:3/806: getdents d5/dc2 0 2024-03-23T15:45:01.443 INFO:tasks.workunit.client.0.smithi076.stdout:3/807: truncate d5/f43 3518173 0 2024-03-23T15:45:01.443 INFO:tasks.workunit.client.0.smithi076.stdout:6/765: unlink d1/dd/l58 0 2024-03-23T15:45:01.445 INFO:tasks.workunit.client.0.smithi076.stdout:6/766: rmdir d1/dd/df/d9b/db3/d8d/d60 39 2024-03-23T15:45:01.448 INFO:tasks.workunit.client.0.smithi076.stdout:8/920: rename d8/d2e/d30/d69/fdb to d8/d26/d33/f135 0 2024-03-23T15:45:01.449 INFO:tasks.workunit.client.0.smithi076.stdout:8/921: fdatasync f3 0 2024-03-23T15:45:01.449 INFO:tasks.workunit.client.0.smithi076.stdout:8/922: chown d8/d4f/d80/da3/fdf 20 1 2024-03-23T15:45:01.449 INFO:tasks.workunit.client.0.smithi076.stdout:8/923: truncate d8/d6b/d8e/fe4 634372 0 2024-03-23T15:45:01.449 INFO:tasks.workunit.client.0.smithi076.stdout:8/924: read d8/d2e/d3e/f9b [1990280,54210] 0 2024-03-23T15:45:01.449 INFO:tasks.workunit.client.0.smithi076.stdout:6/767: creat d1/d4/f110 x:0 0 0 2024-03-23T15:45:01.454 INFO:tasks.workunit.client.0.smithi076.stdout:6/768: unlink d1/d4/d10/c3d 0 2024-03-23T15:45:01.454 INFO:tasks.workunit.client.0.smithi076.stdout:6/769: readlink d1/dd/df/d9b/db3/la8 0 2024-03-23T15:45:01.461 INFO:tasks.workunit.client.0.smithi076.stdout:1/794: dwrite d0/d35/f87 [4194304,4194304] 0 2024-03-23T15:45:01.462 INFO:tasks.workunit.client.0.smithi076.stdout:6/770: dread d1/dd/f26 [0,4194304] 0 2024-03-23T15:45:01.463 INFO:tasks.workunit.client.0.smithi076.stdout:6/771: truncate d1/d2/dc8/fff 1120082 0 2024-03-23T15:45:01.467 INFO:tasks.workunit.client.0.smithi076.stdout:6/772: symlink d1/d2/l111 0 2024-03-23T15:45:01.468 INFO:tasks.workunit.client.0.smithi076.stdout:1/795: write d0/f9d [2290072,92295] 0 2024-03-23T15:45:01.470 INFO:tasks.workunit.client.0.smithi076.stdout:1/796: mkdir d0/d3c/dfd 0 2024-03-23T15:45:01.470 INFO:tasks.workunit.client.0.smithi076.stdout:1/797: stat d0/d2/d62/fae 0 2024-03-23T15:45:01.470 INFO:tasks.workunit.client.0.smithi076.stdout:8/925: dwrite d8/d26/d33/f3a [0,4194304] 0 2024-03-23T15:45:01.473 INFO:tasks.workunit.client.0.smithi076.stdout:3/808: dwrite d5/d17/d1c/d32/d39/f53 [0,4194304] 0 2024-03-23T15:45:01.474 INFO:tasks.workunit.client.0.smithi076.stdout:8/926: creat d8/d2e/d30/d69/dd3/d110/f136 x:0 0 0 2024-03-23T15:45:01.475 INFO:tasks.workunit.client.0.smithi076.stdout:8/927: symlink d8/d2e/d3e/deb/l137 0 2024-03-23T15:45:01.475 INFO:tasks.workunit.client.0.smithi076.stdout:8/928: readlink d8/d2e/d30/lc6 0 2024-03-23T15:45:01.476 INFO:tasks.workunit.client.0.smithi076.stdout:8/929: write d8/d26/d27/f37 [1377996,29462] 0 2024-03-23T15:45:01.476 INFO:tasks.workunit.client.0.smithi076.stdout:6/773: dread d1/d2/f22 [0,4194304] 0 2024-03-23T15:45:01.476 INFO:tasks.workunit.client.0.smithi076.stdout:3/809: dread d5/d17/d1c/d32/d70/d100/f112 [0,4194304] 0 2024-03-23T15:45:01.477 INFO:tasks.workunit.client.0.smithi076.stdout:3/810: stat d5/f28 0 2024-03-23T15:45:01.477 INFO:tasks.workunit.client.0.smithi076.stdout:8/930: rmdir d8 39 2024-03-23T15:45:01.478 INFO:tasks.workunit.client.0.smithi076.stdout:6/774: creat d1/dd/df/d5b/dcd/dd0/f112 x:0 0 0 2024-03-23T15:45:01.484 INFO:tasks.workunit.client.0.smithi076.stdout:8/931: write d8/f48 [3748832,31676] 0 2024-03-23T15:45:01.485 INFO:tasks.workunit.client.0.smithi076.stdout:3/811: write d5/d17/d1c/d32/d39/d59/d79/f8d [6040734,48697] 0 2024-03-23T15:45:01.490 INFO:tasks.workunit.client.0.smithi076.stdout:3/812: write d5/d17/d1c/d32/d4a/ff8 [713074,51915] 0 2024-03-23T15:45:01.491 INFO:tasks.workunit.client.0.smithi076.stdout:6/775: rename d1/dd/df/d9b to d1/dd/d2a/dfd/d113 0 2024-03-23T15:45:01.492 INFO:tasks.workunit.client.0.smithi076.stdout:6/776: mkdir d1/dd/df/d5b/dcd/d10f/d114 0 2024-03-23T15:45:01.492 INFO:tasks.workunit.client.0.smithi076.stdout:6/777: truncate d1/dd/d2a/dfd/d113/db3/d8d/d60/f107 463389 0 2024-03-23T15:45:01.504 INFO:tasks.workunit.client.0.smithi076.stdout:8/932: dwrite d8/d4f/fcd [4194304,4194304] 0 2024-03-23T15:45:01.504 INFO:tasks.workunit.client.0.smithi076.stdout:3/813: dread d5/f13 [0,4194304] 0 2024-03-23T15:45:01.505 INFO:tasks.workunit.client.0.smithi076.stdout:8/933: symlink d8/d26/d33/d8c/l138 0 2024-03-23T15:45:01.508 INFO:tasks.workunit.client.0.smithi076.stdout:3/814: write d5/f6 [237957,68673] 0 2024-03-23T15:45:01.510 INFO:tasks.workunit.client.0.smithi076.stdout:6/778: dwrite d1/dd/d2a/dfd/d113/f104 [0,4194304] 0 2024-03-23T15:45:01.517 INFO:tasks.workunit.client.0.smithi076.stdout:3/815: write d5/d17/d1c/d32/d39/f3e [1097018,119510] 0 2024-03-23T15:45:01.518 INFO:tasks.workunit.client.0.smithi076.stdout:3/816: mkdir d5/d17/d1c/d32/d39/d59/d79/d114/d116 0 2024-03-23T15:45:01.520 INFO:tasks.workunit.client.0.smithi076.stdout:6/779: write d1/d4/d10/f4a [190792,97913] 0 2024-03-23T15:45:01.522 INFO:tasks.workunit.client.0.smithi076.stdout:6/780: write d1/dd/df/d5b/f55 [316919,11079] 0 2024-03-23T15:45:01.525 INFO:tasks.workunit.client.0.smithi076.stdout:3/817: dread d5/d35/d4c/d5a/fb9 [0,4194304] 0 2024-03-23T15:45:01.525 INFO:tasks.workunit.client.0.smithi076.stdout:8/934: dwrite d8/d2e/d30/f6a [0,4194304] 0 2024-03-23T15:45:01.526 INFO:tasks.workunit.client.0.smithi076.stdout:8/935: fsync d8/d6b/d8e/fe4 0 2024-03-23T15:45:01.526 INFO:tasks.workunit.client.0.smithi076.stdout:8/936: truncate d8/d4f/d80/da3/fdf 4565575 0 2024-03-23T15:45:01.526 INFO:tasks.workunit.client.0.smithi076.stdout:3/818: getdents d5/d17/d1c/d32/d39/d59/d79/ddc 0 2024-03-23T15:45:01.527 INFO:tasks.workunit.client.0.smithi076.stdout:8/937: mknod d8/d26/d27/c139 0 2024-03-23T15:45:01.529 INFO:tasks.workunit.client.0.smithi076.stdout:3/819: rename d5/d17/d1c/d32/f5f to d5/d35/d3c/f117 0 2024-03-23T15:45:01.530 INFO:tasks.workunit.client.0.smithi076.stdout:8/938: mknod d8/dd0/c13a 0 2024-03-23T15:45:01.532 INFO:tasks.workunit.client.0.smithi076.stdout:6/781: write d1/d4/d10/f90 [5040146,114714] 0 2024-03-23T15:45:01.532 INFO:tasks.workunit.client.0.smithi076.stdout:8/939: creat d8/d83/f13b x:0 0 0 2024-03-23T15:45:01.537 INFO:tasks.workunit.client.0.smithi076.stdout:8/940: chown d8/d2e/d30/d69/c76 5179 1 2024-03-23T15:45:01.538 INFO:tasks.workunit.client.0.smithi076.stdout:8/941: read d8/f11 [905017,95959] 0 2024-03-23T15:45:01.540 INFO:tasks.workunit.client.0.smithi076.stdout:8/942: creat d8/d4f/f13c x:0 0 0 2024-03-23T15:45:01.541 INFO:tasks.workunit.client.0.smithi076.stdout:8/943: mknod d8/d6b/d9a/c13d 0 2024-03-23T15:45:01.544 INFO:tasks.workunit.client.0.smithi076.stdout:3/820: dwrite d5/d17/d1c/f52 [0,4194304] 0 2024-03-23T15:45:01.545 INFO:tasks.workunit.client.0.smithi076.stdout:3/821: rmdir d5/d35/d3c 39 2024-03-23T15:45:01.546 INFO:tasks.workunit.client.0.smithi076.stdout:6/782: dwrite d1/dd/d2a/d23/faa [0,4194304] 0 2024-03-23T15:45:01.547 INFO:tasks.workunit.client.0.smithi076.stdout:3/822: read d5/d35/d6d/d7f/fef [146426,128102] 0 2024-03-23T15:45:01.547 INFO:tasks.workunit.client.0.smithi076.stdout:3/823: read - d5/d35/d6d/f115 zero size 2024-03-23T15:45:01.547 INFO:tasks.workunit.client.0.smithi076.stdout:6/783: creat d1/d4/d103/f115 x:0 0 0 2024-03-23T15:45:01.548 INFO:tasks.workunit.client.0.smithi076.stdout:3/824: symlink d5/d17/d1c/d32/d39/d59/d79/d114/l118 0 2024-03-23T15:45:01.550 INFO:tasks.workunit.client.0.smithi076.stdout:8/944: write d8/f19 [5130330,87195] 0 2024-03-23T15:45:01.550 INFO:tasks.workunit.client.0.smithi076.stdout:3/825: creat d5/d17/d1c/d32/d4a/f119 x:0 0 0 2024-03-23T15:45:01.551 INFO:tasks.workunit.client.0.smithi076.stdout:3/826: creat d5/d17/d1c/f11a x:0 0 0 2024-03-23T15:45:01.551 INFO:tasks.workunit.client.0.smithi076.stdout:3/827: readlink d5/d17/d1c/d31/d61/d92/d10a/lf4 0 2024-03-23T15:45:01.552 INFO:tasks.workunit.client.0.smithi076.stdout:3/828: write d5/d35/d6d/f99 [617543,82054] 0 2024-03-23T15:45:01.552 INFO:tasks.workunit.client.0.smithi076.stdout:3/829: chown d5/d17/d1c/d32/d5d 48797 1 2024-03-23T15:45:01.552 INFO:tasks.workunit.client.0.smithi076.stdout:3/830: rmdir d5/d17/d1c/d31/d61/d96 39 2024-03-23T15:45:01.553 INFO:tasks.workunit.client.0.smithi076.stdout:3/831: creat d5/d35/d4c/f11b x:0 0 0 2024-03-23T15:45:01.553 INFO:tasks.workunit.client.0.smithi076.stdout:3/832: truncate d5/d35/d3c/f117 4375243 0 2024-03-23T15:45:01.555 INFO:tasks.workunit.client.0.smithi076.stdout:3/833: unlink d5/f13 0 2024-03-23T15:45:01.559 INFO:tasks.workunit.client.0.smithi076.stdout:6/784: dread d1/d4/d10/d39/ff4 [0,4194304] 0 2024-03-23T15:45:01.562 INFO:tasks.workunit.client.0.smithi076.stdout:3/834: dread d5/f56 [0,4194304] 0 2024-03-23T15:45:01.562 INFO:tasks.workunit.client.0.smithi076.stdout:3/835: readlink d5/d17/d1c/d32/d8e/lc0 0 2024-03-23T15:45:01.565 INFO:tasks.workunit.client.0.smithi076.stdout:3/836: unlink d5/d35/d6d/f115 0 2024-03-23T15:45:01.565 INFO:tasks.workunit.client.0.smithi076.stdout:3/837: fdatasync d5/d17/d1c/d32/d39/d59/d79/ddc/fee 0 2024-03-23T15:45:01.565 INFO:tasks.workunit.client.0.smithi076.stdout:3/838: dread - d5/d17/dcf/f10f zero size 2024-03-23T15:45:01.565 INFO:tasks.workunit.client.0.smithi076.stdout:3/839: symlink d5/d17/d1c/d32/d70/dfa/l11c 0 2024-03-23T15:45:01.567 INFO:tasks.workunit.client.0.smithi076.stdout:8/945: dwrite d8/d26/d33/d101/f10a [0,4194304] 0 2024-03-23T15:45:01.568 INFO:tasks.workunit.client.0.smithi076.stdout:3/840: mkdir d5/d35/d4c/d5a/d11d 0 2024-03-23T15:45:01.579 INFO:tasks.workunit.client.0.smithi076.stdout:3/841: read d5/d35/d6d/d7f/fae [2307222,125435] 0 2024-03-23T15:45:01.582 INFO:tasks.workunit.client.0.smithi076.stdout:3/842: mknod d5/d35/d6d/c11e 0 2024-03-23T15:45:01.584 INFO:tasks.workunit.client.0.smithi076.stdout:8/946: dwrite d8/d6b/d8e/fe9 [0,4194304] 0 2024-03-23T15:45:01.588 INFO:tasks.workunit.client.0.smithi076.stdout:3/843: dread d5/f6 [0,4194304] 0 2024-03-23T15:45:01.591 INFO:tasks.workunit.client.0.smithi076.stdout:8/947: rmdir d8/d26/d33/d8c 39 2024-03-23T15:45:01.591 INFO:tasks.workunit.client.0.smithi076.stdout:8/948: chown d8/d89/l58 2462 1 2024-03-23T15:45:01.592 INFO:tasks.workunit.client.0.smithi076.stdout:8/949: chown d8/d6b/d9d/cca 3 1 2024-03-23T15:45:01.594 INFO:tasks.workunit.client.0.smithi076.stdout:6/785: rename d1/dd/df/c6c to d1/d4/d10/c116 0 2024-03-23T15:45:01.595 INFO:tasks.workunit.client.0.smithi076.stdout:6/786: truncate d1/d2/dc8/fff 372258 0 2024-03-23T15:45:01.596 INFO:tasks.workunit.client.0.smithi076.stdout:8/950: symlink d8/d26/d33/l13e 0 2024-03-23T15:45:01.596 INFO:tasks.workunit.client.0.smithi076.stdout:8/951: chown d8/d2e/d3e/l4c 8248 1 2024-03-23T15:45:01.596 INFO:tasks.workunit.client.0.smithi076.stdout:8/952: fsync d8/f14 0 2024-03-23T15:45:01.596 INFO:tasks.workunit.client.0.smithi076.stdout:1/798: sync 2024-03-23T15:45:01.598 INFO:tasks.workunit.client.0.smithi076.stdout:1/799: unlink d0/d35/d40/d47/d55/ca9 0 2024-03-23T15:45:01.601 INFO:tasks.workunit.client.0.smithi076.stdout:3/844: dwrite d5/d35/d3c/d67/fbc [4194304,4194304] 0 2024-03-23T15:45:01.603 INFO:tasks.workunit.client.0.smithi076.stdout:3/845: symlink d5/d17/l11f 0 2024-03-23T15:45:01.603 INFO:tasks.workunit.client.0.smithi076.stdout:3/846: readlink d5/d17/d1c/d31/d61/d96/le6 0 2024-03-23T15:45:01.605 INFO:tasks.workunit.client.0.smithi076.stdout:8/953: write d8/f1d [1648362,12593] 0 2024-03-23T15:45:01.605 INFO:tasks.workunit.client.0.smithi076.stdout:3/847: creat d5/d17/dcf/de2/dd7/f120 x:0 0 0 2024-03-23T15:45:01.606 INFO:tasks.workunit.client.0.smithi076.stdout:1/800: write d0/f26 [4543761,122994] 0 2024-03-23T15:45:01.606 INFO:tasks.workunit.client.0.smithi076.stdout:8/954: mknod d8/dd0/c13f 0 2024-03-23T15:45:01.608 INFO:tasks.workunit.client.0.smithi076.stdout:3/848: creat d5/d17/d1c/d32/d4a/f121 x:0 0 0 2024-03-23T15:45:01.608 INFO:tasks.workunit.client.0.smithi076.stdout:1/801: mkdir d0/d2/d70/da1/dfe 0 2024-03-23T15:45:01.616 INFO:tasks.workunit.client.0.smithi076.stdout:1/802: write d0/d3c/d6e/fe5 [3464576,102338] 0 2024-03-23T15:45:01.618 INFO:tasks.workunit.client.0.smithi076.stdout:3/849: write d5/d17/d1c/d32/d39/f7c [633914,23504] 0 2024-03-23T15:45:01.619 INFO:tasks.workunit.client.0.smithi076.stdout:6/787: rename d1/d81/cfc to d1/d2/df2/c117 0 2024-03-23T15:45:01.619 INFO:tasks.workunit.client.0.smithi076.stdout:6/788: dread - d1/dd/df/d5b/dcd/dd0/f112 zero size 2024-03-23T15:45:01.619 INFO:tasks.workunit.client.0.smithi076.stdout:6/789: fdatasync d1/dd/df/dac/ff8 0 2024-03-23T15:45:01.623 INFO:tasks.workunit.client.0.smithi076.stdout:8/955: dwrite d8/f11 [0,4194304] 0 2024-03-23T15:45:01.623 INFO:tasks.workunit.client.0.smithi076.stdout:8/956: chown d8/d2e/d30/d64/fc0 246 1 2024-03-23T15:45:01.626 INFO:tasks.workunit.client.0.smithi076.stdout:1/803: read d0/d35/d40/d47/d55/fb9 [1664165,104993] 0 2024-03-23T15:45:01.626 INFO:tasks.workunit.client.0.smithi076.stdout:1/804: getdents d0/d3c 0 2024-03-23T15:45:01.627 INFO:tasks.workunit.client.0.smithi076.stdout:6/790: rename d1/d4/d10/d39/l53 to d1/dd/df/dac/dec/l118 0 2024-03-23T15:45:01.628 INFO:tasks.workunit.client.0.smithi076.stdout:1/805: mknod d0/d2/d70/da1/dbf/cff 0 2024-03-23T15:45:01.629 INFO:tasks.workunit.client.0.smithi076.stdout:6/791: dread d1/dd/d2a/f8c [0,4194304] 0 2024-03-23T15:45:01.631 INFO:tasks.workunit.client.0.smithi076.stdout:6/792: creat d1/d2/df2/f119 x:0 0 0 2024-03-23T15:45:01.631 INFO:tasks.workunit.client.0.smithi076.stdout:6/793: readlink d1/dd/d2a/dfd/d113/db3/d8d/d60/lc9 0 2024-03-23T15:45:01.632 INFO:tasks.workunit.client.0.smithi076.stdout:6/794: symlink d1/dd/df/d5b/dcd/dd0/l11a 0 2024-03-23T15:45:01.632 INFO:tasks.workunit.client.0.smithi076.stdout:6/795: fdatasync d1/dd1/ff1 0 2024-03-23T15:45:01.634 INFO:tasks.workunit.client.0.smithi076.stdout:6/796: creat d1/dd/d2a/dfd/d113/db3/d8d/d60/d7c/f11b x:0 0 0 2024-03-23T15:45:01.634 INFO:tasks.workunit.client.0.smithi076.stdout:3/850: dwrite d5/fd [4194304,4194304] 0 2024-03-23T15:45:01.634 INFO:tasks.workunit.client.0.smithi076.stdout:1/806: truncate d0/d2/d91/f5b 8861029 0 2024-03-23T15:45:01.642 INFO:tasks.workunit.client.0.smithi076.stdout:8/957: link d8/d2e/c41 d8/d2e/d30/d69/d93/c140 0 2024-03-23T15:45:01.644 INFO:tasks.workunit.client.0.smithi076.stdout:8/958: chown d8/d26/d27/da9/fd7 2 1 2024-03-23T15:45:01.645 INFO:tasks.workunit.client.0.smithi076.stdout:8/959: write d8/d6b/d9a/fa6 [864248,84079] 0 2024-03-23T15:45:01.645 INFO:tasks.workunit.client.0.smithi076.stdout:8/960: dread - d8/d4f/d80/da3/df7/d121/f128 zero size 2024-03-23T15:45:01.646 INFO:tasks.workunit.client.0.smithi076.stdout:8/961: truncate d8/d6b/d8e/fe9 795955 0 2024-03-23T15:45:01.646 INFO:tasks.workunit.client.0.smithi076.stdout:8/962: write d8/d4f/d80/db7/fbe [1332897,120590] 0 2024-03-23T15:45:01.646 INFO:tasks.workunit.client.0.smithi076.stdout:8/963: fdatasync d8/dd0/f10b 0 2024-03-23T15:45:01.647 INFO:tasks.workunit.client.0.smithi076.stdout:8/964: symlink d8/d2e/l141 0 2024-03-23T15:45:01.650 INFO:tasks.workunit.client.0.smithi076.stdout:6/797: link d1/d4/l15 d1/dd/d2a/dfd/d113/l11c 0 2024-03-23T15:45:01.652 INFO:tasks.workunit.client.0.smithi076.stdout:1/807: dwrite d0/d2/f37 [0,4194304] 0 2024-03-23T15:45:01.655 INFO:tasks.workunit.client.0.smithi076.stdout:1/808: dread d0/f9d [0,4194304] 0 2024-03-23T15:45:01.657 INFO:tasks.workunit.client.0.smithi076.stdout:1/809: mkdir d0/d35/d40/d47/d55/d100 0 2024-03-23T15:45:01.657 INFO:tasks.workunit.client.0.smithi076.stdout:8/965: dread d8/fe [0,4194304] 0 2024-03-23T15:45:01.657 INFO:tasks.workunit.client.0.smithi076.stdout:3/851: dwrite d5/d17/d1c/f2b [0,4194304] 0 2024-03-23T15:45:01.657 INFO:tasks.workunit.client.0.smithi076.stdout:3/852: fsync d5/f56 0 2024-03-23T15:45:01.658 INFO:tasks.workunit.client.0.smithi076.stdout:1/810: creat d0/d2/d70/d7a/d8b/dcb/f101 x:0 0 0 2024-03-23T15:45:01.660 INFO:tasks.workunit.client.0.smithi076.stdout:8/966: creat d8/d6b/f142 x:0 0 0 2024-03-23T15:45:01.660 INFO:tasks.workunit.client.0.smithi076.stdout:8/967: write d8/d4f/d80/db7/fbe [4244351,558] 0 2024-03-23T15:45:01.661 INFO:tasks.workunit.client.0.smithi076.stdout:3/853: truncate d5/d35/d6d/fa7 886215 0 2024-03-23T15:45:01.661 INFO:tasks.workunit.client.0.smithi076.stdout:1/811: creat d0/d2/d70/da1/dfe/f102 x:0 0 0 2024-03-23T15:45:01.664 INFO:tasks.workunit.client.0.smithi076.stdout:3/854: creat d5/d17/dcf/f122 x:0 0 0 2024-03-23T15:45:01.665 INFO:tasks.workunit.client.0.smithi076.stdout:3/855: dread d5/d35/d3c/d67/ff3 [0,4194304] 0 2024-03-23T15:45:01.665 INFO:tasks.workunit.client.0.smithi076.stdout:1/812: symlink d0/d2/d70/da1/dbf/l103 0 2024-03-23T15:45:01.667 INFO:tasks.workunit.client.0.smithi076.stdout:3/856: mkdir d5/d123 0 2024-03-23T15:45:01.668 INFO:tasks.workunit.client.0.smithi076.stdout:6/798: dwrite d1/d4/f86 [0,4194304] 0 2024-03-23T15:45:01.669 INFO:tasks.workunit.client.0.smithi076.stdout:6/799: chown d1/d4/d10/f74 10466051 1 2024-03-23T15:45:01.669 INFO:tasks.workunit.client.0.smithi076.stdout:6/800: write d1/f85 [1080438,86783] 0 2024-03-23T15:45:01.669 INFO:tasks.workunit.client.0.smithi076.stdout:3/857: creat d5/d17/d1c/d32/f124 x:0 0 0 2024-03-23T15:45:01.671 INFO:tasks.workunit.client.0.smithi076.stdout:3/858: symlink d5/d17/d1c/d32/d70/d8c/l125 0 2024-03-23T15:45:01.671 INFO:tasks.workunit.client.0.smithi076.stdout:3/859: truncate d5/d35/d6d/fd3 422730 0 2024-03-23T15:45:01.672 INFO:tasks.workunit.client.0.smithi076.stdout:3/860: fdatasync d5/d17/d1c/d31/f64 0 2024-03-23T15:45:01.674 INFO:tasks.workunit.client.0.smithi076.stdout:6/801: unlink d1/dd/df/l4b 0 2024-03-23T15:45:01.675 INFO:tasks.workunit.client.0.smithi076.stdout:8/968: dread d8/d2e/d30/f61 [4194304,4194304] 0 2024-03-23T15:45:01.676 INFO:tasks.workunit.client.0.smithi076.stdout:8/969: dread d8/d26/d27/d56/fba [0,4194304] 0 2024-03-23T15:45:01.676 INFO:tasks.workunit.client.0.smithi076.stdout:8/970: read - d8/d2e/d30/d69/f71 zero size 2024-03-23T15:45:01.676 INFO:tasks.workunit.client.0.smithi076.stdout:6/802: creat d1/dd/d2a/dfd/f11d x:0 0 0 2024-03-23T15:45:01.686 INFO:tasks.workunit.client.0.smithi076.stdout:6/803: dread d1/dd/df/f66 [0,4194304] 0 2024-03-23T15:45:01.686 INFO:tasks.workunit.client.0.smithi076.stdout:6/804: fsync d1/d4/d10/f4a 0 2024-03-23T15:45:01.686 INFO:tasks.workunit.client.0.smithi076.stdout:6/805: rmdir d1/dd/df/d5b 39 2024-03-23T15:45:01.691 INFO:tasks.workunit.client.0.smithi076.stdout:1/813: dwrite d0/d2/d91/d86/fe8 [0,4194304] 0 2024-03-23T15:45:01.696 INFO:tasks.workunit.client.0.smithi076.stdout:3/861: rename d5/d17/d1c/cd2 to d5/d17/d1c/d32/d39/d59/d79/d114/c126 0 2024-03-23T15:45:01.696 INFO:tasks.workunit.client.0.smithi076.stdout:3/862: creat d5/d17/d1c/d32/d5d/f127 x:0 0 0 2024-03-23T15:45:01.699 INFO:tasks.workunit.client.0.smithi076.stdout:3/863: link d5/d17/d1c/d32/l2c d5/d17/dcf/de2/de7/l128 0 2024-03-23T15:45:01.702 INFO:tasks.workunit.client.0.smithi076.stdout:6/806: link d1/d81/la5 d1/dd1/l11e 0 2024-03-23T15:45:01.704 INFO:tasks.workunit.client.0.smithi076.stdout:6/807: rename d1/d4/d10/d39/lbb to d1/dd/df/d5b/dcd/d10f/d114/l11f 0 2024-03-23T15:45:01.705 INFO:tasks.workunit.client.0.smithi076.stdout:6/808: mknod d1/d81/c120 0 2024-03-23T15:45:01.706 INFO:tasks.workunit.client.0.smithi076.stdout:6/809: mknod d1/d4/d103/c121 0 2024-03-23T15:45:01.715 INFO:tasks.workunit.client.0.smithi076.stdout:1/814: link d0/d35/d40/d47/d74/d83/d8f/l95 d0/d2/d62/l104 0 2024-03-23T15:45:01.715 INFO:tasks.workunit.client.0.smithi076.stdout:1/815: chown d0/d3c/d6e/dc4 12 1 2024-03-23T15:45:01.716 INFO:tasks.workunit.client.0.smithi076.stdout:3/864: dwrite d5/d17/d1c/d32/d4a/fbb [0,4194304] 0 2024-03-23T15:45:01.716 INFO:tasks.workunit.client.0.smithi076.stdout:3/865: dread - d5/d17/d1c/f11a zero size 2024-03-23T15:45:01.717 INFO:tasks.workunit.client.0.smithi076.stdout:1/816: creat d0/d2/d91/f105 x:0 0 0 2024-03-23T15:45:01.717 INFO:tasks.workunit.client.0.smithi076.stdout:1/817: write d0/d2/d70/da1/dbf/fb2 [4792725,104038] 0 2024-03-23T15:45:01.717 INFO:tasks.workunit.client.0.smithi076.stdout:1/818: chown d0/d3c/dfd 3089120 1 2024-03-23T15:45:01.717 INFO:tasks.workunit.client.0.smithi076.stdout:1/819: chown d0/d2/d91/f105 8284127 1 2024-03-23T15:45:01.717 INFO:tasks.workunit.client.0.smithi076.stdout:1/820: write d0/d2/d91/ff9 [936097,103584] 0 2024-03-23T15:45:01.719 INFO:tasks.workunit.client.0.smithi076.stdout:1/821: rename d0/d2/d70/da1/dbf/fb2 to d0/d2/f106 0 2024-03-23T15:45:01.720 INFO:tasks.workunit.client.0.smithi076.stdout:6/810: dwrite d1/dd/d2a/d23/f102 [0,4194304] 0 2024-03-23T15:45:01.720 INFO:tasks.workunit.client.0.smithi076.stdout:6/811: chown d1/d2/dc8 406061663 1 2024-03-23T15:45:01.720 INFO:tasks.workunit.client.0.smithi076.stdout:1/822: mkdir d0/d2/d70/da1/dbf/d107 0 2024-03-23T15:45:01.722 INFO:tasks.workunit.client.0.smithi076.stdout:1/823: link d0/d35/d40/d47/d74/d83/f98 d0/d35/d40/d47/d8c/f108 0 2024-03-23T15:45:01.723 INFO:tasks.workunit.client.0.smithi076.stdout:1/824: creat d0/f109 x:0 0 0 2024-03-23T15:45:01.724 INFO:tasks.workunit.client.0.smithi076.stdout:1/825: getdents d0/d35/d40/d47/d55/d100 0 2024-03-23T15:45:01.724 INFO:tasks.workunit.client.0.smithi076.stdout:3/866: dread d5/d35/d4c/d5a/fb0 [0,4194304] 0 2024-03-23T15:45:01.724 INFO:tasks.workunit.client.0.smithi076.stdout:3/867: getdents d5/d35/d4c/de1 0 2024-03-23T15:45:01.725 INFO:tasks.workunit.client.0.smithi076.stdout:1/826: mkdir d0/d2/d70/d10a 0 2024-03-23T15:45:01.727 INFO:tasks.workunit.client.0.smithi076.stdout:1/827: unlink d0/d2/d70/da1/dbf/c8e 0 2024-03-23T15:45:01.727 INFO:tasks.workunit.client.0.smithi076.stdout:6/812: dread d1/d2/f6d [8388608,4194304] 0 2024-03-23T15:45:01.728 INFO:tasks.workunit.client.0.smithi076.stdout:1/828: creat d0/d35/f10b x:0 0 0 2024-03-23T15:45:01.729 INFO:tasks.workunit.client.0.smithi076.stdout:3/868: truncate d5/d17/d1c/d32/d39/f3e 608407 0 2024-03-23T15:45:01.729 INFO:tasks.workunit.client.0.smithi076.stdout:3/869: chown d5/d17/d1c/d32/f97 65 1 2024-03-23T15:45:01.730 INFO:tasks.workunit.client.0.smithi076.stdout:3/870: symlink d5/d17/d1c/d31/d61/l129 0 2024-03-23T15:45:01.735 INFO:tasks.workunit.client.0.smithi076.stdout:1/829: write d0/d2/d91/f5b [6811330,31105] 0 2024-03-23T15:45:01.735 INFO:tasks.workunit.client.0.smithi076.stdout:1/830: write d0/d35/f3a [2097752,125192] 0 2024-03-23T15:45:01.735 INFO:tasks.workunit.client.0.smithi076.stdout:1/831: readlink d0/d35/l94 0 2024-03-23T15:45:01.744 INFO:tasks.workunit.client.0.smithi076.stdout:1/832: write d0/d35/d40/d47/d17/faf [638240,74196] 0 2024-03-23T15:45:01.744 INFO:tasks.workunit.client.0.smithi076.stdout:1/833: readlink d0/d2/d91/l59 0 2024-03-23T15:45:01.745 INFO:tasks.workunit.client.0.smithi076.stdout:6/813: dwrite d1/dd/df/d5b/d62/f6e [0,4194304] 0 2024-03-23T15:45:01.747 INFO:tasks.workunit.client.0.smithi076.stdout:6/814: creat d1/dd/df/f122 x:0 0 0 2024-03-23T15:45:01.752 INFO:tasks.workunit.client.0.smithi076.stdout:1/834: dread d0/d35/f6f [0,4194304] 0 2024-03-23T15:45:01.752 INFO:tasks.workunit.client.0.smithi076.stdout:1/835: write d0/d2/d62/fe6 [539215,41363] 0 2024-03-23T15:45:01.753 INFO:tasks.workunit.client.0.smithi076.stdout:6/815: dread d1/dd/f26 [4194304,4194304] 0 2024-03-23T15:45:01.754 INFO:tasks.workunit.client.0.smithi076.stdout:1/836: mkdir d0/d35/d40/dd1/d10c 0 2024-03-23T15:45:01.754 INFO:tasks.workunit.client.0.smithi076.stdout:1/837: truncate d0/d2/d91/d86/fcc 570044 0 2024-03-23T15:45:01.756 INFO:tasks.workunit.client.0.smithi076.stdout:1/838: unlink d0/d2/d91/c7c 0 2024-03-23T15:45:01.756 INFO:tasks.workunit.client.0.smithi076.stdout:1/839: getdents d0/d35/d40/d47/d74/d83/dbd 0 2024-03-23T15:45:01.757 INFO:tasks.workunit.client.0.smithi076.stdout:6/816: rename d1/dd/f54 to d1/dd/f123 0 2024-03-23T15:45:01.768 INFO:tasks.workunit.client.0.smithi076.stdout:1/840: link d0/d35/d40/d47/d17/d20/l5d d0/d2/d70/d7a/d8b/dbc/l10d 0 2024-03-23T15:45:01.770 INFO:tasks.workunit.client.0.smithi076.stdout:6/817: dread d1/d4/d10/d39/ff4 [0,4194304] 0 2024-03-23T15:45:01.771 INFO:tasks.workunit.client.0.smithi076.stdout:6/818: creat d1/d4/d10/d39/f124 x:0 0 0 2024-03-23T15:45:01.780 INFO:tasks.workunit.client.0.smithi076.stdout:6/819: unlink d1/d2/dc8/l10c 0 2024-03-23T15:45:01.781 INFO:tasks.workunit.client.0.smithi076.stdout:1/841: dwrite d0/f9d [4194304,4194304] 0 2024-03-23T15:45:01.782 INFO:tasks.workunit.client.0.smithi076.stdout:6/820: creat d1/dd/df/d5b/f125 x:0 0 0 2024-03-23T15:45:01.783 INFO:tasks.workunit.client.0.smithi076.stdout:1/842: rename d0/d35/d40/d47/d17/d20/c32 to d0/d35/d40/d47/d55/c10e 0 2024-03-23T15:45:01.785 INFO:tasks.workunit.client.0.smithi076.stdout:1/843: unlink d0/d2/d70/da1/dbf/f43 0 2024-03-23T15:45:01.786 INFO:tasks.workunit.client.0.smithi076.stdout:6/821: creat d1/dd/df/d5b/dcd/f126 x:0 0 0 2024-03-23T15:45:01.786 INFO:tasks.workunit.client.0.smithi076.stdout:6/822: readlink d1/dd/d2a/dfd/d113/db3/d8d/d60/l2e 0 2024-03-23T15:45:01.788 INFO:tasks.workunit.client.0.smithi076.stdout:6/823: creat d1/f127 x:0 0 0 2024-03-23T15:45:01.788 INFO:tasks.workunit.client.0.smithi076.stdout:6/824: dread d1/dd/d2a/dfd/d113/db3/d8d/d60/f107 [0,4194304] 0 2024-03-23T15:45:01.792 INFO:tasks.workunit.client.0.smithi076.stdout:6/825: mkdir d1/dd/df/d5b/d62/d128 0 2024-03-23T15:45:01.792 INFO:tasks.workunit.client.0.smithi076.stdout:6/826: chown d1/dd/df/d5b/l40 56995295 1 2024-03-23T15:45:01.792 INFO:tasks.workunit.client.0.smithi076.stdout:6/827: read d1/d4/d10/f4a [2210273,89342] 0 2024-03-23T15:45:01.792 INFO:tasks.workunit.client.0.smithi076.stdout:6/828: creat d1/dd/d2a/dfd/d113/db3/d8d/f129 x:0 0 0 2024-03-23T15:45:01.793 INFO:tasks.workunit.client.0.smithi076.stdout:6/829: mknod d1/d81/c12a 0 2024-03-23T15:45:01.794 INFO:tasks.workunit.client.0.smithi076.stdout:1/844: dread d0/d35/d40/d47/d74/d83/d8f/fb1 [0,4194304] 0 2024-03-23T15:45:01.795 INFO:tasks.workunit.client.0.smithi076.stdout:1/845: symlink d0/d3c/d6e/dca/l10f 0 2024-03-23T15:45:01.796 INFO:tasks.workunit.client.0.smithi076.stdout:1/846: chown d0/d35/d40/d47/f6b 2687404 1 2024-03-23T15:45:01.797 INFO:tasks.workunit.client.0.smithi076.stdout:1/847: mkdir d0/d2/dfc/d110 0 2024-03-23T15:45:01.801 INFO:tasks.workunit.client.0.smithi076.stdout:1/848: write d0/d35/d40/fa8 [1786633,75644] 0 2024-03-23T15:45:01.802 INFO:tasks.workunit.client.0.smithi076.stdout:1/849: truncate d0/d35/d40/d47/d17/d20/f3e 3230645 0 2024-03-23T15:45:01.802 INFO:tasks.workunit.client.0.smithi076.stdout:1/850: stat d0/d2/d91/d86 0 2024-03-23T15:45:01.802 INFO:tasks.workunit.client.0.smithi076.stdout:1/851: fsync d0/d35/d40/d47/d17/f38 0 2024-03-23T15:45:01.803 INFO:tasks.workunit.client.0.smithi076.stdout:1/852: write d0/d35/d40/d47/d17/d20/f25 [4299870,128193] 0 2024-03-23T15:45:01.804 INFO:tasks.workunit.client.0.smithi076.stdout:1/853: mkdir d0/d35/d40/d47/d74/d83/dbd/d111 0 2024-03-23T15:45:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:01 smithi192 ceph-mon[36653]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 190 MiB/s wr, 1.50k op/s 2024-03-23T15:45:01.821 INFO:tasks.workunit.client.0.smithi076.stdout:1/854: dwrite d0/d2/d70/ffa [0,4194304] 0 2024-03-23T15:45:01.822 INFO:tasks.workunit.client.0.smithi076.stdout:1/855: mkdir d0/d35/d40/d47/d8c/dd6/d112 0 2024-03-23T15:45:01.824 INFO:tasks.workunit.client.0.smithi076.stdout:1/856: rename d0/d35/d40/d47/d74/d83/f98 to d0/d35/d40/d47/d17/f113 0 2024-03-23T15:45:01.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:01 smithi076 ceph-mon[32176]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 190 MiB/s wr, 1.50k op/s 2024-03-23T15:45:01.826 INFO:tasks.workunit.client.0.smithi076.stdout:1/857: mknod d0/d2/d70/da1/dfe/c114 0 2024-03-23T15:45:01.830 INFO:tasks.workunit.client.0.smithi076.stdout:1/858: creat d0/d2/d70/d7a/d8b/dcb/f115 x:0 0 0 2024-03-23T15:45:01.834 INFO:tasks.workunit.client.0.smithi076.stdout:1/859: unlink d0/d35/d40/d47/d17/faf 0 2024-03-23T15:45:01.849 INFO:tasks.workunit.client.0.smithi076.stdout:1/860: link d0/d35/f68 d0/d35/d40/d47/d8c/d99/f116 0 2024-03-23T15:45:01.850 INFO:tasks.workunit.client.0.smithi076.stdout:1/861: creat d0/d2/d70/da1/dbf/f117 x:0 0 0 2024-03-23T15:45:01.852 INFO:tasks.workunit.client.0.smithi076.stdout:1/862: rmdir d0 39 2024-03-23T15:45:01.854 INFO:tasks.workunit.client.0.smithi076.stdout:1/863: mkdir d0/d35/d82/d118 0 2024-03-23T15:45:01.863 INFO:tasks.workunit.client.0.smithi076.stdout:1/864: write d0/d2/d70/ffa [3743169,14348] 0 2024-03-23T15:45:01.863 INFO:tasks.workunit.client.0.smithi076.stdout:1/865: getdents d0/d2/d70/da1 0 2024-03-23T15:45:01.863 INFO:tasks.workunit.client.0.smithi076.stdout:1/866: getdents d0/d35/d40/d47/d8c/dd6/de1 0 2024-03-23T15:45:01.864 INFO:tasks.workunit.client.0.smithi076.stdout:1/867: unlink d0/d2/d91/fd8 0 2024-03-23T15:45:01.866 INFO:tasks.workunit.client.0.smithi076.stdout:1/868: creat d0/d35/d40/d47/d55/d100/f119 x:0 0 0 2024-03-23T15:45:01.866 INFO:tasks.workunit.client.0.smithi076.stdout:1/869: symlink d0/d2/d70/da1/dbf/d107/l11a 0 2024-03-23T15:45:01.878 INFO:tasks.workunit.client.0.smithi076.stdout:1/870: dwrite d0/d2/d70/da1/dfe/f102 [0,4194304] 0 2024-03-23T15:45:01.878 INFO:tasks.workunit.client.0.smithi076.stdout:1/871: readlink d0/d2/d70/da1/dbf/l24 0 2024-03-23T15:45:01.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:01 smithi183 ceph-mon[36296]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 190 MiB/s wr, 1.50k op/s 2024-03-23T15:45:01.890 INFO:tasks.workunit.client.0.smithi076.stdout:1/872: write d0/d35/d40/d47/d55/fd2 [3648671,80115] 0 2024-03-23T15:45:01.890 INFO:tasks.workunit.client.0.smithi076.stdout:1/873: mknod d0/d2/d70/d7a/d8b/dcb/c11b 0 2024-03-23T15:45:01.890 INFO:tasks.workunit.client.0.smithi076.stdout:1/874: write d0/d35/d40/fa8 [2660815,22183] 0 2024-03-23T15:45:01.890 INFO:tasks.workunit.client.0.smithi076.stdout:1/875: stat d0/d2/led 0 2024-03-23T15:45:01.907 INFO:tasks.workunit.client.0.smithi076.stdout:1/876: dwrite d0/f4b [0,4194304] 0 2024-03-23T15:45:01.907 INFO:tasks.workunit.client.0.smithi076.stdout:1/877: read - d0/d35/d40/d47/d8c/f108 zero size 2024-03-23T15:45:01.907 INFO:tasks.workunit.client.0.smithi076.stdout:1/878: chown d0/d35/d40/d47/d17/d20/l2d 4989418 1 2024-03-23T15:45:01.907 INFO:tasks.workunit.client.0.smithi076.stdout:1/879: fsync d0/d35/f64 0 2024-03-23T15:45:01.908 INFO:tasks.workunit.client.0.smithi076.stdout:1/880: creat d0/d35/d40/d47/d74/d83/dbd/d111/f11c x:0 0 0 2024-03-23T15:45:01.910 INFO:tasks.workunit.client.0.smithi076.stdout:1/881: truncate d0/d35/d40/d47/d55/fb9 3248086 0 2024-03-23T15:45:01.926 INFO:tasks.workunit.client.0.smithi076.stdout:1/882: dwrite d0/d35/d40/d47/d55/fd2 [0,4194304] 0 2024-03-23T15:45:01.928 INFO:tasks.workunit.client.0.smithi076.stdout:1/883: symlink d0/d35/d40/d47/d55/d100/l11d 0 2024-03-23T15:45:01.929 INFO:tasks.workunit.client.0.smithi076.stdout:1/884: creat d0/d2/dfc/d110/f11e x:0 0 0 2024-03-23T15:45:01.929 INFO:tasks.workunit.client.0.smithi076.stdout:1/885: readlink d0/d2/d91/l10 0 2024-03-23T15:45:01.930 INFO:tasks.workunit.client.0.smithi076.stdout:1/886: creat d0/d2/d91/d86/f11f x:0 0 0 2024-03-23T15:45:01.934 INFO:tasks.workunit.client.0.smithi076.stdout:8/971: sync 2024-03-23T15:45:01.934 INFO:tasks.workunit.client.0.smithi076.stdout:3/871: sync 2024-03-23T15:45:01.934 INFO:tasks.workunit.client.0.smithi076.stdout:6/830: sync 2024-03-23T15:45:01.935 INFO:tasks.workunit.client.0.smithi076.stdout:3/872: write d5/d17/d1c/f11a [1009340,35811] 0 2024-03-23T15:45:01.935 INFO:tasks.workunit.client.0.smithi076.stdout:8/972: mknod d8/d6b/d91/c143 0 2024-03-23T15:45:01.935 INFO:tasks.workunit.client.0.smithi076.stdout:3/873: symlink d5/d17/dcf/de2/dd7/l12a 0 2024-03-23T15:45:01.936 INFO:tasks.workunit.client.0.smithi076.stdout:8/973: mkdir d8/d2e/d144 0 2024-03-23T15:45:01.936 INFO:tasks.workunit.client.0.smithi076.stdout:8/974: write d8/d26/d27/d56/fba [501000,39679] 0 2024-03-23T15:45:01.937 INFO:tasks.workunit.client.0.smithi076.stdout:8/975: truncate d8/d26/d27/d56/f8b 910148 0 2024-03-23T15:45:01.951 INFO:tasks.workunit.client.0.smithi076.stdout:1/887: dwrite d0/f26 [4194304,4194304] 0 2024-03-23T15:45:01.952 INFO:tasks.workunit.client.0.smithi076.stdout:1/888: creat d0/d35/d40/d47/d74/d83/d8f/f120 x:0 0 0 2024-03-23T15:45:01.954 INFO:tasks.workunit.client.0.smithi076.stdout:1/889: mknod d0/d3c/d6e/dc4/c121 0 2024-03-23T15:45:01.956 INFO:tasks.workunit.client.0.smithi076.stdout:1/890: creat d0/d3c/d6e/dc4/f122 x:0 0 0 2024-03-23T15:45:01.956 INFO:tasks.workunit.client.0.smithi076.stdout:6/831: dwrite d1/d81/fe3 [0,4194304] 0 2024-03-23T15:45:01.956 INFO:tasks.workunit.client.0.smithi076.stdout:6/832: chown d1/d4/d10 56 1 2024-03-23T15:45:01.956 INFO:tasks.workunit.client.0.smithi076.stdout:3/874: dwrite d5/d35/d6d/f99 [0,4194304] 0 2024-03-23T15:45:01.958 INFO:tasks.workunit.client.0.smithi076.stdout:6/833: stat d1/dd/df/dac 0 2024-03-23T15:45:01.959 INFO:tasks.workunit.client.0.smithi076.stdout:6/834: mknod d1/dd/d2a/dfd/d113/db3/d8d/d60/c12b 0 2024-03-23T15:45:01.959 INFO:tasks.workunit.client.0.smithi076.stdout:6/835: chown d1/dd/d2a/dfd/d113/db3/c70 12554 1 2024-03-23T15:45:01.963 INFO:tasks.workunit.client.0.smithi076.stdout:8/976: dwrite d8/d6b/d8e/f92 [0,4194304] 0 2024-03-23T15:45:01.964 INFO:tasks.workunit.client.0.smithi076.stdout:3/875: dread d5/d17/d1c/f1d [0,4194304] 0 2024-03-23T15:45:01.965 INFO:tasks.workunit.client.0.smithi076.stdout:8/977: creat d8/d26/d33/d8c/dff/f145 x:0 0 0 2024-03-23T15:45:01.965 INFO:tasks.workunit.client.0.smithi076.stdout:8/978: write d8/d26/d33/f3a [4960287,96067] 0 2024-03-23T15:45:01.966 INFO:tasks.workunit.client.0.smithi076.stdout:3/876: mknod d5/c12b 0 2024-03-23T15:45:01.969 INFO:tasks.workunit.client.0.smithi076.stdout:1/891: dwrite d0/f26 [4194304,4194304] 0 2024-03-23T15:45:01.971 INFO:tasks.workunit.client.0.smithi076.stdout:8/979: unlink d8/d2e/c129 0 2024-03-23T15:45:01.972 INFO:tasks.workunit.client.0.smithi076.stdout:3/877: creat d5/d35/d10e/f12c x:0 0 0 2024-03-23T15:45:01.975 INFO:tasks.workunit.client.0.smithi076.stdout:1/892: rename d0/d3c/d6e/dca to d0/d35/d123 0 2024-03-23T15:45:01.976 INFO:tasks.workunit.client.0.smithi076.stdout:1/893: mkdir d0/d35/d40/d47/d74/d124 0 2024-03-23T15:45:01.977 INFO:tasks.workunit.client.0.smithi076.stdout:8/980: mkdir d8/dd0/df0/d146 0 2024-03-23T15:45:01.977 INFO:tasks.workunit.client.0.smithi076.stdout:8/981: chown d8/d2e/d3e/f8d 146 1 2024-03-23T15:45:01.977 INFO:tasks.workunit.client.0.smithi076.stdout:1/894: mkdir d0/d35/d40/d47/d74/d125 0 2024-03-23T15:45:01.978 INFO:tasks.workunit.client.0.smithi076.stdout:1/895: mknod d0/d35/d40/d47/d74/d83/dbd/d111/c126 0 2024-03-23T15:45:01.979 INFO:tasks.workunit.client.0.smithi076.stdout:1/896: symlink d0/d35/d40/d47/d55/l127 0 2024-03-23T15:45:01.980 INFO:tasks.workunit.client.0.smithi076.stdout:3/878: rename d5/d35/d6d/lff to d5/d35/d4c/l12d 0 2024-03-23T15:45:01.982 INFO:tasks.workunit.client.0.smithi076.stdout:3/879: truncate d5/fd 7300663 0 2024-03-23T15:45:01.984 INFO:tasks.workunit.client.0.smithi076.stdout:6/836: rmdir d1/dd 39 2024-03-23T15:45:01.985 INFO:tasks.workunit.client.0.smithi076.stdout:8/982: rmdir d8/d4f/d80/da3 39 2024-03-23T15:45:01.990 INFO:tasks.workunit.client.0.smithi076.stdout:8/983: truncate d8/d26/f4d 3065632 0 2024-03-23T15:45:01.997 INFO:tasks.workunit.client.0.smithi076.stdout:8/984: creat d8/d2e/f147 x:0 0 0 2024-03-23T15:45:01.998 INFO:tasks.workunit.client.0.smithi076.stdout:1/897: dwrite d0/d2/d70/ffa [0,4194304] 0 2024-03-23T15:45:01.999 INFO:tasks.workunit.client.0.smithi076.stdout:8/985: mkdir d8/d26/d27/d148 0 2024-03-23T15:45:01.999 INFO:tasks.workunit.client.0.smithi076.stdout:8/986: fsync d8/d83/f95 0 2024-03-23T15:45:01.999 INFO:tasks.workunit.client.0.smithi076.stdout:8/987: stat d8/d2e/d3e/f44 0 2024-03-23T15:45:02.000 INFO:tasks.workunit.client.0.smithi076.stdout:1/898: creat d0/d35/d40/d47/d74/d124/f128 x:0 0 0 2024-03-23T15:45:02.000 INFO:tasks.workunit.client.0.smithi076.stdout:3/880: dread d5/d17/f104 [0,4194304] 0 2024-03-23T15:45:02.002 INFO:tasks.workunit.client.0.smithi076.stdout:3/881: mknod d5/d17/d1c/d32/d70/dfa/c12e 0 2024-03-23T15:45:02.003 INFO:tasks.workunit.client.0.smithi076.stdout:8/988: mkdir d8/d26/d33/d149 0 2024-03-23T15:45:02.004 INFO:tasks.workunit.client.0.smithi076.stdout:3/882: creat d5/d35/f12f x:0 0 0 2024-03-23T15:45:02.006 INFO:tasks.workunit.client.0.smithi076.stdout:8/989: rename d8/dd0/ce7 to d8/dd0/df0/d146/c14a 0 2024-03-23T15:45:02.008 INFO:tasks.workunit.client.0.smithi076.stdout:3/883: link d5/d17/d1c/d31/d61/d92/d10a/fcb d5/d17/d1c/d31/d61/d96/f130 0 2024-03-23T15:45:02.010 INFO:tasks.workunit.client.0.smithi076.stdout:3/884: creat d5/d17/d1c/d32/d39/d59/d79/ddc/f131 x:0 0 0 2024-03-23T15:45:02.016 INFO:tasks.workunit.client.0.smithi076.stdout:1/899: dwrite d0/d35/f64 [0,4194304] 0 2024-03-23T15:45:02.016 INFO:tasks.workunit.client.0.smithi076.stdout:1/900: read - d0/d35/d40/d47/d55/fc6 zero size 2024-03-23T15:45:02.017 INFO:tasks.workunit.client.0.smithi076.stdout:1/901: mkdir d0/d35/d40/d47/d55/d129 0 2024-03-23T15:45:02.024 INFO:tasks.workunit.client.0.smithi076.stdout:8/990: dwrite d8/d4f/d80/db7/fd8 [0,4194304] 0 2024-03-23T15:45:02.027 INFO:tasks.workunit.client.0.smithi076.stdout:1/902: getdents d0/d35/d40/d47 0 2024-03-23T15:45:02.027 INFO:tasks.workunit.client.0.smithi076.stdout:1/903: chown d0/d3c/d6e/f76 1 1 2024-03-23T15:45:02.031 INFO:tasks.workunit.client.0.smithi076.stdout:8/991: truncate d8/d2e/d30/f61 2773453 0 2024-03-23T15:45:02.041 INFO:tasks.workunit.client.0.smithi076.stdout:1/904: dwrite d0/d2/f48 [4194304,4194304] 0 2024-03-23T15:45:02.042 INFO:tasks.workunit.client.0.smithi076.stdout:8/992: write d8/d26/d27/f9e [1500360,105591] 0 2024-03-23T15:45:02.042 INFO:tasks.workunit.client.0.smithi076.stdout:8/993: stat d8/d2e/d30/f96 0 2024-03-23T15:45:02.042 INFO:tasks.workunit.client.0.smithi076.stdout:1/905: write d0/d35/d40/d47/d17/f30 [5080091,19099] 0 2024-03-23T15:45:02.042 INFO:tasks.workunit.client.0.smithi076.stdout:8/994: write d8/d26/d27/d56/fd6 [187385,85420] 0 2024-03-23T15:45:02.043 INFO:tasks.workunit.client.0.smithi076.stdout:8/995: mknod d8/d2e/d3e/deb/c14b 0 2024-03-23T15:45:02.043 INFO:tasks.workunit.client.0.smithi076.stdout:8/996: read d8/f1d [3321399,56306] 0 2024-03-23T15:45:02.052 INFO:tasks.workunit.client.0.smithi076.stdout:8/997: write d8/d26/d33/d8c/fee [1946535,64416] 0 2024-03-23T15:45:02.053 INFO:tasks.workunit.client.0.smithi076.stdout:1/906: dwrite d0/d2/d70/da1/dfe/f102 [0,4194304] 0 2024-03-23T15:45:02.059 INFO:tasks.workunit.client.0.smithi076.stdout:8/998: write d8/d26/d27/d56/f8b [1610952,51644] 0 2024-03-23T15:45:02.069 INFO:tasks.workunit.client.0.smithi076.stdout:1/907: rename d0/d35/d40/d47/d17/f1d to d0/d3c/d6e/dc4/f12a 0 2024-03-23T15:45:02.070 INFO:tasks.workunit.client.0.smithi076.stdout:1/908: creat d0/d35/d40/d47/d17/d20/f12b x:0 0 0 2024-03-23T15:45:02.070 INFO:tasks.workunit.client.0.smithi076.stdout:1/909: truncate d0/d35/d40/d47/d17/f30 970782 0 2024-03-23T15:45:02.072 INFO:tasks.workunit.client.0.smithi076.stdout:1/910: truncate d0/d2/d70/fb6 76246 0 2024-03-23T15:45:02.072 INFO:tasks.workunit.client.0.smithi076.stdout:1/911: fsync d0/d35/f6f 0 2024-03-23T15:45:02.073 INFO:tasks.workunit.client.0.smithi076.stdout:1/912: truncate d0/d3c/d6e/fe5 340088 0 2024-03-23T15:45:02.073 INFO:tasks.workunit.client.0.smithi076.stdout:1/913: chown d0/d2/fc7 51897 1 2024-03-23T15:45:02.073 INFO:tasks.workunit.client.0.smithi076.stdout:1/914: chown d0/d2/d70/fb6 9355 1 2024-03-23T15:45:02.172 INFO:tasks.workunit.client.0.smithi076.stdout:8/999: sync 2024-03-23T15:45:02.173 INFO:tasks.workunit.client.0.smithi076.stdout:3/885: sync 2024-03-23T15:45:02.173 INFO:tasks.workunit.client.0.smithi076.stdout:1/915: sync 2024-03-23T15:45:02.173 INFO:tasks.workunit.client.0.smithi076.stdout:6/837: sync 2024-03-23T15:45:02.173 INFO:tasks.workunit.client.0.smithi076.stdout:3/886: fsync d5/d35/d4c/d5a/fbd 0 2024-03-23T15:45:02.173 INFO:tasks.workunit.client.0.smithi076.stdout:1/916: read - d0/d35/f10b zero size 2024-03-23T15:45:02.173 INFO:tasks.workunit.client.0.smithi076.stdout:3/887: fdatasync d5/fb1 0 2024-03-23T15:45:02.175 INFO:tasks.workunit.client.0.smithi076.stdout:1/917: truncate d0/d2/d70/da1/fec 6198910 0 2024-03-23T15:45:02.175 INFO:tasks.workunit.client.0.smithi076.stdout:6/838: creat d1/dd/d2a/dfd/f12c x:0 0 0 2024-03-23T15:45:02.175 INFO:tasks.workunit.client.0.smithi076.stdout:3/888: symlink d5/d17/d1c/d32/d70/d8c/l132 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/889: fsync d5/d17/d1c/d32/d8e/fe4 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/890: stat d5/d17/d1c/d32/d39/d59/d79/ddc/f10c 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/891: write d5/dc2/ff0 [55778,451] 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:3/892: readlink d5/d17/dcf/de2/dd7/l12a 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:1/918: mkdir d0/d35/d40/d47/d74/d124/d12c 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:1/919: fsync d0/d35/d40/d47/d55/f56 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:6/839: mkdir d1/dd/d2a/d12d 0 2024-03-23T15:45:02.176 INFO:tasks.workunit.client.0.smithi076.stdout:6/840: readlink d1/d2/laf 0 2024-03-23T15:45:02.177 INFO:tasks.workunit.client.0.smithi076.stdout:6/841: creat d1/dd/df/d5b/d62/d128/f12e x:0 0 0 2024-03-23T15:45:02.179 INFO:tasks.workunit.client.0.smithi076.stdout:3/893: write d5/d17/d1c/f2b [1779944,120526] 0 2024-03-23T15:45:02.180 INFO:tasks.workunit.client.0.smithi076.stdout:3/894: rmdir d5/d35/d3c 39 2024-03-23T15:45:02.182 INFO:tasks.workunit.client.0.smithi076.stdout:3/895: creat d5/d35/d4c/d5a/d11d/f133 x:0 0 0 2024-03-23T15:45:02.183 INFO:tasks.workunit.client.0.smithi076.stdout:3/896: mkdir d5/d17/dcf/de2/d134 0 2024-03-23T15:45:02.192 INFO:tasks.workunit.client.0.smithi076.stdout:1/920: rename d0/d2/d70/d7a/d8b/dcb to d0/d35/d40/d47/d8c/dd6/d112/d12d 0 2024-03-23T15:45:02.195 INFO:tasks.workunit.client.0.smithi076.stdout:1/921: creat d0/d2/d62/f12e x:0 0 0 2024-03-23T15:45:02.195 INFO:tasks.workunit.client.0.smithi076.stdout:6/842: rename d1/dd/df/d5b/f99 to d1/dd/d2a/d12d/f12f 0 2024-03-23T15:45:02.200 INFO:tasks.workunit.client.0.smithi076.stdout:3/897: dwrite d5/d35/d3c/fb5 [0,4194304] 0 2024-03-23T15:45:02.202 INFO:tasks.workunit.client.0.smithi076.stdout:3/898: symlink d5/l135 0 2024-03-23T15:45:02.204 INFO:tasks.workunit.client.0.smithi076.stdout:3/899: truncate d5/d17/f29 5555763 0 2024-03-23T15:45:02.207 INFO:tasks.workunit.client.0.smithi076.stdout:1/922: dwrite d0/d35/d40/d47/d8c/fc5 [0,4194304] 0 2024-03-23T15:45:02.209 INFO:tasks.workunit.client.0.smithi076.stdout:1/923: mknod d0/d35/d40/d47/d8c/dd6/c12f 0 2024-03-23T15:45:02.209 INFO:tasks.workunit.client.0.smithi076.stdout:1/924: read d0/d35/d40/f90 [2273701,87363] 0 2024-03-23T15:45:02.210 INFO:tasks.workunit.client.0.smithi076.stdout:1/925: dread d0/d35/d40/d47/d8c/d99/ff1 [0,4194304] 0 2024-03-23T15:45:02.210 INFO:tasks.workunit.client.0.smithi076.stdout:1/926: fdatasync d0/d2/d70/da1/dbf/f117 0 2024-03-23T15:45:02.211 INFO:tasks.workunit.client.0.smithi076.stdout:6/843: dwrite d1/dd/f26 [4194304,4194304] 0 2024-03-23T15:45:02.212 INFO:tasks.workunit.client.0.smithi076.stdout:6/844: rmdir d1/dd/df/d5b/dcd/d10f/d114 39 2024-03-23T15:45:02.214 INFO:tasks.workunit.client.0.smithi076.stdout:3/900: write d5/d17/d1c/d32/d39/d59/f88 [1594715,30097] 0 2024-03-23T15:45:02.216 INFO:tasks.workunit.client.0.smithi076.stdout:3/901: creat d5/d17/dcf/f136 x:0 0 0 2024-03-23T15:45:02.217 INFO:tasks.workunit.client.0.smithi076.stdout:3/902: mkdir d5/d17/d1c/d31/d61/d137 0 2024-03-23T15:45:02.218 INFO:tasks.workunit.client.0.smithi076.stdout:3/903: getdents d5/d17/d1c/d32/d39/d59/d79/ddc 0 2024-03-23T15:45:02.218 INFO:tasks.workunit.client.0.smithi076.stdout:3/904: chown d5/d35/c38 7 1 2024-03-23T15:45:02.221 INFO:tasks.workunit.client.0.smithi076.stdout:3/905: creat d5/d17/dcf/de2/dd7/f138 x:0 0 0 2024-03-23T15:45:02.221 INFO:tasks.workunit.client.0.smithi076.stdout:3/906: write d5/d35/d4c/d5a/f49 [1878807,99590] 0 2024-03-23T15:45:02.229 INFO:tasks.workunit.client.0.smithi076.stdout:1/927: link d0/d3c/d6e/dc4/f122 d0/d35/d40/d47/d8c/dd6/f130 0 2024-03-23T15:45:02.231 INFO:tasks.workunit.client.0.smithi076.stdout:1/928: rmdir d0/d35/d40 39 2024-03-23T15:45:02.231 INFO:tasks.workunit.client.0.smithi076.stdout:1/929: fsync d0/d3c/d6e/f76 0 2024-03-23T15:45:02.232 INFO:tasks.workunit.client.0.smithi076.stdout:1/930: chown d0/c3f 87047 1 2024-03-23T15:45:02.232 INFO:tasks.workunit.client.0.smithi076.stdout:1/931: chown d0/d35/f64 0 1 2024-03-23T15:45:02.233 INFO:tasks.workunit.client.0.smithi076.stdout:1/932: getdents d0/d2/d91 0 2024-03-23T15:45:02.233 INFO:tasks.workunit.client.0.smithi076.stdout:6/845: dwrite d1/dd/df/d5b/f55 [0,4194304] 0 2024-03-23T15:45:02.234 INFO:tasks.workunit.client.0.smithi076.stdout:1/933: symlink d0/d35/d40/d47/d74/d83/dbd/d111/l131 0 2024-03-23T15:45:02.235 INFO:tasks.workunit.client.0.smithi076.stdout:1/934: chown d0/d2/d70/da1/dbf/f117 43811 1 2024-03-23T15:45:02.235 INFO:tasks.workunit.client.0.smithi076.stdout:1/935: truncate d0/d35/f10b 581793 0 2024-03-23T15:45:02.236 INFO:tasks.workunit.client.0.smithi076.stdout:3/907: dwrite d5/d17/d1c/d32/d8e/faf [0,4194304] 0 2024-03-23T15:45:02.237 INFO:tasks.workunit.client.0.smithi076.stdout:3/908: mknod d5/d17/d1c/d31/d61/d137/c139 0 2024-03-23T15:45:02.238 INFO:tasks.workunit.client.0.smithi076.stdout:6/846: symlink d1/dd/df/d5b/dcd/l130 0 2024-03-23T15:45:02.241 INFO:tasks.workunit.client.0.smithi076.stdout:6/847: read d1/d4/d10/fe6 [180625,108474] 0 2024-03-23T15:45:02.241 INFO:tasks.workunit.client.0.smithi076.stdout:1/936: dread d0/d35/d40/d47/d8c/d99/f116 [0,4194304] 0 2024-03-23T15:45:02.241 INFO:tasks.workunit.client.0.smithi076.stdout:3/909: unlink d5/d17/d1c/d32/d4a/fa9 0 2024-03-23T15:45:02.241 INFO:tasks.workunit.client.0.smithi076.stdout:3/910: fsync d5/ffb 0 2024-03-23T15:45:02.242 INFO:tasks.workunit.client.0.smithi076.stdout:1/937: mknod d0/d2/d70/da1/dbf/d107/c132 0 2024-03-23T15:45:02.242 INFO:tasks.workunit.client.0.smithi076.stdout:1/938: stat d0/d2/f37 0 2024-03-23T15:45:02.243 INFO:tasks.workunit.client.0.smithi076.stdout:1/939: creat d0/d2/d62/ddd/f133 x:0 0 0 2024-03-23T15:45:02.246 INFO:tasks.workunit.client.0.smithi076.stdout:1/940: creat d0/d35/d40/d47/d8c/dd6/d112/d12d/f134 x:0 0 0 2024-03-23T15:45:02.248 INFO:tasks.workunit.client.0.smithi076.stdout:3/911: write d5/d17/d1c/d32/d39/f75 [2436495,90288] 0 2024-03-23T15:45:02.250 INFO:tasks.workunit.client.0.smithi076.stdout:3/912: stat d5/d35/d4c/d5a/d9d/fed 0 2024-03-23T15:45:02.252 INFO:tasks.workunit.client.0.smithi076.stdout:3/913: creat d5/d17/d1c/d32/d70/dfa/f13a x:0 0 0 2024-03-23T15:45:02.255 INFO:tasks.workunit.client.0.smithi076.stdout:1/941: dread d0/d3c/d6e/f76 [0,4194304] 0 2024-03-23T15:45:02.256 INFO:tasks.workunit.client.0.smithi076.stdout:6/848: dwrite d1/dd/df/fa4 [0,4194304] 0 2024-03-23T15:45:02.263 INFO:tasks.workunit.client.0.smithi076.stdout:1/942: write d0/d35/d40/d47/d74/d83/f9c [1020683,122521] 0 2024-03-23T15:45:02.263 INFO:tasks.workunit.client.0.smithi076.stdout:1/943: read d0/d35/d40/d47/d74/d83/f9c [1438106,47129] 0 2024-03-23T15:45:02.263 INFO:tasks.workunit.client.0.smithi076.stdout:6/849: dread d1/dd/d2a/d23/faa [0,4194304] 0 2024-03-23T15:45:02.265 INFO:tasks.workunit.client.0.smithi076.stdout:6/850: unlink d1/d4/d10/f74 0 2024-03-23T15:45:02.272 INFO:tasks.workunit.client.0.smithi076.stdout:3/914: dwrite d5/d17/d1c/d32/d39/f6c [0,4194304] 0 2024-03-23T15:45:02.273 INFO:tasks.workunit.client.0.smithi076.stdout:1/944: rename d0/d35/d40/d47/d74 to d0/d35/d40/d47/d55/d135 0 2024-03-23T15:45:02.279 INFO:tasks.workunit.client.0.smithi076.stdout:1/945: unlink d0/d35/d40/d47/d17/d20/f3e 0 2024-03-23T15:45:02.279 INFO:tasks.workunit.client.0.smithi076.stdout:1/946: write d0/d2/d70/d7a/fea [11962,61517] 0 2024-03-23T15:45:02.279 INFO:tasks.workunit.client.0.smithi076.stdout:1/947: chown d0/f26 607967080 1 2024-03-23T15:45:02.279 INFO:tasks.workunit.client.0.smithi076.stdout:1/948: stat d0/d35/d40/d47/d17/f54 0 2024-03-23T15:45:02.281 INFO:tasks.workunit.client.0.smithi076.stdout:6/851: link d1/dd/df/d5b/d62/d128/f12e d1/d4/f131 0 2024-03-23T15:45:02.282 INFO:tasks.workunit.client.0.smithi076.stdout:1/949: mkdir d0/d2/dfc/d136 0 2024-03-23T15:45:02.283 INFO:tasks.workunit.client.0.smithi076.stdout:6/852: rmdir d1/dd/df/d5b 39 2024-03-23T15:45:02.283 INFO:tasks.workunit.client.0.smithi076.stdout:6/853: chown d1/dd/d2a/dfd/d113/db3/d8d/d60/c12b 0 1 2024-03-23T15:45:02.284 INFO:tasks.workunit.client.0.smithi076.stdout:1/950: creat d0/d35/d40/d47/d8c/dd6/d112/f137 x:0 0 0 2024-03-23T15:45:02.284 INFO:tasks.workunit.client.0.smithi076.stdout:1/951: fsync d0/f4b 0 2024-03-23T15:45:02.284 INFO:tasks.workunit.client.0.smithi076.stdout:6/854: mknod d1/d2/df2/c132 0 2024-03-23T15:45:02.284 INFO:tasks.workunit.client.0.smithi076.stdout:3/915: dwrite d5/d35/d3c/d67/fbc [4194304,4194304] 0 2024-03-23T15:45:02.285 INFO:tasks.workunit.client.0.smithi076.stdout:6/855: symlink d1/dd/d2a/dfd/l133 0 2024-03-23T15:45:02.285 INFO:tasks.workunit.client.0.smithi076.stdout:3/916: creat d5/d35/d4c/de1/f13b x:0 0 0 2024-03-23T15:45:02.288 INFO:tasks.workunit.client.0.smithi076.stdout:6/856: creat d1/dd/df/d5b/dcd/d10f/d114/f134 x:0 0 0 2024-03-23T15:45:02.288 INFO:tasks.workunit.client.0.smithi076.stdout:6/857: chown d1/d4/d10/l50 12716 1 2024-03-23T15:45:02.288 INFO:tasks.workunit.client.0.smithi076.stdout:3/917: mkdir d5/d17/dcf/de2/d13c 0 2024-03-23T15:45:02.288 INFO:tasks.workunit.client.0.smithi076.stdout:6/858: mknod d1/dd/d2a/dfd/d113/db3/d8d/d60/c135 0 2024-03-23T15:45:02.288 INFO:tasks.workunit.client.0.smithi076.stdout:3/918: rmdir d5/d17/dcf/de2/d134 0 2024-03-23T15:45:02.289 INFO:tasks.workunit.client.0.smithi076.stdout:3/919: getdents d5/d17/d1c/d32 0 2024-03-23T15:45:02.291 INFO:tasks.workunit.client.0.smithi076.stdout:3/920: mkdir d5/d17/d1c/d32/d70/d13d 0 2024-03-23T15:45:02.292 INFO:tasks.workunit.client.0.smithi076.stdout:3/921: mknod d5/d17/d1c/d32/d39/d59/d79/d114/d116/c13e 0 2024-03-23T15:45:02.292 INFO:tasks.workunit.client.0.smithi076.stdout:3/922: readlink d5/d17/d1c/d31/d61/d92/d10a/lf2 0 2024-03-23T15:45:02.292 INFO:tasks.workunit.client.0.smithi076.stdout:3/923: chown d5/d17/d1c/d31/d47/cdd 2961493 1 2024-03-23T15:45:02.293 INFO:tasks.workunit.client.0.smithi076.stdout:3/924: fsync d5/f73 0 2024-03-23T15:45:02.294 INFO:tasks.workunit.client.0.smithi076.stdout:3/925: mkdir d5/d17/d1c/d31/d61/d92/d10a/d13f 0 2024-03-23T15:45:02.298 INFO:tasks.workunit.client.0.smithi076.stdout:6/859: dread d1/d4/d10/f4a [0,4194304] 0 2024-03-23T15:45:02.298 INFO:tasks.workunit.client.0.smithi076.stdout:6/860: symlink d1/dd/d2a/l136 0 2024-03-23T15:45:02.300 INFO:tasks.workunit.client.0.smithi076.stdout:1/952: dwrite d0/d35/d40/d47/d8c/dd6/d112/d12d/fef [0,4194304] 0 2024-03-23T15:45:02.305 INFO:tasks.workunit.client.0.smithi076.stdout:1/953: dread d0/d35/d40/fa8 [0,4194304] 0 2024-03-23T15:45:02.306 INFO:tasks.workunit.client.0.smithi076.stdout:1/954: stat d0/d2/c29 0 2024-03-23T15:45:02.306 INFO:tasks.workunit.client.0.smithi076.stdout:1/955: write d0/d2/d62/fa7 [90920,16662] 0 2024-03-23T15:45:02.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-23T15:45:02.307 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-23T15:45:02.314 INFO:tasks.workunit.client.0.smithi076.stdout:3/926: dwrite d5/f43 [0,4194304] 0 2024-03-23T15:45:02.317 INFO:tasks.workunit.client.0.smithi076.stdout:3/927: mknod d5/d35/d10e/c140 0 2024-03-23T15:45:02.317 INFO:tasks.workunit.client.0.smithi076.stdout:3/928: fdatasync d5/d17/d1c/d32/d70/dfa/f13a 0 2024-03-23T15:45:02.317 INFO:tasks.workunit.client.0.smithi076.stdout:3/929: chown d5/d17/d1c/d31/d61/fb8 131788 1 2024-03-23T15:45:02.318 INFO:tasks.workunit.client.0.smithi076.stdout:6/861: dwrite d1/dd/df/fb2 [0,4194304] 0 2024-03-23T15:45:02.318 INFO:tasks.workunit.client.0.smithi076.stdout:6/862: write d1/d4/fcf [69188,90441] 0 2024-03-23T15:45:02.325 INFO:tasks.workunit.client.0.smithi076.stdout:3/930: truncate d5/d17/d1c/d32/d39/d59/f88 1976358 0 2024-03-23T15:45:02.330 INFO:tasks.workunit.client.0.smithi076.stdout:1/956: dwrite d0/d2/d70/da1/dbf/fd3 [0,4194304] 0 2024-03-23T15:45:02.331 INFO:tasks.workunit.client.0.smithi076.stdout:6/863: dwrite d1/dd/df/d5b/fa2 [0,4194304] 0 2024-03-23T15:45:02.335 INFO:tasks.workunit.client.0.smithi076.stdout:6/864: symlink d1/dd/df/l137 0 2024-03-23T15:45:02.337 INFO:tasks.workunit.client.0.smithi076.stdout:6/865: rmdir d1/dd/df/dce/df0 39 2024-03-23T15:45:02.339 INFO:tasks.workunit.client.0.smithi076.stdout:3/931: dwrite d5/d17/d1c/d32/d39/f3a [0,4194304] 0 2024-03-23T15:45:02.342 INFO:tasks.workunit.client.0.smithi076.stdout:3/932: stat d5/d17/d1c/d32/d39/d59/f88 0 2024-03-23T15:45:02.342 INFO:tasks.workunit.client.0.smithi076.stdout:6/866: unlink d1/dd/d2a/dfd/d113/db3/c70 0 2024-03-23T15:45:02.345 INFO:tasks.workunit.client.0.smithi076.stdout:3/933: link d5/d17/d1c/d32/d39/ld5 d5/d17/d1c/d32/d39/d59/d79/d114/l141 0 2024-03-23T15:45:02.347 INFO:tasks.workunit.client.0.smithi076.stdout:3/934: unlink d5/d35/d4c/d5a/f41 0 2024-03-23T15:45:02.351 INFO:tasks.workunit.client.0.smithi076.stdout:3/935: unlink d5/d17/d1c/lc3 0 2024-03-23T15:45:02.353 INFO:tasks.workunit.client.0.smithi076.stdout:1/957: rename d0/f9d to d0/d35/d40/d47/d8c/d99/f138 0 2024-03-23T15:45:02.354 INFO:tasks.workunit.client.0.smithi076.stdout:1/958: creat d0/d35/d40/d47/d55/d135/d124/f139 x:0 0 0 2024-03-23T15:45:02.369 INFO:tasks.workunit.client.0.smithi076.stdout:3/936: dwrite d5/d35/d6d/fbe [0,4194304] 0 2024-03-23T15:45:02.370 INFO:tasks.workunit.client.0.smithi076.stdout:1/959: dwrite d0/d35/d40/d47/d17/d20/f33 [0,4194304] 0 2024-03-23T15:45:02.370 INFO:tasks.workunit.client.0.smithi076.stdout:1/960: fdatasync d0/d35/d40/d47/d17/f30 0 2024-03-23T15:45:02.370 INFO:tasks.workunit.client.0.smithi076.stdout:3/937: creat d5/d35/d4c/de1/f142 x:0 0 0 2024-03-23T15:45:02.371 INFO:tasks.workunit.client.0.smithi076.stdout:3/938: write d5/d17/dcf/f136 [897583,96731] 0 2024-03-23T15:45:02.371 INFO:tasks.workunit.client.0.smithi076.stdout:1/961: creat d0/d35/d40/f13a x:0 0 0 2024-03-23T15:45:02.372 INFO:tasks.workunit.client.0.smithi076.stdout:3/939: rmdir d5/d17/dcf/de2/de7 39 2024-03-23T15:45:02.372 INFO:tasks.workunit.client.0.smithi076.stdout:1/962: mknod d0/d2/d62/ddd/c13b 0 2024-03-23T15:45:02.373 INFO:tasks.workunit.client.0.smithi076.stdout:1/963: fdatasync d0/d2/f37 0 2024-03-23T15:45:02.374 INFO:tasks.workunit.client.0.smithi076.stdout:3/940: rename d5/d17/l11f to d5/d17/d1c/d32/d70/l143 0 2024-03-23T15:45:02.374 INFO:tasks.workunit.client.0.smithi076.stdout:3/941: stat d5/d17/d1c/d32/d39/c84 0 2024-03-23T15:45:02.374 INFO:tasks.workunit.client.0.smithi076.stdout:3/942: truncate d5/d35/d6d/fa7 554109 0 2024-03-23T15:45:02.375 INFO:tasks.workunit.client.0.smithi076.stdout:3/943: dread d5/d17/d1c/d32/d39/d59/fd6 [0,4194304] 0 2024-03-23T15:45:02.380 INFO:tasks.workunit.client.0.smithi076.stdout:3/944: dread d5/d35/d6d/d7f/fae [0,4194304] 0 2024-03-23T15:45:02.385 INFO:tasks.workunit.client.0.smithi076.stdout:1/964: dwrite d0/d35/d40/d47/d8c/fc5 [0,4194304] 0 2024-03-23T15:45:02.385 INFO:tasks.workunit.client.0.smithi076.stdout:1/965: fsync d0/d35/d40/d47/d17/d20/fd4 0 2024-03-23T15:45:02.387 INFO:tasks.workunit.client.0.smithi076.stdout:1/966: dread d0/d35/d40/d47/d17/f30 [0,4194304] 0 2024-03-23T15:45:02.388 INFO:tasks.workunit.client.0.smithi076.stdout:1/967: creat d0/d2/d91/d86/f13c x:0 0 0 2024-03-23T15:45:02.401 INFO:tasks.workunit.client.0.smithi076.stdout:3/945: rename d5/d17/d1c/d32/f97 to d5/d17/d1c/d32/d4a/f144 0 2024-03-23T15:45:02.402 INFO:tasks.workunit.client.0.smithi076.stdout:3/946: getdents d5/d17/d1c/d32/d39/d59/d79/d114 0 2024-03-23T15:45:02.403 INFO:tasks.workunit.client.0.smithi076.stdout:3/947: mkdir d5/d17/dcf/de2/d145 0 2024-03-23T15:45:02.404 INFO:tasks.workunit.client.0.smithi076.stdout:3/948: chown d5/d17/f29 6 1 2024-03-23T15:45:02.404 INFO:tasks.workunit.client.0.smithi076.stdout:3/949: write d5/d17/d1c/d32/d5d/f127 [726422,105186] 0 2024-03-23T15:45:02.404 INFO:tasks.workunit.client.0.smithi076.stdout:1/968: dwrite d0/d35/d40/d47/d17/d20/f25 [0,4194304] 0 2024-03-23T15:45:02.404 INFO:tasks.workunit.client.0.smithi076.stdout:3/950: mknod d5/d17/d1c/d32/d39/d59/d79/d114/d116/c146 0 2024-03-23T15:45:02.405 INFO:tasks.workunit.client.0.smithi076.stdout:1/969: getdents d0/d35/d40/d47/d55/d100 0 2024-03-23T15:45:02.405 INFO:tasks.workunit.client.0.smithi076.stdout:3/951: rmdir d5/d17/dcf/de2 39 2024-03-23T15:45:02.419 INFO:tasks.workunit.client.0.smithi076.stdout:3/952: link d5/d35/d4c/d5a/d9d/fed d5/d17/d1c/d31/d61/f147 0 2024-03-23T15:45:02.419 INFO:tasks.workunit.client.0.smithi076.stdout:3/953: readlink d5/d35/d6d/d7f/ldf 0 2024-03-23T15:45:02.420 INFO:tasks.workunit.client.0.smithi076.stdout:3/954: symlink d5/d35/d4c/de1/l148 0 2024-03-23T15:45:02.421 INFO:tasks.workunit.client.0.smithi076.stdout:1/970: link d0/d2/d70/da1/dbf/cff d0/d35/d82/c13d 0 2024-03-23T15:45:02.421 INFO:tasks.workunit.client.0.smithi076.stdout:1/971: chown d0/d2/f75 0 1 2024-03-23T15:45:02.421 INFO:tasks.workunit.client.0.smithi076.stdout:1/972: truncate d0/d2/d70/da1/dbf/f117 310046 0 2024-03-23T15:45:02.422 INFO:tasks.workunit.client.0.smithi076.stdout:1/973: creat d0/d35/d40/d47/d17/d20/f13e x:0 0 0 2024-03-23T15:45:02.425 INFO:tasks.workunit.client.0.smithi076.stdout:3/955: truncate d5/d35/d4c/d5a/d9d/fed 1784179 0 2024-03-23T15:45:02.426 INFO:tasks.workunit.client.0.smithi076.stdout:3/956: symlink d5/d35/d4c/l149 0 2024-03-23T15:45:02.426 INFO:tasks.workunit.client.0.smithi076.stdout:3/957: fsync d5/d35/d10e/f12c 0 2024-03-23T15:45:02.427 INFO:tasks.workunit.client.0.smithi076.stdout:1/974: dread d0/d3c/f6c [0,4194304] 0 2024-03-23T15:45:02.428 INFO:tasks.workunit.client.0.smithi076.stdout:3/958: unlink d5/d35/d6d/cc8 0 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.425+0000 7f9a31cc9640 1 -- 172.21.15.76:0/3641710334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0692b0 msgr2=0x7f9a2c10e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.425+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/3641710334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0692b0 0x7f9a2c10e550 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009cd0 tx=0x7f9a2402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 -- 172.21.15.76:0/3641710334 shutdown_connections 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/3641710334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0692b0 0x7f9a2c10e550 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/3641710334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a2c0688f0 0x7f9a2c068d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/3641710334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c107690 0x7f9a2c107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 -- 172.21.15.76:0/3641710334 >> 172.21.15.76:0/3641710334 conn(0x7f9a2c077710 msgr2=0x7f9a2c077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 -- 172.21.15.76:0/3641710334 shutdown_connections 2024-03-23T15:45:02.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 -- 172.21.15.76:0/3641710334 wait complete. 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 Processor -- start 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.426+0000 7f9a31cc9640 1 -- start start 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a31cc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0688f0 0x7f9a2c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a31cc9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a2c0692b0 0x7f9a2c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a31cc9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c107690 0x7f9a2c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a31cc9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1980d0 con 0x7f9a2c0688f0 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a31cc9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c198240 con 0x7f9a2c0692b0 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a31cc9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c198400 con 0x7f9a2c107690 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0688f0 0x7f9a2c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a23fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a2c0692b0 0x7f9a2c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0688f0 0x7f9a2c19d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35702/0 (socket says 172.21.15.76:35702) 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/2567935620 learned_addr learned my addr 172.21.15.76:0/2567935620 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2bfff640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c107690 0x7f9a2c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/2567935620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c107690 msgr2=0x7f9a2c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c107690 0x7f9a2c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/2567935620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a2c0692b0 msgr2=0x7f9a2c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a2c0692b0 0x7f9a2c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/2567935620 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a24009900 con 0x7f9a2c0688f0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0688f0 0x7f9a2c19d8d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c920 tx=0x7f9a1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a297fa640 1 -- 172.21.15.76:0/2567935620 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c007c70 con 0x7f9a2c0688f0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c1986e0 con 0x7f9a2c0688f0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a297fa640 1 -- 172.21.15.76:0/2567935620 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c007dd0 con 0x7f9a2c0688f0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.427+0000 7f9a297fa640 1 -- 172.21.15.76:0/2567935620 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c018860 con 0x7f9a2c0688f0 2024-03-23T15:45:02.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.428+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a4970 con 0x7f9a2c0688f0 2024-03-23T15:45:02.434 INFO:tasks.workunit.client.0.smithi076.stdout:1/975: mkdir d0/d35/d40/d47/d13f 0 2024-03-23T15:45:02.434 INFO:tasks.workunit.client.0.smithi076.stdout:1/976: stat d0/d2/d91 0 2024-03-23T15:45:02.434 INFO:tasks.workunit.client.0.smithi076.stdout:3/959: write d5/d17/dcf/de2/f8a [3836797,3089] 0 2024-03-23T15:45:02.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.430+0000 7f9a34984740 1 Processor -- start 2024-03-23T15:45:02.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a34984740 1 -- start start 2024-03-23T15:45:02.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a297fa640 1 -- 172.21.15.76:0/2567935620 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f9a1c00fac0 con 0x7f9a2c0688f0 2024-03-23T15:45:02.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a34984740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffaa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a34984740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a34984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a34984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558dcfecef30 con 0x558dcffb7ae0 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a34984740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558dcff18810 con 0x558dcffa9e80 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a34984740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558dcff15d40 con 0x558dcffaa7a0 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffaa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffaa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:45152/0 (socket says 172.21.15.76:45152) 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2bfff640 1 -- 172.21.15.76:0/1018965212 learned_addr learned my addr 172.21.15.76:0/1018965212 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/1018965212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 msgr2=0x558dcffaa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffaa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/1018965212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 msgr2=0x558dcffb9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/1018965212 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dcfed2ea0 con 0x558dcffaa7a0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.431+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a297fa640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a180a2230 0x7f9a180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb75a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0cb8e0 tx=0x7f9a1c0d62d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8956d82d23f81fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a2bfff640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a180a2230 0x7f9a180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a297fa640 1 -- 172.21.15.76:0/2567935620 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9a1c0ce0f0 con 0x7f9a2c0688f0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a21ffb640 1 -- 172.21.15.76:0/1018965212 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c096e20 con 0x558dcffaa7a0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a21ffb640 1 -- 172.21.15.76:0/1018965212 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c09a950 con 0x558dcffaa7a0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a2bfff640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a180a2230 0x7f9a180a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009430 tx=0x7f9a24009800 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.432+0000 7f9a21ffb640 1 -- 172.21.15.76:0/1018965212 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0d1d80 con 0x558dcffaa7a0 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 -- 172.21.15.76:0/1018965212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 msgr2=0x558dcffb75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb75a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0cb8e0 tx=0x7f9a1c0d62d0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 -- 172.21.15.76:0/1018965212 shutdown_connections 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb75a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 --2- 172.21.15.76:0/1018965212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffaa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 -- 172.21.15.76:0/1018965212 >> 172.21.15.76:0/1018965212 conn(0x558dcffa4900 msgr2=0x558dcffa4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 -- 172.21.15.76:0/1018965212 shutdown_connections 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 -- 172.21.15.76:0/1018965212 wait complete. 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.433+0000 7f9a34984740 1 -- start start 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 1 Processor -- start 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffb4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558dcff11520 con 0x558dcffb7ae0 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558dcfbe4430 con 0x558dcffa9e80 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558dcfba0e00 con 0x558dcffaa7a0 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a34984740 10 client.0 _ll_register_callbacks cb 0x558dcfeafd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:02.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffb4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffb4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:45164/0 (socket says 172.21.15.76:45164) 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/3308764918 learned_addr learned my addr 172.21.15.76:0/3308764918 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a23fff640 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2bfff640 1 -- 172.21.15.76:0/3308764918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 msgr2=0x558dcffb5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2bfff640 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2bfff640 1 -- 172.21.15.76:0/3308764918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 msgr2=0x558dcffb4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2bfff640 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffb4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.434+0000 7f9a2bfff640 1 -- 172.21.15.76:0/3308764918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a4970 con 0x558dcffb7ae0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a2bfff640 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb5560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9a24006010 tx=0x7f9a240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da898b338e112ff server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a0affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a0affd640 1 -- 172.21.15.76:0/3308764918 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24003b70 con 0x558dcffb7ae0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a0affd640 1 -- 172.21.15.76:0/3308764918 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9a24003cd0 con 0x558dcffb7ae0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558dcfc107e0 con 0x558dcffb7ae0 2024-03-23T15:45:02.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a0affd640 1 -- 172.21.15.76:0/3308764918 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9a24003e90 con 0x558dcffb7ae0 2024-03-23T15:45:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a34984740 10 client.15906 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.435+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558dcff0fba0 con 0x558dcffb7ae0 2024-03-23T15:45:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a0affd640 1 -- 172.21.15.76:0/3308764918 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558dcfc107e0 con 0x558dcffb7ae0 2024-03-23T15:45:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a0affd640 1 -- 172.21.15.76:0/3308764918 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9a24051020 con 0x558dcffb7ae0 2024-03-23T15:45:02.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a34984740 10 client.15906 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:02.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a34984740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a34984740 20 client.15906 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a34984740 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dcffc8050 0x558dcffca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a34984740 4 client.15906 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.436+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dcfc4d720 con 0x558dcffc8050 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.437+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dcffc8050 0x558dcffca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.438+0000 7f9a0affd640 10 client.15906 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.438+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dcffc8050 0x558dcffca470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a4cf7cdfd19f26 server_cookie=fe24b918c159230d in_seq=0 out_seq=0 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.439+0000 7f9a0affd640 1 -- 172.21.15.76:0/3308764918 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558dcfc4d720 con 0x558dcffc8050 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.439+0000 7f9a0affd640 10 client.15906 handle_command_reply: tid=0 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.439+0000 7f9a34984740 1 client.15906 shutdown 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.439+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 msgr2=0x558dcffb5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.439+0000 7f9a34984740 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb5560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9a24006010 tx=0x7f9a240095c0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.439+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 shutdown_connections 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.439+0000 7f9a34984740 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffb7ae0 0x558dcffb5560 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a34984740 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffb5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a34984740 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dcffc8050 0x558dcffca470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a34984740 1 --2- 172.21.15.76:0/3308764918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffb4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a0affd640 0 client.15906 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 >> 172.21.15.76:0/3308764918 conn(0x558dcffa4900 msgr2=0x558dcffa6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 shutdown_connections 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a34984740 1 -- 172.21.15.76:0/3308764918 wait complete. 2024-03-23T15:45:02.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.440+0000 7f9a34984740 20 client.15906 trim_cache size 0 max 16384 2024-03-23T15:45:02.450 INFO:tasks.workunit.client.0.smithi076.stdout:1/977: dwrite d0/d2/d91/d86/fcc [0,4194304] 0 2024-03-23T15:45:02.451 INFO:tasks.workunit.client.0.smithi076.stdout:1/978: truncate d0/d35/d40/d47/d17/d20/f13e 362605 0 2024-03-23T15:45:02.451 INFO:tasks.workunit.client.0.smithi076.stdout:3/960: dwrite d5/d17/d1c/d32/d70/fe5 [0,4194304] 0 2024-03-23T15:45:02.455 INFO:tasks.workunit.client.0.smithi076.stdout:3/961: creat d5/d35/d6d/d7f/f14a x:0 0 0 2024-03-23T15:45:02.462 INFO:tasks.workunit.client.0.smithi076.stdout:3/962: dread d5/d35/d6d/f99 [0,4194304] 0 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.460+0000 7f9a34984740 1 Processor -- start 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.460+0000 7f9a34984740 1 -- start start 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a34984740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffc3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a34984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffaa7a0 0x558dcffb5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a34984740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffb7ae0 0x558dcffa39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a34984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558dcfe96430 con 0x558dcffaa7a0 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a34984740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558dcffba830 con 0x558dcffa9e80 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a34984740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558dcffb9ed0 con 0x558dcffb7ae0 2024-03-23T15:45:02.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffaa7a0 0x558dcffb5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffaa7a0 0x558dcffb5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35732/0 (socket says 172.21.15.76:35732) 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 -- 172.21.15.76:0/1334347882 learned_addr learned my addr 172.21.15.76:0/1334347882 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffc3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 -- 172.21.15.76:0/1334347882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffb7ae0 msgr2=0x558dcffa39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffb7ae0 0x558dcffa39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 -- 172.21.15.76:0/1334347882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 msgr2=0x558dcffc3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffc3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 -- 172.21.15.76:0/1334347882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dcfeabb20 con 0x558dcffaa7a0 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffc3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:45:02.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffaa7a0 0x558dcffb5380 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009db0 tx=0x7f9a14005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7f36e0b5dc1375 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.466 INFO:tasks.workunit.client.0.smithi076.stdout:1/979: link d0/d3c/d6e/cf0 d0/d35/d40/d47/d8c/dd6/c140 0 2024-03-23T15:45:02.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.461+0000 7f9a217fa640 1 -- 172.21.15.76:0/1334347882 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14020070 con 0x558dcffaa7a0 2024-03-23T15:45:02.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.462+0000 7f9a217fa640 1 -- 172.21.15.76:0/1334347882 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a14018040 con 0x558dcffaa7a0 2024-03-23T15:45:02.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.462+0000 7f9a217fa640 1 -- 172.21.15.76:0/1334347882 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14017050 con 0x558dcffaa7a0 2024-03-23T15:45:02.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.462+0000 7f9a34984740 1 -- 172.21.15.76:0/1334347882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffaa7a0 msgr2=0x558dcffb5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.462+0000 7f9a34984740 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffaa7a0 0x558dcffb5380 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009db0 tx=0x7f9a14005f90 comp rx=0 tx=0).stop 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- 172.21.15.76:0/1334347882 shutdown_connections 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffb7ae0 0x558dcffa39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffaa7a0 0x558dcffb5380 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 --2- 172.21.15.76:0/1334347882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffa9e80 0x558dcffc3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- 172.21.15.76:0/1334347882 >> 172.21.15.76:0/1334347882 conn(0x558dcffa4900 msgr2=0x558dcffa32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- 172.21.15.76:0/1334347882 shutdown_connections 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- 172.21.15.76:0/1334347882 wait complete. 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- start start 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 Processor -- start 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffa9e80 0x558dcffbfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffbafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffb7ae0 0x558dcffbb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558dcffbbbf0 con 0x558dcffa9e80 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558dcffbbd60 con 0x558dcffb7ae0 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558dd002d350 con 0x558dcffaa7a0 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.463+0000 7f9a34984740 10 client.0 _ll_register_callbacks cb 0x7f9a14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffa9e80 0x558dcffbfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffa9e80 0x558dcffbfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35738/0 (socket says 172.21.15.76:35738) 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/1691485982 learned_addr learned my addr 172.21.15.76:0/1691485982 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2bfff640 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffb7ae0 0x558dcffbb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffbafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/1691485982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 msgr2=0x558dcffbafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffbafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffb7ae0 msgr2=0x558dcffbb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffb7ae0 0x558dcffbb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 -- 172.21.15.76:0/1691485982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a14009a90 con 0x558dcffa9e80 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a2b7fe640 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffa9e80 0x558dcffbfc70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c8f0 tx=0x7f9a1c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cc8c84f70a6130 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a0affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a0affd640 1 -- 172.21.15.76:0/1691485982 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0d17c0 con 0x558dcffa9e80 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a0affd640 1 -- 172.21.15.76:0/1691485982 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9a1c09b280 con 0x558dcffa9e80 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c09a950 con 0x558dcffa9e80 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.464+0000 7f9a0affd640 1 -- 172.21.15.76:0/1691485982 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9a1c0b5080 con 0x558dcffa9e80 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 10 client.15912 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558dcfed1700 con 0x558dcffa9e80 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a0affd640 1 -- 172.21.15.76:0/1691485982 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c09a950 con 0x558dcffa9e80 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a0affd640 1 -- 172.21.15.76:0/1691485982 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9a1c016020 con 0x558dcffa9e80 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 20 client.15912 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dd002e650 0x558dd0030a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 4 client.15912 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dcfc9c890 con 0x558dd002e650 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.465+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dd002e650 0x558dd0030a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a23fff640 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dd002e650 0x558dd0030a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82eaea6a0e0bb64 server_cookie=9e57235edc4c446c in_seq=0 out_seq=0 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a0affd640 10 client.15912 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a0affd640 1 -- 172.21.15.76:0/1691485982 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558dcfc9c890 con 0x558dd002e650 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a0affd640 10 client.15912 handle_command_reply: tid=0 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 client.15912 shutdown 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffa9e80 msgr2=0x558dcffbfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffa9e80 0x558dcffbfc70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c8f0 tx=0x7f9a1c00fec0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 shutdown_connections 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x558dd002e650 0x558dd0030a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558dcffb7ae0 0x558dcffbb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dcffaa7a0 0x558dcffbafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 --2- 172.21.15.76:0/1691485982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558dcffa9e80 0x558dcffbfc70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a0affd640 0 client.15912 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.466+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 >> 172.21.15.76:0/1691485982 conn(0x558dcffa4900 msgr2=0x558dcffa3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 shutdown_connections 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a34984740 1 -- 172.21.15.76:0/1691485982 wait complete. 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a34984740 20 client.15912 trim_cache size 0 max 16384 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-03-23T15:45:02.476 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:45:02.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a180a2230 msgr2=0x7f9a180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a180a2230 0x7f9a180a46f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009430 tx=0x7f9a24009800 comp rx=0 tx=0).stop 2024-03-23T15:45:02.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0688f0 msgr2=0x7f9a2c19d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0688f0 0x7f9a2c19d8d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00c920 tx=0x7f9a1c00cde0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 shutdown_connections 2024-03-23T15:45:02.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9a180a2230 0x7f9a180a46f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c107690 0x7f9a2c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a2c0692b0 0x7f9a2c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 --2- 172.21.15.76:0/2567935620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2c0688f0 0x7f9a2c19d8d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.467+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 >> 172.21.15.76:0/2567935620 conn(0x7f9a2c077710 msgr2=0x7f9a2c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.468+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 shutdown_connections 2024-03-23T15:45:02.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.468+0000 7f9a31cc9640 1 -- 172.21.15.76:0/2567935620 wait complete. 2024-03-23T15:45:02.478 INFO:tasks.workunit.client.0.smithi076.stdout:1/980: dread d0/d2/d91/d86/fcc [0,4194304] 0 2024-03-23T15:45:02.479 INFO:tasks.workunit.client.0.smithi076.stdout:1/981: write d0/d35/d40/d47/d8c/d99/f116 [2917693,52294] 0 2024-03-23T15:45:02.479 INFO:tasks.workunit.client.0.smithi076.stdout:3/963: dwrite d5/d17/d1c/d32/d70/d8c/fab [0,4194304] 0 2024-03-23T15:45:02.486 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-23T15:45:02.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:45:02.487 INFO:tasks.workunit.client.0.smithi076.stdout:1/982: dread d0/d2/d70/da1/fec [0,4194304] 0 2024-03-23T15:45:02.491 INFO:tasks.workunit.client.0.smithi076.stdout:3/964: unlink d5/d35/d6d/d7f/c9a 0 2024-03-23T15:45:02.492 INFO:tasks.workunit.client.0.smithi076.stdout:6/867: sync 2024-03-23T15:45:02.493 INFO:tasks.workunit.client.0.smithi076.stdout:6/868: getdents d1/dd/df/d5b/d62/d128 0 2024-03-23T15:45:02.493 INFO:tasks.workunit.client.0.smithi076.stdout:6/869: write d1/dd/df/d5b/dcd/f126 [503941,102903] 0 2024-03-23T15:45:02.503 INFO:tasks.workunit.client.0.smithi076.stdout:1/983: dwrite d0/d35/d40/d47/d17/d20/f46 [0,4194304] 0 2024-03-23T15:45:02.504 INFO:tasks.workunit.client.0.smithi076.stdout:1/984: write d0/d35/d40/d47/d17/f2c [4219809,36440] 0 2024-03-23T15:45:02.505 INFO:tasks.workunit.client.0.smithi076.stdout:1/985: symlink d0/d35/d40/dd1/d10c/l141 0 2024-03-23T15:45:02.513 INFO:tasks.workunit.client.0.smithi076.stdout:1/986: dread d0/d35/d40/d47/d55/fd2 [0,4194304] 0 2024-03-23T15:45:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 -- 172.21.15.76:0/2119707298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 msgr2=0x7f95d8100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 --2- 172.21.15.76:0/2119707298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 0x7f95d8100610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009900 tx=0x7f95c802bcd0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 -- 172.21.15.76:0/2119707298 shutdown_connections 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 --2- 172.21.15.76:0/2119707298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d8100d40 0x7f95d810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 --2- 172.21.15.76:0/2119707298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 0x7f95d8100610 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 --2- 172.21.15.76:0/2119707298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d81066b0 0x7f95d8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 -- 172.21.15.76:0/2119707298 >> 172.21.15.76:0/2119707298 conn(0x7f95d8077710 msgr2=0x7f95d8077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 -- 172.21.15.76:0/2119707298 shutdown_connections 2024-03-23T15:45:02.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.592+0000 7f95e0743640 1 -- 172.21.15.76:0/2119707298 wait complete. 2024-03-23T15:45:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 Processor -- start 2024-03-23T15:45:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 -- start start 2024-03-23T15:45:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d8100d40 0x7f95d81022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d81066b0 0x7f95d81027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 0x7f95d8102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81a6100 con 0x7f95d8106fd0 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81a6270 con 0x7f95d81066b0 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95e0743640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81a6430 con 0x7f95d8100d40 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95de4df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d8100d40 0x7f95d81022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95dece0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 0x7f95d8102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95de4df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d8100d40 0x7f95d81022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:55288/0 (socket says 172.21.15.76:55288) 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95de4df640 1 -- 172.21.15.76:0/1759073455 learned_addr learned my addr 172.21.15.76:0/1759073455 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.593+0000 7f95ddcde640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d81066b0 0x7f95d81027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95dece0640 1 -- 172.21.15.76:0/1759073455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d8100d40 msgr2=0x7f95d81022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95dece0640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d8100d40 0x7f95d81022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95dece0640 1 -- 172.21.15.76:0/1759073455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d81066b0 msgr2=0x7f95d81027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95dece0640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d81066b0 0x7f95d81027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95dece0640 1 -- 172.21.15.76:0/1759073455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c8009560 con 0x7f95d8106fd0 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95dece0640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 0x7f95d8102d30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009fd0 tx=0x7f95d000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95cf7fe640 1 -- 172.21.15.76:0/1759073455 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d000ce20 con 0x7f95d8106fd0 2024-03-23T15:45:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95cf7fe640 1 -- 172.21.15.76:0/1759073455 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d0005020 con 0x7f95d8106fd0 2024-03-23T15:45:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95cf7fe640 1 -- 172.21.15.76:0/1759073455 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0010690 con 0x7f95d8106fd0 2024-03-23T15:45:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d81a6710 con 0x7f95d8106fd0 2024-03-23T15:45:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.594+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a6ba0 con 0x7f95d8106fd0 2024-03-23T15:45:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.595+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d8068500 con 0x7f95d8106fd0 2024-03-23T15:45:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.596+0000 7f95cf7fe640 1 -- 172.21.15.76:0/1759073455 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f95d0005500 con 0x7f95d8106fd0 2024-03-23T15:45:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.597+0000 7f95cf7fe640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f95c40a2260 0x7f95c40a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.597+0000 7f95cf7fe640 1 -- 172.21.15.76:0/1759073455 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f95d0014070 con 0x7f95d8106fd0 2024-03-23T15:45:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.597+0000 7f95de4df640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f95c40a2260 0x7f95c40a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.597+0000 7f95de4df640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f95c40a2260 0x7f95c40a4720 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f95d40058c0 tx=0x7f95d4005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.602+0000 7f95cf7fe640 1 -- 172.21.15.76:0/1759073455 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f95d0097430 con 0x7f95d8106fd0 2024-03-23T15:45:02.636 INFO:tasks.workunit.client.0.smithi076.stdout:3/965: sync 2024-03-23T15:45:02.637 INFO:tasks.workunit.client.0.smithi076.stdout:3/966: chown d5/d17/d1c/d31/d61/f147 237 1 2024-03-23T15:45:02.638 INFO:tasks.workunit.client.0.smithi076.stdout:3/967: getdents d5/d17/d1c/d32/d8e 0 2024-03-23T15:45:02.641 INFO:tasks.workunit.client.0.smithi076.stdout:3/968: mknod d5/d35/d10e/c14b 0 2024-03-23T15:45:02.641 INFO:tasks.workunit.client.0.smithi076.stdout:3/969: readlink d5/d17/d1c/d32/d8e/l76 0 2024-03-23T15:45:02.643 INFO:tasks.workunit.client.0.smithi076.stdout:3/970: write d5/f28 [306177,127082] 0 2024-03-23T15:45:02.643 INFO:tasks.workunit.client.0.smithi076.stdout:3/971: chown d5/d35/d6d/fa7 1 1 2024-03-23T15:45:02.646 INFO:tasks.workunit.client.0.smithi076.stdout:3/972: creat d5/d17/d1c/f14c x:0 0 0 2024-03-23T15:45:02.650 INFO:tasks.workunit.client.0.smithi076.stdout:3/973: rename d5/d17/d1c/d32/d5d/f127 to d5/d123/f14d 0 2024-03-23T15:45:02.650 INFO:tasks.workunit.client.0.smithi076.stdout:3/974: chown d5/d17/dcf/f122 121111904 1 2024-03-23T15:45:02.652 INFO:tasks.workunit.client.0.smithi076.stdout:3/975: mknod d5/d35/c14e 0 2024-03-23T15:45:02.662 INFO:tasks.workunit.client.0.smithi076.stdout:3/976: read d5/d17/d1c/f52 [2927798,49021] 0 2024-03-23T15:45:02.681 INFO:tasks.workunit.client.0.smithi076.stdout:3/977: dwrite d5/d35/d6d/d7f/fae [0,4194304] 0 2024-03-23T15:45:02.682 INFO:tasks.workunit.client.0.smithi076.stdout:3/978: fsync d5/d35/d6d/fd3 0 2024-03-23T15:45:02.682 INFO:tasks.workunit.client.0.smithi076.stdout:3/979: chown d5/d17/d1c/d32/d39/d59/d79/ca4 970 1 2024-03-23T15:45:02.682 INFO:tasks.workunit.client.0.smithi076.stdout:3/980: readlink d5/d17/d1c/d32/d70/d8c/l125 0 2024-03-23T15:45:02.688 INFO:tasks.workunit.client.0.smithi076.stdout:3/981: write d5/d35/d6d/fbe [3258960,93821] 0 2024-03-23T15:45:02.693 INFO:tasks.workunit.client.0.smithi076.stdout:1/987: sync 2024-03-23T15:45:02.693 INFO:tasks.workunit.client.0.smithi076.stdout:6/870: sync 2024-03-23T15:45:02.694 INFO:tasks.workunit.client.0.smithi076.stdout:1/988: creat d0/d35/d40/f142 x:0 0 0 2024-03-23T15:45:02.696 INFO:tasks.workunit.client.0.smithi076.stdout:6/871: write d1/dd/d2a/d23/faa [3972494,79299] 0 2024-03-23T15:45:02.696 INFO:tasks.workunit.client.0.smithi076.stdout:6/872: dread - d1/d4/f131 zero size 2024-03-23T15:45:02.699 INFO:tasks.workunit.client.0.smithi076.stdout:3/982: rmdir d5/d17/d1c/d31 39 2024-03-23T15:45:02.700 INFO:tasks.workunit.client.0.smithi076.stdout:3/983: symlink d5/d17/d1c/d32/d39/d59/d79/d114/d116/l14f 0 2024-03-23T15:45:02.706 INFO:tasks.workunit.client.0.smithi076.stdout:3/984: creat d5/d35/d3c/d67/df6/f150 x:0 0 0 2024-03-23T15:45:02.706 INFO:tasks.workunit.client.0.smithi076.stdout:6/873: write d1/dd/d2a/dfd/d113/db3/d8d/d60/d7c/f83 [1958233,116491] 0 2024-03-23T15:45:02.706 INFO:tasks.workunit.client.0.smithi076.stdout:6/874: write d1/dd/d2a/dfd/d113/db3/d8d/d60/fae [783156,39175] 0 2024-03-23T15:45:02.707 INFO:tasks.workunit.client.0.smithi076.stdout:6/875: chown d1/dd/d2a/dfd/d113/db3/d8d/d60/f107 6 1 2024-03-23T15:45:02.707 INFO:tasks.workunit.client.0.smithi076.stdout:6/876: fdatasync d1/d4/d10/fe9 0 2024-03-23T15:45:02.707 INFO:tasks.workunit.client.0.smithi076.stdout:3/985: write d5/d35/d3c/d67/fce [275215,61114] 0 2024-03-23T15:45:02.707 INFO:tasks.workunit.client.0.smithi076.stdout:3/986: mkdir d5/d17/d1c/d32/d5d/d151 0 2024-03-23T15:45:02.708 INFO:tasks.workunit.client.0.smithi076.stdout:3/987: mknod d5/d17/d1c/d32/d70/d8c/c152 0 2024-03-23T15:45:02.708 INFO:tasks.workunit.client.0.smithi076.stdout:6/877: dread d1/dd/df/f66 [0,4194304] 0 2024-03-23T15:45:02.708 INFO:tasks.workunit.client.0.smithi076.stdout:6/878: fdatasync d1/d81/fcb 0 2024-03-23T15:45:02.709 INFO:tasks.workunit.client.0.smithi076.stdout:3/988: creat d5/d17/d1c/d31/d61/f153 x:0 0 0 2024-03-23T15:45:02.711 INFO:tasks.workunit.client.0.smithi076.stdout:3/989: mknod d5/d17/d1c/d32/d70/d10d/c154 0 2024-03-23T15:45:02.712 INFO:tasks.workunit.client.0.smithi076.stdout:3/990: creat d5/d17/d1c/d31/d61/d137/f155 x:0 0 0 2024-03-23T15:45:02.717 INFO:tasks.workunit.client.0.smithi076.stdout:6/879: dread d1/d81/f9a [0,4194304] 0 2024-03-23T15:45:02.727 INFO:tasks.workunit.client.0.smithi076.stdout:3/991: dwrite d5/d35/d6d/fbe [0,4194304] 0 2024-03-23T15:45:02.730 INFO:tasks.workunit.client.0.smithi076.stdout:1/989: mkdir d0/d2/dfc/d110/d143 0 2024-03-23T15:45:02.731 INFO:tasks.workunit.client.0.smithi076.stdout:1/990: mknod d0/d35/d40/d47/d8c/d99/c144 0 2024-03-23T15:45:02.731 INFO:tasks.workunit.client.0.smithi076.stdout:1/991: getdents d0/d35/d40/d47/d55 0 2024-03-23T15:45:02.734 INFO:tasks.workunit.client.0.smithi076.stdout:1/992: symlink d0/d35/d40/d47/d55/l145 0 2024-03-23T15:45:02.734 INFO:tasks.workunit.client.0.smithi076.stdout:1/993: truncate d0/d35/d40/d47/d17/d20/f58 1633194 0 2024-03-23T15:45:02.734 INFO:tasks.workunit.client.0.smithi076.stdout:1/994: chown d0/d35/d40/d47/d17/d20/l96 27657840 1 2024-03-23T15:45:02.734 INFO:tasks.workunit.client.0.smithi076.stdout:1/995: truncate d0/d35/d40/d47/d55/f56 991607 0 2024-03-23T15:45:02.737 INFO:tasks.workunit.client.0.smithi076.stdout:1/996: write d0/d35/f3a [1888179,27929] 0 2024-03-23T15:45:02.738 INFO:tasks.workunit.client.0.smithi076.stdout:1/997: truncate d0/d35/d40/d47/d55/fd2 5066975 0 2024-03-23T15:45:02.739 INFO:tasks.workunit.client.0.smithi076.stdout:1/998: mknod d0/d3c/d6e/dc4/c146 0 2024-03-23T15:45:02.740 INFO:tasks.workunit.client.0.smithi076.stdout:6/880: rename d1/dd/d2a/dfd/d113/l11c to d1/dd/df/d5b/dcd/l138 0 2024-03-23T15:45:02.742 INFO:tasks.workunit.client.0.smithi076.stdout:6/881: rename d1/dd/d2a/dfd/f11d to d1/dd/df/db5/f139 0 2024-03-23T15:45:02.743 INFO:tasks.workunit.client.0.smithi076.stdout:3/992: dwrite d5/fd [0,4194304] 0 2024-03-23T15:45:02.744 INFO:tasks.workunit.client.0.smithi076.stdout:3/993: creat d5/d35/d4c/de1/f156 x:0 0 0 2024-03-23T15:45:02.745 INFO:tasks.workunit.client.0.smithi076.stdout:3/994: getdents d5/d123 0 2024-03-23T15:45:02.745 INFO:tasks.workunit.client.0.smithi076.stdout:3/995: readlink d5/d35/la3 0 2024-03-23T15:45:02.746 INFO:tasks.workunit.client.0.smithi076.stdout:6/882: stat d1/dd/df/dac/le7 0 2024-03-23T15:45:02.748 INFO:tasks.workunit.client.0.smithi076.stdout:1/999: write d0/d2/f48 [6222156,67715] 0 2024-03-23T15:45:02.748 INFO:tasks.workunit.client.0.smithi076.stdout:6/883: getdents d1/dd/df/dac/dec 0 2024-03-23T15:45:02.749 INFO:tasks.workunit.client.0.smithi076.stdout:6/884: write d1/d4/f109 [539451,71379] 0 2024-03-23T15:45:02.750 INFO:tasks.workunit.client.0.smithi076.stdout:3/996: mkdir d5/d17/d1c/d32/d5d/d157 0 2024-03-23T15:45:02.751 INFO:tasks.workunit.client.0.smithi076.stdout:3/997: mkdir d5/d17/d1c/d32/d39/d59/d79/d114/d116/d158 0 2024-03-23T15:45:02.751 INFO:tasks.workunit.client.0.smithi076.stdout:3/998: fsync d5/d35/d4c/f11b 0 2024-03-23T15:45:02.752 INFO:tasks.workunit.client.0.smithi076.stdout:6/885: rmdir d1/dd/df/d5b/d62 39 2024-03-23T15:45:02.752 INFO:tasks.workunit.client.0.smithi076.stdout:6/886: write d1/dd/d2a/f91 [383571,130119] 0 2024-03-23T15:45:02.760 INFO:tasks.workunit.client.0.smithi076.stdout:6/887: write d1/dd/d2a/dfd/d113/f104 [1782519,128333] 0 2024-03-23T15:45:02.761 INFO:tasks.workunit.client.0.smithi076.stdout:6/888: fsync d1/d2/df2/f119 0 2024-03-23T15:45:02.761 INFO:tasks.workunit.client.0.smithi076.stdout:6/889: dread - d1/d4/d10/d39/f124 zero size 2024-03-23T15:45:02.762 INFO:tasks.workunit.client.0.smithi076.stdout:6/890: rmdir d1/d81 39 2024-03-23T15:45:02.766 INFO:tasks.workunit.client.0.smithi076.stdout:6/891: symlink d1/dd/l13a 0 2024-03-23T15:45:02.766 INFO:tasks.workunit.client.0.smithi076.stdout:6/892: write d1/dd/df/d5b/dcd/f126 [534307,97258] 0 2024-03-23T15:45:02.767 INFO:tasks.workunit.client.0.smithi076.stdout:3/999: dwrite d5/d35/d4c/de1/f13b [0,4194304] 0 2024-03-23T15:45:02.771 INFO:tasks.workunit.client.0.smithi076.stdout:6/893: dread d1/d2/f38 [0,4194304] 0 2024-03-23T15:45:02.771 INFO:tasks.workunit.client.0.smithi076.stdout:6/894: dread - d1/dd/d2a/dfd/d113/db3/d8d/f129 zero size 2024-03-23T15:45:02.776 INFO:tasks.workunit.client.0.smithi076.stdout:6/895: dread d1/d4/d10/f90 [4194304,4194304] 0 2024-03-23T15:45:02.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:02 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:45:02.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:02 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:45:02.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.855+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95d81107a0 con 0x7f95d8106fd0 2024-03-23T15:45:02.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.856+0000 7f95cf7fe640 1 -- 172.21.15.76:0/1759073455 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11883 (secure 0 0 0) 0x7f95d0005190 con 0x7f95d8106fd0 2024-03-23T15:45:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:45:02.859 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:45:00.940962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":60,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":57,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:45:02.860 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 27 2024-03-23T15:45:02.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f95c40a2260 msgr2=0x7f95c40a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f95c40a2260 0x7f95c40a4720 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f95d40058c0 tx=0x7f95d4005f90 comp rx=0 tx=0).stop 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 msgr2=0x7f95d8102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 0x7f95d8102d30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009fd0 tx=0x7f95d000ef10 comp rx=0 tx=0).stop 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 shutdown_connections 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f95c40a2260 0x7f95c40a4720 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d8106fd0 0x7f95d8102d30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d81066b0 0x7f95d81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 --2- 172.21.15.76:0/1759073455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95d8100d40 0x7f95d81022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 >> 172.21.15.76:0/1759073455 conn(0x7f95d8077710 msgr2=0x7f95d80ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 shutdown_connections 2024-03-23T15:45:02.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:02.858+0000 7f95e0743640 1 -- 172.21.15.76:0/1759073455 wait complete. 2024-03-23T15:45:02.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-23T15:45:02.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:02 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:45:02.901 INFO:tasks.workunit.client.0.smithi076.stdout:6/896: sync 2024-03-23T15:45:02.902 INFO:tasks.workunit.client.0.smithi076.stdout:6/897: chown d1/d4/d103 797 1 2024-03-23T15:45:02.902 INFO:tasks.workunit.client.0.smithi076.stdout:6/898: fsync d1/dd/d2a/dfd/d113/db3/d8d/d60/d7c/f11b 0 2024-03-23T15:45:02.914 INFO:tasks.workunit.client.0.smithi076.stdout:6/899: dwrite d1/d2/dc8/fff [0,4194304] 0 2024-03-23T15:45:02.914 INFO:tasks.workunit.client.0.smithi076.stdout:6/900: creat d1/dd/d2a/dfd/d113/db3/d8d/db8/de0/f13b x:0 0 0 2024-03-23T15:45:02.915 INFO:tasks.workunit.client.0.smithi076.stdout:6/901: creat d1/d81/df6/f13c x:0 0 0 2024-03-23T15:45:02.915 INFO:tasks.workunit.client.0.smithi076.stdout:6/902: read - d1/d81/df6/f13c zero size 2024-03-23T15:45:02.917 INFO:tasks.workunit.client.0.smithi076.stdout:6/903: mknod d1/dd/df/d5b/dcd/c13d 0 2024-03-23T15:45:02.918 INFO:tasks.workunit.client.0.smithi076.stdout:6/904: symlink d1/dd/d2a/dfd/d113/db3/l13e 0 2024-03-23T15:45:02.918 INFO:tasks.workunit.client.0.smithi076.stdout:6/905: stat d1/dd/d2a/dfd/d113/db3/d8d/db8/fc0 0 2024-03-23T15:45:02.927 INFO:tasks.workunit.client.0.smithi076.stdout:6/906: unlink d1/d4/de2/c10e 0 2024-03-23T15:45:03.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.004+0000 7f4c25bd0640 1 -- 172.21.15.76:0/932304994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201076b0 msgr2=0x7f4c20107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.004+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/932304994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201076b0 0x7f4c20107a90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009930 tx=0x7f4c1002bc30 comp rx=0 tx=0).stop 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.004+0000 7f4c25bd0640 1 -- 172.21.15.76:0/932304994 shutdown_connections 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.004+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/932304994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c20102060 0x7f4c2010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.004+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/932304994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c201016a0 0x7f4c20101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.004+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/932304994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201076b0 0x7f4c20107a90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.004+0000 7f4c25bd0640 1 -- 172.21.15.76:0/932304994 >> 172.21.15.76:0/932304994 conn(0x7f4c20078b10 msgr2=0x7f4c20078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 -- 172.21.15.76:0/932304994 shutdown_connections 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 -- 172.21.15.76:0/932304994 wait complete. 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 Processor -- start 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 -- start start 2024-03-23T15:45:03.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201016a0 0x7f4c2019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c20102060 0x7f4c2019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c201076b0 0x7f4c20197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4c201982b0 con 0x7f4c201016a0 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20198420 con 0x7f4c20102060 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c25bd0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c201985e0 con 0x7f4c201076b0 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.005+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201016a0 0x7f4c2019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201016a0 0x7f4c2019da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35774/0 (socket says 172.21.15.76:35774) 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/4003334456 learned_addr learned my addr 172.21.15.76:0/4003334456 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c201076b0 0x7f4c20197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1effd640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c20102060 0x7f4c2019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/4003334456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c201076b0 msgr2=0x7f4c20197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c201076b0 0x7f4c20197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/4003334456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c20102060 msgr2=0x7f4c2019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c20102060 0x7f4c2019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/4003334456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c10009560 con 0x7f4c201016a0 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201016a0 0x7f4c2019da60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009230 tx=0x7f4c1002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1cff9640 1 -- 172.21.15.76:0/4003334456 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10004350 con 0x7f4c201016a0 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1cff9640 1 -- 172.21.15.76:0/4003334456 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c100044b0 con 0x7f4c201016a0 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c1cff9640 1 -- 172.21.15.76:0/4003334456 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10003d00 con 0x7f4c201016a0 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c20198860 con 0x7f4c201016a0 2024-03-23T15:45:03.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.006+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c201a49e0 con 0x7f4c201016a0 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.008+0000 7f4c1cff9640 1 -- 172.21.15.76:0/4003334456 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f4c10004df0 con 0x7f4c201016a0 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.008+0000 7f4c287b3740 1 Processor -- start 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c287b3740 1 -- start start 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1cff9640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4c080a2230 0x7f4c080a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135e3f0 0x561c0135e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135ed10 0x561c013673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01369ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c287b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561c012cc840 con 0x561c013678f0 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c287b3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c012c9d70 con 0x561c0135e3f0 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c287b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561c012c5550 con 0x561c0135ed10 2024-03-23T15:45:03.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1cff9640 1 -- 172.21.15.76:0/4003334456 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c100f00a0 con 0x7f4c201016a0 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4c080a2230 0x7f4c080a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135ed10 0x561c013673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135e3f0 0x561c0135e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01369ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135ed10 0x561c013673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:55318/0 (socket says 172.21.15.76:55318) 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1effd640 1 -- 172.21.15.76:0/3879198308 learned_addr learned my addr 172.21.15.76:0/3879198308 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/3879198308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135ed10 msgr2=0x561c013673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135ed10 0x561c013673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/3879198308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 msgr2=0x561c01369ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01369ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/3879198308 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0125ade0 con 0x561c0135e3f0 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1effd640 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135ed10 0x561c013673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4c080a2230 0x7f4c080a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005fd0 tx=0x7f4c18012a70 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135e3f0 0x561c0135e7d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c100ed900 tx=0x7f4c10009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb25b1b942dacdd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.009+0000 7f4c05ffb640 1 -- 172.21.15.76:0/3879198308 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1003d3b0 con 0x561c0135e3f0 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c05ffb640 1 -- 172.21.15.76:0/3879198308 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c100bd2a0 con 0x561c0135e3f0 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c05ffb640 1 -- 172.21.15.76:0/3879198308 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100b9210 con 0x561c0135e3f0 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 -- 172.21.15.76:0/3879198308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135e3f0 msgr2=0x561c0135e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135e3f0 0x561c0135e7d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c100ed900 tx=0x7f4c10009260 comp rx=0 tx=0).stop 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 -- 172.21.15.76:0/3879198308 shutdown_connections 2024-03-23T15:45:03.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01369ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135ed10 0x561c013673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3879198308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135e3f0 0x561c0135e7d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 -- 172.21.15.76:0/3879198308 >> 172.21.15.76:0/3879198308 conn(0x561c01358b20 msgr2=0x561c01358f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 -- 172.21.15.76:0/3879198308 shutdown_connections 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.010+0000 7f4c287b3740 1 -- 172.21.15.76:0/3879198308 wait complete. 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 -- start start 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 Processor -- start 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135e3f0 0x561c013648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01364de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01365320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561c00f98430 con 0x561c013678f0 2024-03-23T15:45:03.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c00f54e00 con 0x561c0135ed10 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561c0124a590 con 0x561c0135e3f0 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c287b3740 10 client.0 _ll_register_callbacks cb 0x561c012b9fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01365320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135e3f0 0x561c013648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01365320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35788/0 (socket says 172.21.15.76:35788) 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 -- 172.21.15.76:0/3074263466 learned_addr learned my addr 172.21.15.76:0/3074263466 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1effd640 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01364de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 -- 172.21.15.76:0/3074263466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135e3f0 msgr2=0x561c013648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135e3f0 0x561c013648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 -- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 msgr2=0x561c01364de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01364de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.011+0000 7f4c1ffff640 1 -- 172.21.15.76:0/3074263466 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c201a49e0 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01365320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800e980 tx=0x7f4c18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e154b821819b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4bfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4bfeffd640 1 -- 172.21.15.76:0/3074263466 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18017040 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4bfeffd640 1 -- 172.21.15.76:0/3074263466 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c1800ee40 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c00fc47e0 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4bfeffd640 1 -- 172.21.15.76:0/3074263466 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c1801f430 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 10 client.15930 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c012c3bd0 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4bfeffd640 1 -- 172.21.15.76:0/3074263466 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c18033020 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4bfeffd640 1 -- 172.21.15.76:0/3074263466 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4c18026600 con 0x561c013678f0 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:03.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 10 client.15930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 20 client.15930 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c01377770 0x561c01379b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 4 client.15930 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.012+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c01001720 con 0x561c01377770 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.013+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c01377770 0x561c01379b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.013+0000 7f4bfeffd640 10 client.15930 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.013+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c01377770 0x561c01379b90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0a4bb175b916a server_cookie=285a91fd82e39878 in_seq=0 out_seq=0 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.013+0000 7f4bfeffd640 1 -- 172.21.15.76:0/3074263466 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c01001720 con 0x561c01377770 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.013+0000 7f4bfeffd640 10 client.15930 handle_command_reply: tid=0 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 client.15930 shutdown 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 msgr2=0x561c01365320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01365320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800e980 tx=0x7f4c18004ab0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 shutdown_connections 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c01377770 0x561c01379b90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c013678f0 0x561c01365320 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01364de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 --2- 172.21.15.76:0/3074263466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c0135e3f0 0x561c013648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4bfeffd640 0 client.15930 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 >> 172.21.15.76:0/3074263466 conn(0x561c01358b20 msgr2=0x561c0135a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 shutdown_connections 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 1 -- 172.21.15.76:0/3074263466 wait complete. 2024-03-23T15:45:03.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.014+0000 7f4c287b3740 20 client.15930 trim_cache size 0 max 16384 2024-03-23T15:45:03.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 Processor -- start 2024-03-23T15:45:03.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 -- start start 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01366fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01365140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c0137f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561c01375e70 con 0x561c0135e3f0 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c0135ae50 con 0x561c0135ed10 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c287b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561c013604c0 con 0x561c013678f0 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01366fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01366fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35798/0 (socket says 172.21.15.76:35798) 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/1491442430 learned_addr learned my addr 172.21.15.76:0/1491442430 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1effd640 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01365140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c0137f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/1491442430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 msgr2=0x561c0137f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c0137f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/1491442430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 msgr2=0x561c01365140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01365140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.027+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/1491442430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0125fc80 con 0x561c0135e3f0 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01366fd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c100bae50 tx=0x7f4c10000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3f7443f6da3e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c057fa640 1 -- 172.21.15.76:0/1491442430 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100bcc20 con 0x561c0135e3f0 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c057fa640 1 -- 172.21.15.76:0/1491442430 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c100c0cf0 con 0x561c0135e3f0 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c057fa640 1 -- 172.21.15.76:0/1491442430 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c100c4eb0 con 0x561c0135e3f0 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 -- 172.21.15.76:0/1491442430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 msgr2=0x561c01366fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01366fd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c100bae50 tx=0x7f4c10000bc0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 -- 172.21.15.76:0/1491442430 shutdown_connections 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c0137f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01365140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 --2- 172.21.15.76:0/1491442430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01366fd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 -- 172.21.15.76:0/1491442430 >> 172.21.15.76:0/1491442430 conn(0x561c01358b20 msgr2=0x561c01357430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 -- 172.21.15.76:0/1491442430 shutdown_connections 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.028+0000 7f4c287b3740 1 -- 172.21.15.76:0/1491442430 wait complete. 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 -- start start 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 Processor -- start 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01361650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01361b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c013620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561c013dcb60 con 0x561c0135e3f0 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c013dccd0 con 0x561c0135ed10 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561c013dce90 con 0x561c013678f0 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c287b3740 10 client.0 _ll_register_callbacks cb 0x7f4c18003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01361650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01361650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35814/0 (socket says 172.21.15.76:35814) 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/691192015 learned_addr learned my addr 172.21.15.76:0/691192015 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1effd640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01361b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c013620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/691192015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 msgr2=0x561c013620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c013620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 msgr2=0x561c01361b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01361b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1f7fe640 1 -- 172.21.15.76:0/691192015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0125ade0 con 0x561c0135e3f0 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.029+0000 7f4c1ffff640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c013620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c1f7fe640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01361650 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4c10006010 tx=0x7f4c100037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fdef65ece90fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4bfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4bfeffd640 1 -- 172.21.15.76:0/691192015 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10088070 con 0x561c0135e3f0 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4bfeffd640 1 -- 172.21.15.76:0/691192015 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c100c5340 con 0x561c0135e3f0 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c100bd2a0 con 0x561c0135e3f0 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4bfeffd640 1 -- 172.21.15.76:0/691192015 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c100bcc20 con 0x561c0135e3f0 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 10 client.15936 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:03.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c01285920 con 0x561c0135e3f0 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4bfeffd640 1 -- 172.21.15.76:0/691192015 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c100bd2a0 con 0x561c0135e3f0 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4bfeffd640 1 -- 172.21.15.76:0/691192015 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4c1003c070 con 0x561c0135e3f0 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 10 client.15936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 20 client.15936 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c013de200 0x561c013e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 4 client.15936 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 --> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c01050890 con 0x561c013de200 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.030+0000 7f4c1effd640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c013de200 0x561c013e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4bfeffd640 10 client.15936 ms_handle_connect on v2:172.21.15.183:6836/3732793280 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c1effd640 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c013de200 0x561c013e05f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d2fa7ecb2b6ec3a server_cookie=5d915acfed7e16b3 in_seq=0 out_seq=0 2024-03-23T15:45:03.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4bfeffd640 1 -- 172.21.15.76:0/691192015 <== mds.2 v2:172.21.15.183:6836/3732793280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1153 (crc 0 0 0) 0x561c01050890 con 0x561c013de200 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4bfeffd640 10 client.15936 handle_command_reply: tid=0 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 client.15936 shutdown 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 msgr2=0x561c01361650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01361650 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4c10006010 tx=0x7f4c100037d0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 shutdown_connections 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561c013678f0 0x561c013620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c0135ed10 0x561c01361b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.183:6836/3732793280,v1:172.21.15.183:6837/3732793280] conn(0x561c013de200 0x561c013e05f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 --2- 172.21.15.76:0/691192015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c0135e3f0 0x561c01361650 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4bfeffd640 0 client.15936 ms_handle_reset on v2:172.21.15.183:6836/3732793280 2024-03-23T15:45:03.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 >> 172.21.15.76:0/691192015 conn(0x561c01358b20 msgr2=0x561c01357cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 shutdown_connections 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 1 -- 172.21.15.76:0/691192015 wait complete. 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.031+0000 7f4c287b3740 20 client.15936 trim_cache size 0 max 16384 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 429803148, 2024-03-23T15:45:03.044 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637463, 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.sh" 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 638993242, 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637466, 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.c" 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3667474593, 2024-03-23T15:45:03.045 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637465, 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_noioctl.c" 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4285727850, 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637464, 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/Makefile" 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:45:03.046 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4c080a2230 msgr2=0x7f4c080a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4c080a2230 0x7f4c080a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005fd0 tx=0x7f4c18012a70 comp rx=0 tx=0).stop 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201016a0 msgr2=0x7f4c2019da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201016a0 0x7f4c2019da60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009230 tx=0x7f4c1002c650 comp rx=0 tx=0).stop 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 shutdown_connections 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4c080a2230 0x7f4c080a46f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c201076b0 0x7f4c20197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c20102060 0x7f4c2019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 --2- 172.21.15.76:0/4003334456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c201016a0 0x7f4c2019da60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 >> 172.21.15.76:0/4003334456 conn(0x7f4c20078b10 msgr2=0x7f4c20104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 shutdown_connections 2024-03-23T15:45:03.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.032+0000 7f4c25bd0640 1 -- 172.21.15.76:0/4003334456 wait complete. 2024-03-23T15:45:03.049 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-23T15:45:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.188+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/3383873343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a181005a0 msgr2=0x7f7a18100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/3383873343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a181005a0 0x7f7a18100a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009890 tx=0x7f7a0c02bb70 comp rx=0 tx=0).stop 2024-03-23T15:45:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/3383873343 shutdown_connections 2024-03-23T15:45:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/3383873343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a18100f60 0x7f7a1810d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/3383873343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a181005a0 0x7f7a18100a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/3383873343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 0x7f7a18106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/3383873343 >> 172.21.15.76:0/3383873343 conn(0x7f7a18077710 msgr2=0x7f7a18077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/3383873343 shutdown_connections 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/3383873343 wait complete. 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 Processor -- start 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 -- start start 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.189+0000 7f7a1e2cd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a181005a0 0x7f7a1819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1e2cd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a18100f60 0x7f7a1819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1e2cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 0x7f7a18196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1e2cd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18197050 con 0x7f7a18106660 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1e2cd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181971c0 con 0x7f7a181005a0 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1e2cd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18197380 con 0x7f7a18100f60 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 0x7f7a18196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a17fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a181005a0 0x7f7a1819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 0x7f7a18196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35830/0 (socket says 172.21.15.76:35830) 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2203800707 learned_addr learned my addr 172.21.15.76:0/2203800707 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a177fe640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a18100f60 0x7f7a1819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2203800707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a18100f60 msgr2=0x7f7a1819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a18100f60 0x7f7a1819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2203800707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a181005a0 msgr2=0x7f7a1819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a181005a0 0x7f7a1819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2203800707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c009490 con 0x7f7a18106660 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a17fff640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a181005a0 0x7f7a1819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 0x7f7a18196a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fd0 tx=0x7f7a1000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a157fa640 1 -- 172.21.15.76:0/2203800707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10009b50 con 0x7f7a18106660 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a18197660 con 0x7f7a18106660 2024-03-23T15:45:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a157fa640 1 -- 172.21.15.76:0/2203800707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a10003970 con 0x7f7a18106660 2024-03-23T15:45:03.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a157fa640 1 -- 172.21.15.76:0/2203800707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10004810 con 0x7f7a18106660 2024-03-23T15:45:03.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.190+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181a38f0 con 0x7f7a18106660 2024-03-23T15:45:03.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.192+0000 7f7a157fa640 1 -- 172.21.15.76:0/2203800707 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f7a1000a040 con 0x7f7a18106660 2024-03-23T15:45:03.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 Processor -- start 2024-03-23T15:45:03.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a157fa640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a000a2230 0x7f7a000a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 -- start start 2024-03-23T15:45:03.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a177fe640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a000a2230 0x7f7a000a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a157fa640 1 -- 172.21.15.76:0/2203800707 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7a100cdff0 con 0x7f7a18106660 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106096ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a105ff7840 con 0x55a106094b00 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a105ff4d70 con 0x55a106089780 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a20f89740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a105ff0550 con 0x55a106088dd0 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106096ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a17fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106096ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35840/0 (socket says 172.21.15.76:35840) 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a1c86a640 1 -- 172.21.15.76:0/759378771 learned_addr learned my addr 172.21.15.76:0/759378771 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a177fe640 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a17fff640 1 -- 172.21.15.76:0/759378771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 msgr2=0x55a1060945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a17fff640 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a17fff640 1 -- 172.21.15.76:0/759378771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 msgr2=0x55a106096ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a17fff640 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106096ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.193+0000 7f7a17fff640 1 -- 172.21.15.76:0/759378771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a105f85de0 con 0x55a106088dd0 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106096ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f7a177fe640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a000a2230 0x7f7a000a46f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c000c00 tx=0x7f7a0c034be0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f7a17fff640 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060891b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009fa0 tx=0x7f7a080118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce302d1e4109593 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f79fe7fc640 1 -- 172.21.15.76:0/759378771 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08004030 con 0x55a106088dd0 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f79fe7fc640 1 -- 172.21.15.76:0/759378771 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a08005500 con 0x55a106088dd0 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f79fe7fc640 1 -- 172.21.15.76:0/759378771 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08004c70 con 0x55a106088dd0 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f7a20f89740 1 -- 172.21.15.76:0/759378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 msgr2=0x55a1060891b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.194+0000 7f7a20f89740 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060891b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009fa0 tx=0x7f7a080118d0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- 172.21.15.76:0/759378771 shutdown_connections 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106096ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 --2- 172.21.15.76:0/759378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060891b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- 172.21.15.76:0/759378771 >> 172.21.15.76:0/759378771 conn(0x55a106083b20 msgr2=0x55a106083f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- 172.21.15.76:0/759378771 shutdown_connections 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- 172.21.15.76:0/759378771 wait complete. 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- start start 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 Processor -- start 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a106091b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106092600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a105cc3430 con 0x55a106094b00 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a105c7fe00 con 0x55a106089780 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a105f75590 con 0x55a106088dd0 2024-03-23T15:45:03.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a20f89740 10 client.0 _ll_register_callbacks cb 0x55a105fe4fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106092600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106092600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35850/0 (socket says 172.21.15.76:35850) 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2997654008 learned_addr learned my addr 172.21.15.76:0/2997654008 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.195+0000 7f7a177fe640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a17fff640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a106091b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 msgr2=0x55a106091b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a106091b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2997654008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 msgr2=0x55a1060920c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060920c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a1c86a640 1 -- 172.21.15.76:0/2997654008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a08009c00 con 0x55a106094b00 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a177fe640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106092600 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fa0 tx=0x7f7a10003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25be4309459574 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f79f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:03.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f79f77fe640 1 -- 172.21.15.76:0/2997654008 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a100a2ef0 con 0x55a106094b00 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f79f77fe640 1 -- 172.21.15.76:0/2997654008 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7a1009eb30 con 0x55a106094b00 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a105cef7e0 con 0x55a106094b00 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f79f77fe640 1 -- 172.21.15.76:0/2997654008 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a100a3180 con 0x55a106094b00 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a20f89740 10 client.15948 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a105feebd0 con 0x55a106094b00 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f79f77fe640 1 -- 172.21.15.76:0/2997654008 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1007d020 con 0x55a106094b00 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.196+0000 7f79f77fe640 1 -- 172.21.15.76:0/2997654008 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a100851d0 con 0x55a106094b00 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a20f89740 10 client.15948 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a20f89740 10 client.15948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a20f89740 20 client.15948 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a20f89740 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a1060a4ca0 0x55a1060a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a20f89740 4 client.15948 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a105d2c720 con 0x55a1060a4ca0 2024-03-23T15:45:03.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a17fff640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a1060a4ca0 0x55a1060a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f7a17fff640 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a1060a4ca0 0x55a1060a70c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b82ceb9239a750f server_cookie=63dbeaeae6c995f5 in_seq=0 out_seq=0 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.197+0000 7f79f77fe640 10 client.15948 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f79f77fe640 1 -- 172.21.15.76:0/2997654008 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a105d2c720 con 0x55a1060a4ca0 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f79f77fe640 10 client.15948 handle_command_reply: tid=0 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 client.15948 shutdown 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 msgr2=0x55a106092600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106092600 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fa0 tx=0x7f7a10003ba0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 shutdown_connections 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a1060a4ca0 0x55a1060a70c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a106092600 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f79f77fe640 0 client.15948 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a1060920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 --2- 172.21.15.76:0/2997654008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a106091b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 >> 172.21.15.76:0/2997654008 conn(0x55a106083b20 msgr2=0x55a106085870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 shutdown_connections 2024-03-23T15:45:03.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 1 -- 172.21.15.76:0/2997654008 wait complete. 2024-03-23T15:45:03.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.198+0000 7f7a20f89740 20 client.15948 trim_cache size 0 max 16384 2024-03-23T15:45:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 Processor -- start 2024-03-23T15:45:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 -- start start 2024-03-23T15:45:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a106092420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a1060ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a1060a49a0 con 0x55a106094b00 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1060a08e0 con 0x55a106089780 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a20f89740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a106088730 con 0x55a106088dd0 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a1060ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a1c86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a1060ac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35852/0 (socket says 172.21.15.76:35852) 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a1c86a640 1 -- 172.21.15.76:0/3219572199 learned_addr learned my addr 172.21.15.76:0/3219572199 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a17fff640 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.211+0000 7f7a177fe640 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a106092420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a1c86a640 1 -- 172.21.15.76:0/3219572199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 msgr2=0x55a1060a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a1c86a640 1 -- 172.21.15.76:0/3219572199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 msgr2=0x55a106092420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a106092420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a1c86a640 1 -- 172.21.15.76:0/3219572199 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a105f8ac80 con 0x55a106094b00 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a1060ac990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000cd00 tx=0x7f7a1009a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3387fa2bfa538ce0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f79fdffb640 1 -- 172.21.15.76:0/3219572199 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000ce30 con 0x55a106094b00 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f79fdffb640 1 -- 172.21.15.76:0/3219572199 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a1007d070 con 0x55a106094b00 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f79fdffb640 1 -- 172.21.15.76:0/3219572199 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10085310 con 0x55a106094b00 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 -- 172.21.15.76:0/3219572199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 msgr2=0x55a1060ac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a1060ac990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000cd00 tx=0x7f7a1009a340 comp rx=0 tx=0).stop 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 -- 172.21.15.76:0/3219572199 shutdown_connections 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106094b00 0x55a1060ac990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106089780 0x55a106092420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 --2- 172.21.15.76:0/3219572199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106088dd0 0x55a1060a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 -- 172.21.15.76:0/3219572199 >> 172.21.15.76:0/3219572199 conn(0x55a106083b20 msgr2=0x55a106082cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 -- 172.21.15.76:0/3219572199 shutdown_connections 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.212+0000 7f7a20f89740 1 -- 172.21.15.76:0/3219572199 wait complete. 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 -- start start 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 Processor -- start 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106088dd0 0x55a10608e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106089780 0x55a10608ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106094b00 0x55a10608f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a10610a040 con 0x55a106088dd0 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a10610a1b0 con 0x55a106094b00 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a10610a370 con 0x55a106089780 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a20f89740 10 client.0 _ll_register_callbacks cb 0x7f7a1009a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106088dd0 0x55a10608e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106088dd0 0x55a10608e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35856/0 (socket says 172.21.15.76:35856) 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 -- 172.21.15.76:0/1574417649 learned_addr learned my addr 172.21.15.76:0/1574417649 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a177fe640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106089780 0x55a10608ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a1c86a640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106094b00 0x55a10608f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 -- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106089780 msgr2=0x55a10608ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106089780 0x55a10608ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 -- 172.21.15.76:0/1574417649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106094b00 msgr2=0x55a10608f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106094b00 0x55a10608f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a17fff640 1 -- 172.21.15.76:0/1574417649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a105feebd0 con 0x55a106088dd0 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.213+0000 7f7a177fe640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106089780 0x55a10608ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a17fff640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106088dd0 0x55a10608e8a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009730 tx=0x7f7a0800f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170068f35bd34a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f79f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f79f77fe640 1 -- 172.21.15.76:0/1574417649 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08029a30 con 0x55a106088dd0 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f79f77fe640 1 -- 172.21.15.76:0/1574417649 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7a08027040 con 0x55a106088dd0 2024-03-23T15:45:03.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a08005500 con 0x55a106088dd0 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f79f77fe640 1 -- 172.21.15.76:0/1574417649 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a08029b90 con 0x55a106088dd0 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 10 client.15954 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a105fb0920 con 0x55a106088dd0 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f79f77fe640 1 -- 172.21.15.76:0/1574417649 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a08005500 con 0x55a106088dd0 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f79f77fe640 1 -- 172.21.15.76:0/1574417649 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a0800fb10 con 0x55a106088dd0 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 10 client.15954 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 10 client.15954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 20 client.15954 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a10610bbc0 0x55a10610dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 4 client.15954 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.214+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 --> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a105d7b890 con 0x55a10610bbc0 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a177fe640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a10610bbc0 0x55a10610dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f79f77fe640 10 client.15954 ms_handle_connect on v2:172.21.15.192:6838/2120704815 2024-03-23T15:45:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a177fe640 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a10610bbc0 0x55a10610dfe0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79b05a05eb0cdccd server_cookie=42209a57cc68ae38 in_seq=0 out_seq=0 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f79f77fe640 1 -- 172.21.15.76:0/1574417649 <== mds.1 v2:172.21.15.192:6838/2120704815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4563 (crc 0 0 0) 0x55a105d7b890 con 0x55a10610bbc0 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f79f77fe640 10 client.15954 handle_command_reply: tid=0 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 client.15954 shutdown 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106088dd0 msgr2=0x55a10608e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106088dd0 0x55a10608e8a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009730 tx=0x7f7a0800f4e0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 shutdown_connections 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:6838/2120704815,v1:172.21.15.192:6839/2120704815] conn(0x55a10610bbc0 0x55a10610dfe0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a106094b00 0x55a10608f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f79f77fe640 0 client.15954 ms_handle_reset on v2:172.21.15.192:6838/2120704815 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a106089780 0x55a10608ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 --2- 172.21.15.76:0/1574417649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a106088dd0 0x55a10608e8a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 >> 172.21.15.76:0/1574417649 conn(0x55a106083b20 msgr2=0x55a106084b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 shutdown_connections 2024-03-23T15:45:03.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.215+0000 7f7a20f89740 1 -- 172.21.15.76:0/1574417649 wait complete. 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a20f89740 20 client.15954 trim_cache size 0 max 16384 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 911299051, 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637494, 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/selinux_rxdir.sh" 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1154926179, 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636440, 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile.inc" 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1263995757, 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636361, 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/TEST_PLAN.txt" 2024-03-23T15:45:03.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1368042080, 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637495, 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/Makefile" 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1457475207, 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637474, 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare.sh" 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.229 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1490005321, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636374, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/README" 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1679498754, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636364, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/sysvipc" 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1996395839, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637471, 2024-03-23T15:45:03.230 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_child.c" 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 2258762834, 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637472, 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/Makefile" 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 2471885066, 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636405, 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/netns" 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 2640685105, 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636354, 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/utsname" 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3053510143, 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636401, 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/libclone" 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3193885220, 2024-03-23T15:45:03.232 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636383, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/pidns" 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3525759293, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636362, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile" 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3683330549, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511637473, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_parent.c" 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3852089973, 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636375, 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/mqns" 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3905940617, 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511636363, 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/container_test.sh" 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:45:03.234 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a000a2230 msgr2=0x7f7a000a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a000a2230 0x7f7a000a46f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c000c00 tx=0x7f7a0c034be0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 msgr2=0x7f7a18196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 0x7f7a18196a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fd0 tx=0x7f7a1000c800 comp rx=0 tx=0).stop 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 shutdown_connections 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f7a000a2230 0x7f7a000a46f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a18106660 0x7f7a18196a00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a18100f60 0x7f7a1819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 --2- 172.21.15.76:0/2203800707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a181005a0 0x7f7a1819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 >> 172.21.15.76:0/2203800707 conn(0x7f7a18077710 msgr2=0x7f7a180ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 shutdown_connections 2024-03-23T15:45:03.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.216+0000 7f7a1e2cd640 1 -- 172.21.15.76:0/2203800707 wait complete. 2024-03-23T15:45:03.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 -- 172.21.15.76:0/830615200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f895010bd50 msgr2=0x7f895010e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 --2- 172.21.15.76:0/830615200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f895010bd50 0x7f895010e1c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8948009ca0 tx=0x7f894802c180 comp rx=0 tx=0).stop 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 -- 172.21.15.76:0/830615200 shutdown_connections 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 --2- 172.21.15.76:0/830615200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f895010bd50 0x7f895010e1c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 --2- 172.21.15.76:0/830615200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89501032a0 0x7f895010b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 --2- 172.21.15.76:0/830615200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8950102980 0x7f8950102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 -- 172.21.15.76:0/830615200 >> 172.21.15.76:0/830615200 conn(0x7f8950077770 msgr2=0x7f8950077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 -- 172.21.15.76:0/830615200 shutdown_connections 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.338+0000 7f89569a7640 1 -- 172.21.15.76:0/830615200 wait complete. 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 Processor -- start 2024-03-23T15:45:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 -- start start 2024-03-23T15:45:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950102980 0x7f8950199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89501032a0 0x7f895019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895010bd50 0x7f895019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f895019ee10 con 0x7f8950102980 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f895019ef80 con 0x7f89501032a0 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f89569a7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f895019f140 con 0x7f895010bd50 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950102980 0x7f8950199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950102980 0x7f8950199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35862/0 (socket says 172.21.15.76:35862) 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 -- 172.21.15.76:0/2229228379 learned_addr learned my addr 172.21.15.76:0/2229228379 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f8954f44640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895010bd50 0x7f895019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894f7fe640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89501032a0 0x7f895019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 -- 172.21.15.76:0/2229228379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895010bd50 msgr2=0x7f895019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895010bd50 0x7f895019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 -- 172.21.15.76:0/2229228379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89501032a0 msgr2=0x7f895019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89501032a0 0x7f895019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.339+0000 7f894ffff640 1 -- 172.21.15.76:0/2229228379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8944009490 con 0x7f8950102980 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.340+0000 7f894ffff640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950102980 0x7f8950199f50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f894400ea60 tx=0x7f894400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.340+0000 7f894d7fa640 1 -- 172.21.15.76:0/2229228379 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894400cde0 con 0x7f8950102980 2024-03-23T15:45:03.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.340+0000 7f894d7fa640 1 -- 172.21.15.76:0/2229228379 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89440035f0 con 0x7f8950102980 2024-03-23T15:45:03.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.340+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8948009900 con 0x7f8950102980 2024-03-23T15:45:03.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.340+0000 7f894d7fa640 1 -- 172.21.15.76:0/2229228379 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944010690 con 0x7f8950102980 2024-03-23T15:45:03.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.340+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89501a6ac0 con 0x7f8950102980 2024-03-23T15:45:03.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.342+0000 7f894d7fa640 1 -- 172.21.15.76:0/2229228379 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f8944003af0 con 0x7f8950102980 2024-03-23T15:45:03.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.342+0000 7f8959581740 1 Processor -- start 2024-03-23T15:45:03.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.342+0000 7f8959581740 1 -- start start 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.342+0000 7f8959581740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c43f0 0x55b2229c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.342+0000 7f894d7fa640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f89380a2230 0x7f89380a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8959581740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c4d10 0x55b2229cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8959581740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8959581740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b222932840 con 0x55b2229cd8f0 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8959581740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b22292fd70 con 0x55b2229c43f0 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8959581740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b22292b550 con 0x55b2229c4d10 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894ffff640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f89380a2230 0x7f89380a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8954f44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c4d10 0x55b2229cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8954f44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35872/0 (socket says 172.21.15.76:35872) 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8954f44640 1 -- 172.21.15.76:0/1872676846 learned_addr learned my addr 172.21.15.76:0/1872676846 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894d7fa640 1 -- 172.21.15.76:0/2229228379 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8944014070 con 0x7f8950102980 2024-03-23T15:45:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8954f44640 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c43f0 0x55b2229c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894f7fe640 1 -- 172.21.15.76:0/1872676846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c43f0 msgr2=0x55b2229c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894f7fe640 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c43f0 0x55b2229c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894f7fe640 1 -- 172.21.15.76:0/1872676846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 msgr2=0x55b2229cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894f7fe640 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894f7fe640 1 -- 172.21.15.76:0/1872676846 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2228c0de0 con 0x55b2229c4d10 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f8954f44640 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894ffff640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f89380a2230 0x7f89380a46f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8944099da0 tx=0x7f894406cea0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f894f7fe640 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c4d10 0x55b2229cd3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8940009bd0 tx=0x7f8940011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36ed503b50a85f57 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f89367fc640 1 -- 172.21.15.76:0/1872676846 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940004050 con 0x55b2229c4d10 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f89367fc640 1 -- 172.21.15.76:0/1872676846 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8940005500 con 0x55b2229c4d10 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.343+0000 7f89367fc640 1 -- 172.21.15.76:0/1872676846 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940004c00 con 0x55b2229c4d10 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 -- 172.21.15.76:0/1872676846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c4d10 msgr2=0x55b2229cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c4d10 0x55b2229cd3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8940009bd0 tx=0x7f8940011810 comp rx=0 tx=0).stop 2024-03-23T15:45:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 -- 172.21.15.76:0/1872676846 shutdown_connections 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c4d10 0x55b2229cd3b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 --2- 172.21.15.76:0/1872676846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c43f0 0x55b2229c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 -- 172.21.15.76:0/1872676846 >> 172.21.15.76:0/1872676846 conn(0x55b2229beb20 msgr2=0x55b2229bef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 -- 172.21.15.76:0/1872676846 shutdown_connections 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 -- 172.21.15.76:0/1872676846 wait complete. 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.344+0000 7f8959581740 1 -- start start 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 1 Processor -- start 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b2225fe430 con 0x55b2229cd8f0 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2225bae00 con 0x55b2229c4d10 2024-03-23T15:45:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b2228b0590 con 0x55b2229c43f0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8959581740 10 client.0 _ll_register_callbacks cb 0x55b22291ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35874/0 (socket says 172.21.15.76:35874) 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 -- 172.21.15.76:0/2170105491 learned_addr learned my addr 172.21.15.76:0/2170105491 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f894f7fe640 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 -- 172.21.15.76:0/2170105491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 msgr2=0x55b2229ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 -- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 msgr2=0x55b2229caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.345+0000 7f8954f44640 1 -- 172.21.15.76:0/2170105491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8940009830 con 0x55b2229cd8f0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8954f44640 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cb400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8948005cc0 tx=0x7f894802c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29227aa888bfb13 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f892f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:03.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f892f7fe640 1 -- 172.21.15.76:0/2170105491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948032ba0 con 0x55b2229cd8f0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b22262a7e0 con 0x55b2229cd8f0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f892f7fe640 1 -- 172.21.15.76:0/2170105491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8948032d00 con 0x55b2229cd8f0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f892f7fe640 1 -- 172.21.15.76:0/2170105491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8948032ec0 con 0x55b2229cd8f0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 10 client.15966 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b222929bd0 con 0x55b2229cd8f0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f892f7fe640 1 -- 172.21.15.76:0/2170105491 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b22262a7e0 con 0x55b2229cd8f0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f892f7fe640 1 -- 172.21.15.76:0/2170105491 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8948042540 con 0x55b2229cd8f0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 10 client.15966 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 20 client.15966 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b2229dd9d0 0x55b2229dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 4 client.15966 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b222667720 con 0x55b2229dd9d0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.346+0000 7f894f7fe640 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b2229dd9d0 0x55b2229dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.347+0000 7f892f7fe640 10 client.15966 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.347+0000 7f894f7fe640 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b2229dd9d0 0x55b2229dfdf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2c0ed0c6524d79 server_cookie=b6b13d6a01abc73a in_seq=0 out_seq=0 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.347+0000 7f892f7fe640 1 -- 172.21.15.76:0/2170105491 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b222667720 con 0x55b2229dd9d0 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.347+0000 7f892f7fe640 10 client.15966 handle_command_reply: tid=0 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.347+0000 7f8959581740 1 client.15966 shutdown 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 msgr2=0x55b2229cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cb400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8948005cc0 tx=0x7f894802c180 comp rx=0 tx=0).stop 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 shutdown_connections 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229cb400 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b2229dd9d0 0x55b2229dfdf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 --2- 172.21.15.76:0/2170105491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 >> 172.21.15.76:0/2170105491 conn(0x55b2229beb20 msgr2=0x55b2229c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f892f7fe640 0 client.15966 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:03.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 shutdown_connections 2024-03-23T15:45:03.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 1 -- 172.21.15.76:0/2170105491 wait complete. 2024-03-23T15:45:03.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.348+0000 7f8959581740 20 client.15966 trim_cache size 0 max 16384 2024-03-23T15:45:03.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 Processor -- start 2024-03-23T15:45:03.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 -- start start 2024-03-23T15:45:03.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229d9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b2229cbe10 con 0x55b2229cd8f0 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2229dc200 con 0x55b2229c4d10 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8959581740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b2229cd3e0 con 0x55b2229c43f0 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229e56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35888/0 (socket says 172.21.15.76:35888) 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 -- 172.21.15.76:0/2985401601 learned_addr learned my addr 172.21.15.76:0/2985401601 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f894f7fe640 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 -- 172.21.15.76:0/2985401601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 msgr2=0x55b2229c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 -- 172.21.15.76:0/2985401601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 msgr2=0x55b2229d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 -- 172.21.15.76:0/2985401601 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2228c5c80 con 0x55b2229cd8f0 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f894f7fe640 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8954f44640 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229e56a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8948005c90 tx=0x7f8948034540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c98dbcd7cac1e13 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8935ffb640 1 -- 172.21.15.76:0/2985401601 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948003b20 con 0x55b2229cd8f0 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.361+0000 7f8935ffb640 1 -- 172.21.15.76:0/2985401601 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8948048040 con 0x55b2229cd8f0 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8935ffb640 1 -- 172.21.15.76:0/2985401601 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948003e90 con 0x55b2229cd8f0 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 -- 172.21.15.76:0/2985401601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 msgr2=0x55b2229e56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229e56a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8948005c90 tx=0x7f8948034540 comp rx=0 tx=0).stop 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 -- 172.21.15.76:0/2985401601 shutdown_connections 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229cd8f0 0x55b2229e56a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229c4d10 0x55b2229d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 --2- 172.21.15.76:0/2985401601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 -- 172.21.15.76:0/2985401601 >> 172.21.15.76:0/2985401601 conn(0x55b2229beb20 msgr2=0x55b2229c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 -- 172.21.15.76:0/2985401601 shutdown_connections 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 -- 172.21.15.76:0/2985401601 wait complete. 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.362+0000 7f8959581740 1 -- start start 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 Processor -- start 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229d5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229c4d10 0x55b2229d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229cd8f0 0x55b2229d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b2229d0f20 con 0x55b2229c4d10 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2229d1090 con 0x55b2229cd8f0 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b2229d1250 con 0x55b2229c43f0 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 10 client.0 _ll_register_callbacks cb 0x7f8948009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229c4d10 0x55b2229d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229c4d10 0x55b2229d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35892/0 (socket says 172.21.15.76:35892) 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 -- 172.21.15.76:0/582168219 learned_addr learned my addr 172.21.15.76:0/582168219 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:03.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8954f44640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229d5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8954f44640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229cd8f0 0x55b2229d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 -- 172.21.15.76:0/582168219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 msgr2=0x55b2229d5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229d5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 -- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229cd8f0 msgr2=0x55b2229d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229cd8f0 0x55b2229d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 -- 172.21.15.76:0/582168219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b222929bd0 con 0x55b2229c4d10 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8954f44640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229cd8f0 0x55b2229d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f894f7fe640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229c4d10 0x55b2229d5990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8940009bd0 tx=0x7f89400044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6f7f3bfa5834de server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f892f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f892f7fe640 1 -- 172.21.15.76:0/582168219 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940029070 con 0x55b2229c4d10 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f892f7fe640 1 -- 172.21.15.76:0/582168219 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f894002eab0 con 0x55b2229c4d10 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.363+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8940005500 con 0x55b2229c4d10 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f892f7fe640 1 -- 172.21.15.76:0/582168219 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8940009ea0 con 0x55b2229c4d10 2024-03-23T15:45:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 10 client.15972 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2228eb920 con 0x55b2229c4d10 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f892f7fe640 1 -- 172.21.15.76:0/582168219 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894003b020 con 0x55b2229c4d10 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f892f7fe640 1 -- 172.21.15.76:0/582168219 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8940021040 con 0x55b2229c4d10 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 20 client.15972 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b222a44020 0x55b222a46440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 4 client.15972 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2226b6890 con 0x55b222a44020 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.364+0000 7f8954f44640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b222a44020 0x55b222a46440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8954f44640 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b222a44020 0x55b222a46440 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a85bbda19ecdd1 server_cookie=55d167e2f6b4193 in_seq=0 out_seq=0 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f892f7fe640 10 client.15972 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f892f7fe640 1 -- 172.21.15.76:0/582168219 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2226b6890 con 0x55b222a44020 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f892f7fe640 10 client.15972 handle_command_reply: tid=0 2024-03-23T15:45:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 client.15972 shutdown 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229c4d10 msgr2=0x55b2229d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229c4d10 0x55b2229d5990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8940009bd0 tx=0x7f89400044c0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 shutdown_connections 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b222a44020 0x55b222a46440 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2229cd8f0 0x55b2229d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b2229c4d10 0x55b2229d5990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f892f7fe640 0 client.15972 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 --2- 172.21.15.76:0/582168219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2229c43f0 0x55b2229d5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 >> 172.21.15.76:0/582168219 conn(0x55b2229beb20 msgr2=0x55b2229c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 shutdown_connections 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 1 -- 172.21.15.76:0/582168219 wait complete. 2024-03-23T15:45:03.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.365+0000 7f8959581740 20 client.15972 trim_cache size 0 max 16384 2024-03-23T15:45:03.378 INFO:teuthology.orchestra.run.smithi076.stdout:[] 2024-03-23T15:45:03.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f89380a2230 msgr2=0x7f89380a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f89380a2230 0x7f89380a46f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8944099da0 tx=0x7f894406cea0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950102980 msgr2=0x7f8950199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950102980 0x7f8950199f50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f894400ea60 tx=0x7f894400ef20 comp rx=0 tx=0).stop 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 shutdown_connections 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f89380a2230 0x7f89380a46f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f895010bd50 0x7f895019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89501032a0 0x7f895019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 --2- 172.21.15.76:0/2229228379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950102980 0x7f8950199f50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 >> 172.21.15.76:0/2229228379 conn(0x7f8950077770 msgr2=0x7f8950107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 shutdown_connections 2024-03-23T15:45:03.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:45:03.366+0000 7f89569a7640 1 -- 172.21.15.76:0/2229228379 wait complete. 2024-03-23T15:45:03.380 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 429803148, 'ino': 1099511637463, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.sh'}, {'damage_type': 'backtrace', 'id': 638993242, 'ino': 1099511637466, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_ioctl.c'}, {'damage_type': 'backtrace', 'id': 3667474593, 'ino': 1099511637465, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/selinux_noioctl.c'}, {'damage_type': 'backtrace', 'id': 4285727850, 'ino': 1099511637464, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/ioctl/Makefile'}] 2024-03-23T15:45:03.381 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 911299051, 'ino': 1099511637494, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/selinux_rxdir.sh'}, {'damage_type': 'backtrace', 'id': 1154926179, 'ino': 1099511636440, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile.inc'}, {'damage_type': 'backtrace', 'id': 1263995757, 'ino': 1099511636361, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/TEST_PLAN.txt'}, {'damage_type': 'backtrace', 'id': 1368042080, 'ino': 1099511637495, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/rxdir/Makefile'}, {'damage_type': 'backtrace', 'id': 1457475207, 'ino': 1099511637474, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare.sh'}, {'damage_type': 'backtrace', 'id': 1490005321, 'ino': 1099511636374, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/README'}, {'damage_type': 'backtrace', 'id': 1679498754, 'ino': 1099511636364, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/sysvipc'}, {'damage_type': 'backtrace', 'id': 1996395839, 'ino': 1099511637471, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_child.c'}, {'damage_type': 'backtrace', 'id': 2258762834, 'ino': 1099511637472, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/Makefile'}, {'damage_type': 'backtrace', 'id': 2471885066, 'ino': 1099511636405, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/netns'}, {'damage_type': 'backtrace', 'id': 2640685105, 'ino': 1099511636354, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/utsname'}, {'damage_type': 'backtrace', 'id': 3053510143, 'ino': 1099511636401, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/libclone'}, {'damage_type': 'backtrace', 'id': 3193885220, 'ino': 1099511636383, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/pidns'}, {'damage_type': 'backtrace', 'id': 3525759293, 'ino': 1099511636362, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/Makefile'}, {'damage_type': 'backtrace', 'id': 3683330549, 'ino': 1099511637473, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/security/selinux-testsuite/tests/execshare/selinux_execshare_parent.c'}, {'damage_type': 'backtrace', 'id': 3852089973, 'ino': 1099511636375, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/mqns'}, {'damage_type': 'backtrace', 'id': 3905940617, 'ino': 1099511636363, 'path': '/volumes/qa/sv_1/3c3ad643-b4c6-4143-9aae-2dd532c299c4/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/containers/container_test.sh'}] 2024-03-23T15:45:03.381 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-23T15:45:03.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:03 smithi192 ceph-mon[36653]: pgmap v54: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 MiB/s rd, 327 MiB/s wr, 1.73k op/s 2024-03-23T15:45:03.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:03 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1759073455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:45:03.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:03 smithi076 ceph-mon[32176]: pgmap v54: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 MiB/s rd, 327 MiB/s wr, 1.73k op/s 2024-03-23T15:45:03.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:03 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1759073455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:45:03.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:03 smithi183 ceph-mon[36296]: pgmap v54: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 264 MiB/s rd, 327 MiB/s wr, 1.73k op/s 2024-03-23T15:45:03.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:03 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1759073455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:45:05.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:05 smithi192 ceph-mon[36653]: pgmap v55: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 208 MiB/s rd, 253 MiB/s wr, 1.13k op/s 2024-03-23T15:45:05.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:05 smithi076 ceph-mon[32176]: pgmap v55: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 208 MiB/s rd, 253 MiB/s wr, 1.13k op/s 2024-03-23T15:45:05.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:05 smithi183 ceph-mon[36296]: pgmap v55: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 208 MiB/s rd, 253 MiB/s wr, 1.13k op/s 2024-03-23T15:45:07.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:07 smithi192 ceph-mon[36653]: pgmap v56: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 MiB/s rd, 259 MiB/s wr, 1.14k op/s 2024-03-23T15:45:07.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:07 smithi076 ceph-mon[32176]: pgmap v56: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 MiB/s rd, 259 MiB/s wr, 1.14k op/s 2024-03-23T15:45:07.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:07 smithi183 ceph-mon[36296]: pgmap v56: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 MiB/s rd, 259 MiB/s wr, 1.14k op/s 2024-03-23T15:45:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.098+0000 7ff036683640 1 -- 172.21.15.76:0/4165861585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301020a0 msgr2=0x7ff03010e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.098+0000 7ff036683640 1 --2- 172.21.15.76:0/4165861585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301020a0 0x7ff03010e5b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff028009cd0 tx=0x7ff02802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.098+0000 7ff036683640 1 -- 172.21.15.76:0/4165861585 shutdown_connections 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.098+0000 7ff036683640 1 --2- 172.21.15.76:0/4165861585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301020a0 0x7ff03010e5b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.098+0000 7ff036683640 1 --2- 172.21.15.76:0/4165861585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0301016e0 0x7ff030101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.098+0000 7ff036683640 1 --2- 172.21.15.76:0/4165861585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0301076f0 0x7ff030107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.098+0000 7ff036683640 1 -- 172.21.15.76:0/4165861585 >> 172.21.15.76:0/4165861585 conn(0x7ff0300fd1f0 msgr2=0x7ff0300ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.099+0000 7ff036683640 1 -- 172.21.15.76:0/4165861585 shutdown_connections 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.099+0000 7ff036683640 1 -- 172.21.15.76:0/4165861585 wait complete. 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.099+0000 7ff036683640 1 Processor -- start 2024-03-23T15:45:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.099+0000 7ff036683640 1 -- start start 2024-03-23T15:45:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff036683640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0301016e0 0x7ff03019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff036683640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0301020a0 0x7ff03019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff036683640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301076f0 0x7ff030197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff036683640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301980b0 con 0x7ff0301076f0 2024-03-23T15:45:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff036683640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff030198220 con 0x7ff0301020a0 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff036683640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301983e0 con 0x7ff0301016e0 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301076f0 0x7ff030197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301076f0 0x7ff030197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47868/0 (socket says 172.21.15.76:47868) 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 -- 172.21.15.76:0/2429696313 learned_addr learned my addr 172.21.15.76:0/2429696313 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff02ffff640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0301016e0 0x7ff03019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff02f7fe640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0301020a0 0x7ff03019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 -- 172.21.15.76:0/2429696313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0301016e0 msgr2=0x7ff03019d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0301016e0 0x7ff03019d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 -- 172.21.15.76:0/2429696313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0301020a0 msgr2=0x7ff03019ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0301020a0 0x7ff03019ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 -- 172.21.15.76:0/2429696313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff020009490 con 0x7ff0301076f0 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff02ffff640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0301016e0 0x7ff03019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff034c20640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301076f0 0x7ff030197a60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff028004af0 tx=0x7ff028032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff02d7fa640 1 -- 172.21.15.76:0/2429696313 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280328b0 con 0x7ff0301076f0 2024-03-23T15:45:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff028009900 con 0x7ff0301076f0 2024-03-23T15:45:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.100+0000 7ff02d7fa640 1 -- 172.21.15.76:0/2429696313 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff028003e50 con 0x7ff0301076f0 2024-03-23T15:45:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.101+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0301a4770 con 0x7ff0301076f0 2024-03-23T15:45:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.101+0000 7ff02d7fa640 1 -- 172.21.15.76:0/2429696313 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028004700 con 0x7ff0301076f0 2024-03-23T15:45:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff02d7fa640 1 -- 172.21.15.76:0/2429696313 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7ff02800a040 con 0x7ff0301076f0 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 Processor -- start 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff02d7fa640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff0180a22d0 0x7ff0180a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 -- start start 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff02d7fa640 1 -- 172.21.15.76:0/2429696313 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff0280f0aa0 con 0x7ff0301076f0 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff02f7fe640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff0180a22d0 0x7ff0180a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96ccf00 0x5563e96d5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96d5cd0 0x5563e96d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563e96871b0 con 0x5563e96ccf00 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563e9639f80 con 0x5563e96d5cd0 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff03925a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5563e96374b0 con 0x5563e96cc5e0 2024-03-23T15:45:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96ccf00 0x5563e96d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.103+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96ccf00 0x5563e96d5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47870/0 (socket says 172.21.15.76:47870) 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02f7fe640 1 -- 172.21.15.76:0/2195035030 learned_addr learned my addr 172.21.15.76:0/2195035030 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02ffff640 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff034c20640 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96d5cd0 0x5563e96d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02f7fe640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff0180a22d0 0x7ff0180a4790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff024013570 tx=0x7ff024013970 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02ffff640 1 -- 172.21.15.76:0/2195035030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96d5cd0 msgr2=0x5563e96d80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02ffff640 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96d5cd0 0x5563e96d80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02ffff640 1 -- 172.21.15.76:0/2195035030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96ccf00 msgr2=0x5563e96d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02ffff640 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96ccf00 0x5563e96d5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02ffff640 1 -- 172.21.15.76:0/2195035030 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563e95c8810 con 0x5563e96cc5e0 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02f7fe640 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96ccf00 0x5563e96d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff034c20640 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96d5cd0 0x5563e96d80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff02ffff640 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96cc9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0200114f0 tx=0x7ff0200119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4c83af1f813ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff0167fc640 1 -- 172.21.15.76:0/2195035030 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020004030 con 0x5563e96cc5e0 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff0167fc640 1 -- 172.21.15.76:0/2195035030 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff020005020 con 0x5563e96cc5e0 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.104+0000 7ff0167fc640 1 -- 172.21.15.76:0/2195035030 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020004c60 con 0x5563e96cc5e0 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 -- 172.21.15.76:0/2195035030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 msgr2=0x5563e96cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96cc9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0200114f0 tx=0x7ff0200119b0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 -- 172.21.15.76:0/2195035030 shutdown_connections 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96d5cd0 0x5563e96d80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96ccf00 0x5563e96d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 --2- 172.21.15.76:0/2195035030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96cc9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 -- 172.21.15.76:0/2195035030 >> 172.21.15.76:0/2195035030 conn(0x5563e96c6f00 msgr2=0x5563e96c72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 -- 172.21.15.76:0/2195035030 shutdown_connections 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 -- 172.21.15.76:0/2195035030 wait complete. 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.105+0000 7ff03925a740 1 -- start start 2024-03-23T15:45:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 1 Processor -- start 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96cc5e0 0x5563e96d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96ccf00 0x5563e96d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563e9632c90 con 0x5563e96d5cd0 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563e95d7c50 con 0x5563e96cc5e0 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5563e9306250 con 0x5563e96ccf00 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff03925a740 10 client.0 _ll_register_callbacks cb 0x5563e9626290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96cc5e0 0x5563e96d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff034c20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96ccf00 0x5563e96d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96ccf00 0x5563e96d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:53448/0 (socket says 172.21.15.76:53448) 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96cc5e0 0x5563e96d2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:33564/0 (socket says 172.21.15.76:33564) 2024-03-23T15:45:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff02f7fe640 1 -- 172.21.15.76:0/1109134949 learned_addr learned my addr 172.21.15.76:0/1109134949 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff034c20640 1 -- 172.21.15.76:0/1109134949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96ccf00 msgr2=0x5563e96d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff034c20640 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96ccf00 0x5563e96d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff034c20640 1 -- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96cc5e0 msgr2=0x5563e96d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff034c20640 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96cc5e0 0x5563e96d2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff034c20640 1 -- 172.21.15.76:0/1109134949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff020009d90 con 0x5563e96d5cd0 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff02ffff640 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96cc5e0 0x5563e96d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff034c20640 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96d3760 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0280ee300 tx=0x7ff0280bd1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1c7734af3e1a14 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.106+0000 7ff00f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff00f7fe640 1 -- 172.21.15.76:0/1109134949 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280bdca0 con 0x5563e96d5cd0 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff00f7fe640 1 -- 172.21.15.76:0/1109134949 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff0280bd5a0 con 0x5563e96d5cd0 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5563e9657790 con 0x5563e96d5cd0 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff00f7fe640 1 -- 172.21.15.76:0/1109134949 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff0280c16f0 con 0x5563e96d5cd0 2024-03-23T15:45:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 10 client.15984 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563e9631310 con 0x5563e96d5cd0 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff00f7fe640 1 -- 172.21.15.76:0/1109134949 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5563e9657790 con 0x5563e96d5cd0 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff00f7fe640 1 -- 172.21.15.76:0/1109134949 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff0280bd800 con 0x5563e96d5cd0 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 10 client.15984 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 20 client.15984 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e96e59f0 0x5563e96e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 4 client.15984 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563e9332600 con 0x5563e96e59f0 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.107+0000 7ff02ffff640 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e96e59f0 0x5563e96e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.108+0000 7ff02ffff640 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e96e59f0 0x5563e96e7e10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68750ff59902f0db server_cookie=9f801268a4737c81 in_seq=0 out_seq=0 2024-03-23T15:45:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.108+0000 7ff00f7fe640 10 client.15984 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.108+0000 7ff00f7fe640 1 -- 172.21.15.76:0/1109134949 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5563e9332600 con 0x5563e96e59f0 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.108+0000 7ff00f7fe640 10 client.15984 handle_command_reply: tid=0 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 client.15984 shutdown 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 msgr2=0x5563e96d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96d3760 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0280ee300 tx=0x7ff0280bd1b0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 shutdown_connections 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96d3760 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96ccf00 0x5563e96d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e96e59f0 0x5563e96e7e10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 --2- 172.21.15.76:0/1109134949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96cc5e0 0x5563e96d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff00f7fe640 0 client.15984 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 >> 172.21.15.76:0/1109134949 conn(0x5563e96c6f00 msgr2=0x5563e96c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 shutdown_connections 2024-03-23T15:45:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 1 -- 172.21.15.76:0/1109134949 wait complete. 2024-03-23T15:45:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.109+0000 7ff03925a740 20 client.15984 trim_cache size 0 max 16384 2024-03-23T15:45:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.123+0000 7ff03925a740 1 Processor -- start 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff03925a740 1 -- start start 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff03925a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff03925a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff03925a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff03925a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563e95b8660 con 0x5563e96d5cd0 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff03925a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563e96c5420 con 0x5563e96ccf00 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff03925a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5563e96e1960 con 0x5563e96cc5e0 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47888/0 (socket says 172.21.15.76:47888) 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 -- 172.21.15.76:0/1149617753 learned_addr learned my addr 172.21.15.76:0/1149617753 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff02ffff640 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 -- 172.21.15.76:0/1149617753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 msgr2=0x5563e96e13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96e13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 -- 172.21.15.76:0/1149617753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 msgr2=0x5563e96d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 -- 172.21.15.76:0/1149617753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563e95cd810 con 0x5563e96d5cd0 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff034c20640 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96c5d20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff028009430 tx=0x7ff0280bbab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4513fa313403e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff015ffb640 1 -- 172.21.15.76:0/1149617753 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280bd390 con 0x5563e96d5cd0 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff015ffb640 1 -- 172.21.15.76:0/1149617753 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff028004860 con 0x5563e96d5cd0 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.124+0000 7ff015ffb640 1 -- 172.21.15.76:0/1149617753 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0280dd0b0 con 0x5563e96d5cd0 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 -- 172.21.15.76:0/1149617753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 msgr2=0x5563e96c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96c5d20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff028009430 tx=0x7ff0280bbab0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 -- 172.21.15.76:0/1149617753 shutdown_connections 2024-03-23T15:45:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96d5cd0 0x5563e96c5d20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 --2- 172.21.15.76:0/1149617753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96cc5e0 0x5563e96e13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 -- 172.21.15.76:0/1149617753 >> 172.21.15.76:0/1149617753 conn(0x5563e96c6f00 msgr2=0x5563e96c5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 -- 172.21.15.76:0/1149617753 shutdown_connections 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 -- 172.21.15.76:0/1149617753 wait complete. 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.125+0000 7ff03925a740 1 -- start start 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 1 Processor -- start 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96cc5e0 0x5563e96cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96d5cd0 0x5563e96d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563e974ad50 con 0x5563e96cc5e0 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563e974ae90 con 0x5563e96ccf00 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5563e974b050 con 0x5563e96d5cd0 2024-03-23T15:45:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff03925a740 10 client.0 _ll_register_callbacks cb 0x7ff0280b9930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96cc5e0 0x5563e96cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96cc5e0 0x5563e96cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47896/0 (socket says 172.21.15.76:47896) 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 -- 172.21.15.76:0/264276200 learned_addr learned my addr 172.21.15.76:0/264276200 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02f7fe640 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff034c20640 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96d5cd0 0x5563e96d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 -- 172.21.15.76:0/264276200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96d5cd0 msgr2=0x5563e96d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96d5cd0 0x5563e96d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 -- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 msgr2=0x5563e96cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 -- 172.21.15.76:0/264276200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563e9631310 con 0x5563e96cc5e0 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff02ffff640 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96cc5e0 0x5563e96cfa30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93eed16c78357226 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff00f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff00f7fe640 1 -- 172.21.15.76:0/264276200 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02002c040 con 0x5563e96cc5e0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.126+0000 7ff00f7fe640 1 -- 172.21.15.76:0/264276200 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff020004440 con 0x5563e96cc5e0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff020005020 con 0x5563e96cc5e0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff00f7fe640 1 -- 172.21.15.76:0/264276200 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff020004600 con 0x5563e96cc5e0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 10 client.15990 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5563e95f35a0 con 0x5563e96cc5e0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff00f7fe640 1 -- 172.21.15.76:0/264276200 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff02003e020 con 0x5563e96cc5e0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff00f7fe640 1 -- 172.21.15.76:0/264276200 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff020021040 con 0x5563e96cc5e0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 20 client.15990 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e974c8b0 0x5563e974ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 4 client.15990 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563e93831b0 con 0x5563e974c8b0 2024-03-23T15:45:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.127+0000 7ff02f7fe640 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e974c8b0 0x5563e974ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.128+0000 7ff00f7fe640 10 client.15990 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:08.128+0000 7ff02f7fe640 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e974c8b0 0x5563e974ecd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3baabcb49debebd3 server_cookie=d404f240d168b3c2 in_seq=0 out_seq=0 2024-03-23T15:45:09.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:09 smithi192 ceph-mon[36653]: pgmap v57: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 221 MiB/s rd, 268 MiB/s wr, 1.14k op/s 2024-03-23T15:45:09.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:09 smithi076 ceph-mon[32176]: pgmap v57: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 221 MiB/s rd, 268 MiB/s wr, 1.14k op/s 2024-03-23T15:45:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:09 smithi183 ceph-mon[36296]: pgmap v57: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 221 MiB/s rd, 268 MiB/s wr, 1.14k op/s 2024-03-23T15:45:11.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:11 smithi192 ceph-mon[36653]: pgmap v58: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s rd, 166 MiB/s wr, 310 op/s 2024-03-23T15:45:11.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:11 smithi076 ceph-mon[32176]: pgmap v58: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s rd, 166 MiB/s wr, 310 op/s 2024-03-23T15:45:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:11 smithi183 ceph-mon[36296]: pgmap v58: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s rd, 166 MiB/s wr, 310 op/s 2024-03-23T15:45:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:13 smithi192 ceph-mon[36653]: pgmap v59: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s rd, 166 MiB/s wr, 545 op/s 2024-03-23T15:45:13.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:13 smithi076 ceph-mon[32176]: pgmap v59: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s rd, 166 MiB/s wr, 545 op/s 2024-03-23T15:45:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:13 smithi183 ceph-mon[36296]: pgmap v59: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 142 MiB/s rd, 166 MiB/s wr, 545 op/s 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff00f7fe640 1 -- 172.21.15.76:0/264276200 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563e93831b0 con 0x5563e974c8b0 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff00f7fe640 10 client.15990 handle_command_reply: tid=0 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 client.15990 shutdown 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96cc5e0 msgr2=0x5563e96cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96cc5e0 0x5563e96cfa30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff020000c00 tx=0x7ff020005ce0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 shutdown_connections 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563e96d5cd0 0x5563e96d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x5563e974c8b0 0x5563e974ecd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563e96ccf00 0x5563e96cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 --2- 172.21.15.76:0/264276200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563e96cc5e0 0x5563e96cfa30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff00f7fe640 0 client.15990 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 >> 172.21.15.76:0/264276200 conn(0x5563e96c6f00 msgr2=0x5563e96c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 shutdown_connections 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 1 -- 172.21.15.76:0/264276200 wait complete. 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.903+0000 7ff03925a740 20 client.15990 trim_cache size 0 max 16384 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:45:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff0180a22d0 msgr2=0x7ff0180a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff0180a22d0 0x7ff0180a4790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff024013570 tx=0x7ff024013970 comp rx=0 tx=0).stop 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301076f0 msgr2=0x7ff030197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301076f0 0x7ff030197a60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff028004af0 tx=0x7ff028032570 comp rx=0 tx=0).stop 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 shutdown_connections 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7ff0180a22d0 0x7ff0180a4790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0301076f0 0x7ff030197a60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0301020a0 0x7ff03019ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 --2- 172.21.15.76:0/2429696313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0301016e0 0x7ff03019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 >> 172.21.15.76:0/2429696313 conn(0x7ff0300fd1f0 msgr2=0x7ff0300fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 shutdown_connections 2024-03-23T15:45:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:13.904+0000 7ff036683640 1 -- 172.21.15.76:0/2429696313 wait complete. 2024-03-23T15:45:13.909 INFO:tasks.workunit.client.0.smithi076.stdout:6/907: write d1/dd/df/d5b/dcd/dd0/fd9 [408770,95565] 0 2024-03-23T15:45:13.918 INFO:tasks.workunit.client.0.smithi076.stdout:6/908: dread d1/dd/d2a/d23/f3a [4194304,4194304] 0 2024-03-23T15:45:13.918 INFO:tasks.workunit.client.0.smithi076.stdout:6/909: truncate d1/dd/df/d5b/d62/d128/f12e 667491 0 2024-03-23T15:45:13.919 INFO:tasks.workunit.client.0.smithi076.stdout:6/910: dread d1/dd/d2a/f8c [0,4194304] 0 2024-03-23T15:45:13.921 INFO:tasks.workunit.client.0.smithi076.stdout:6/911: rmdir d1/d81 39 2024-03-23T15:45:13.921 INFO:tasks.workunit.client.0.smithi076.stdout:6/912: truncate d1/dd/df/f122 458438 0 2024-03-23T15:45:13.922 INFO:tasks.workunit.client.0.smithi076.stdout:6/913: creat d1/dd/df/db5/f13f x:0 0 0 2024-03-23T15:45:13.923 INFO:tasks.workunit.client.0.smithi076.stdout:6/914: symlink d1/d2/dc8/l140 0 2024-03-23T15:45:13.924 INFO:tasks.workunit.client.0.smithi076.stdout:6/915: mknod d1/dd/d2a/dfd/d113/db3/d8d/db8/c141 0 2024-03-23T15:45:13.924 INFO:tasks.workunit.client.0.smithi076.stdout:6/916: read d1/d4/d10/f90 [5968530,74077] 0 2024-03-23T15:45:13.925 INFO:tasks.workunit.client.0.smithi076.stdout:6/917: unlink d1/d4/d10/d39/ca3 0 2024-03-23T15:45:13.946 INFO:tasks.workunit.client.0.smithi076.stdout:6/918: dwrite d1/dd/df/d5b/d62/f7e [0,4194304] 0 2024-03-23T15:45:13.947 INFO:tasks.workunit.client.0.smithi076.stdout:6/919: creat d1/dd/d2a/dfd/d113/db3/d8d/d60/d7c/f142 x:0 0 0 2024-03-23T15:45:14.279 INFO:tasks.workunit.client.0.smithi076.stdout:6/920: sync 2024-03-23T15:45:14.279 INFO:tasks.workunit.client.0.smithi076.stdout:6/921: chown d1/d4/c9d 5 1 2024-03-23T15:45:14.292 INFO:tasks.workunit.client.0.smithi076.stdout:6/922: dwrite d1/d81/f9a [4194304,4194304] 0 2024-03-23T15:45:14.293 INFO:tasks.workunit.client.0.smithi076.stdout:6/923: mkdir d1/dd/d2a/d143 0 2024-03-23T15:45:14.302 INFO:tasks.workunit.client.0.smithi076.stdout:6/924: write d1/dd/df/fa4 [1431715,128285] 0 2024-03-23T15:45:14.304 INFO:tasks.workunit.client.0.smithi076.stdout:6/925: creat d1/d4/f144 x:0 0 0 2024-03-23T15:45:14.307 INFO:tasks.workunit.client.0.smithi076.stdout:6/926: link d1/dd/d2a/dfd/d113/db3/l13e d1/d2/l145 0 2024-03-23T15:45:14.309 INFO:tasks.workunit.client.0.smithi076.stdout:6/927: mkdir d1/d81/df6/d146 0 2024-03-23T15:45:14.310 INFO:tasks.workunit.client.0.smithi076.stdout:6/928: rename d1/dd/d2a/dfd/d113/db3/d8d/db8/de0/f13b to d1/dd/d2a/dfd/d113/db3/f147 0 2024-03-23T15:45:14.311 INFO:tasks.workunit.client.0.smithi076.stdout:6/929: rmdir d1/d4/de2 39 2024-03-23T15:45:14.313 INFO:tasks.workunit.client.0.smithi076.stdout:6/930: creat d1/d2/f148 x:0 0 0 2024-03-23T15:45:14.313 INFO:tasks.workunit.client.0.smithi076.stdout:6/931: readlink d1/dd/d2a/dfd/d113/db3/la8 0 2024-03-23T15:45:14.314 INFO:tasks.workunit.client.0.smithi076.stdout:6/932: rmdir d1/d2/df2 39 2024-03-23T15:45:14.316 INFO:tasks.workunit.client.0.smithi076.stdout:6/933: unlink d1/d4/f144 0 2024-03-23T15:45:14.318 INFO:tasks.workunit.client.0.smithi076.stdout:6/934: symlink d1/dd/df/dac/dec/l149 0 2024-03-23T15:45:14.318 INFO:tasks.workunit.client.0.smithi076.stdout:6/935: fsync d1/d4/f109 0 2024-03-23T15:45:14.326 INFO:tasks.workunit.client.0.smithi076.stdout:6/936: write d1/d81/f9a [4251970,76165] 0 2024-03-23T15:45:14.332 INFO:tasks.workunit.client.0.smithi076.stdout:6/937: read d1/d4/d10/f90 [3867521,44778] 0 2024-03-23T15:45:14.347 INFO:tasks.workunit.client.0.smithi076.stdout:6/938: link d1/dd/d2a/dfd/d113/db3/d8d/db8/fc0 d1/d4/d10/d39/f14a 0 2024-03-23T15:45:14.348 INFO:tasks.workunit.client.0.smithi076.stdout:6/939: truncate d1/dd/df/d5b/d62/d128/f12e 810930 0 2024-03-23T15:45:14.353 INFO:tasks.workunit.client.0.smithi076.stdout:6/940: truncate d1/d4/f131 1558473 0 2024-03-23T15:45:15.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:15 smithi192 ceph-mon[36653]: pgmap v60: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 29 MiB/s wr, 305 op/s 2024-03-23T15:45:15.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:15 smithi076 ceph-mon[32176]: pgmap v60: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 29 MiB/s wr, 305 op/s 2024-03-23T15:45:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:15 smithi183 ceph-mon[36296]: pgmap v60: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 29 MiB/s wr, 305 op/s 2024-03-23T15:45:15.956 INFO:tasks.workunit.client.0.smithi076.stdout:6/941: rename d1/f2c to d1/dd/df/d5b/d62/dc7/f14b 0 2024-03-23T15:45:15.958 INFO:tasks.workunit.client.0.smithi076.stdout:6/942: dread d1/d4/d10/fe9 [0,4194304] 0 2024-03-23T15:45:15.980 INFO:tasks.workunit.client.0.smithi076.stdout:6/943: dwrite d1/d4/f86 [0,4194304] 0 2024-03-23T15:45:15.982 INFO:tasks.workunit.client.0.smithi076.stdout:6/944: mknod d1/d2/c14c 0 2024-03-23T15:45:15.984 INFO:tasks.workunit.client.0.smithi076.stdout:6/945: mkdir d1/dd/d2a/dfd/d113/db3/d8d/d14d 0 2024-03-23T15:45:15.985 INFO:tasks.workunit.client.0.smithi076.stdout:6/946: creat d1/d81/df6/d146/f14e x:0 0 0 2024-03-23T15:45:15.986 INFO:tasks.workunit.client.0.smithi076.stdout:6/947: mknod d1/dd/d2a/dfd/c14f 0 2024-03-23T15:45:15.986 INFO:tasks.workunit.client.0.smithi076.stdout:6/948: truncate d1/dd/df/d5b/d62/dc7/fd6 333804 0 2024-03-23T15:45:15.987 INFO:tasks.workunit.client.0.smithi076.stdout:6/949: unlink d1/dd/d2a/dfd/d113/db3/d8d/f46 0 2024-03-23T15:45:15.988 INFO:tasks.workunit.client.0.smithi076.stdout:6/950: creat d1/d4/de2/f150 x:0 0 0 2024-03-23T15:45:15.990 INFO:tasks.workunit.client.0.smithi076.stdout:6/951: creat d1/d2/df2/f151 x:0 0 0 2024-03-23T15:45:15.992 INFO:tasks.workunit.client.0.smithi076.stdout:6/952: symlink d1/dd/df/d5b/dcd/d10f/l152 0 2024-03-23T15:45:15.993 INFO:tasks.workunit.client.0.smithi076.stdout:6/953: symlink d1/d4/de2/l153 0 2024-03-23T15:45:15.993 INFO:tasks.workunit.client.0.smithi076.stdout:6/954: readlink d1/d4/le4 0 2024-03-23T15:45:16.001 INFO:tasks.workunit.client.0.smithi076.stdout:6/955: read d1/d81/fe3 [1375924,121596] 0 2024-03-23T15:45:16.006 INFO:tasks.workunit.client.0.smithi076.stdout:6/956: getdents d1/d81/df6/d146 0 2024-03-23T15:45:16.007 INFO:tasks.workunit.client.0.smithi076.stdout:6/957: write d1/dd/df/d5b/d62/df7/fa6 [177640,118599] 0 2024-03-23T15:45:16.022 INFO:tasks.workunit.client.0.smithi076.stdout:6/958: rename d1/dd/df/d5b/d62/l69 to d1/dd/d2a/d143/l154 0 2024-03-23T15:45:16.023 INFO:tasks.workunit.client.0.smithi076.stdout:6/959: mkdir d1/dd/d2a/dfd/d113/db3/d8d/db8/de0/d155 0 2024-03-23T15:45:16.036 INFO:tasks.workunit.client.0.smithi076.stdout:6/960: dwrite d1/dd/df/d5b/dcd/f126 [0,4194304] 0 2024-03-23T15:45:16.039 INFO:tasks.workunit.client.0.smithi076.stdout:6/961: creat d1/dd/df/d5b/d62/f156 x:0 0 0 2024-03-23T15:45:16.041 INFO:tasks.workunit.client.0.smithi076.stdout:6/962: creat d1/dd/d2a/dfd/d113/db3/d8d/d60/d7c/f157 x:0 0 0 2024-03-23T15:45:16.041 INFO:tasks.workunit.client.0.smithi076.stdout:6/963: stat d1/dd/d2a/d143 0 2024-03-23T15:45:16.044 INFO:tasks.workunit.client.0.smithi076.stdout:6/964: unlink d1/d4/l15 0 2024-03-23T15:45:16.048 INFO:tasks.workunit.client.0.smithi076.stdout:6/965: write d1/dd/d2a/dfd/d113/db3/fa1 [1739327,31101] 0 2024-03-23T15:45:16.049 INFO:tasks.workunit.client.0.smithi076.stdout:6/966: creat d1/dd/df/f158 x:0 0 0 2024-03-23T15:45:16.049 INFO:tasks.workunit.client.0.smithi076.stdout:6/967: dread - d1/dd/df/d5b/dcd/dd0/f112 zero size 2024-03-23T15:45:16.050 INFO:tasks.workunit.client.0.smithi076.stdout:6/968: chown d1/dd/df/dac/dec/l118 445630125 1 2024-03-23T15:45:16.050 INFO:tasks.workunit.client.0.smithi076.stdout:6/969: dread - d1/d2/df2/f119 zero size 2024-03-23T15:45:16.050 INFO:tasks.workunit.client.0.smithi076.stdout:6/970: symlink d1/dd/df/dac/dec/l159 0 2024-03-23T15:45:16.052 INFO:tasks.workunit.client.0.smithi076.stdout:6/971: creat d1/d81/df6/d146/f15a x:0 0 0 2024-03-23T15:45:16.055 INFO:tasks.workunit.client.0.smithi076.stdout:6/972: unlink d1/d4/c9d 0 2024-03-23T15:45:16.055 INFO:tasks.workunit.client.0.smithi076.stdout:6/973: fsync d1/dd/d2a/dfd/d113/db3/d8d/d60/fae 0 2024-03-23T15:45:16.056 INFO:tasks.workunit.client.0.smithi076.stdout:6/974: mknod d1/d4/de2/c15b 0 2024-03-23T15:45:16.076 INFO:tasks.workunit.client.0.smithi076.stdout:6/975: dwrite d1/dd/df/f122 [0,4194304] 0 2024-03-23T15:45:16.078 INFO:tasks.workunit.client.0.smithi076.stdout:6/976: truncate d1/dd/df/d5b/d62/f7e 3645897 0 2024-03-23T15:45:16.078 INFO:tasks.workunit.client.0.smithi076.stdout:6/977: chown d1 1138022823 1 2024-03-23T15:45:16.078 INFO:tasks.workunit.client.0.smithi076.stdout:6/978: chown d1/dd/df/d5b/d62/f8b 324 1 2024-03-23T15:45:16.078 INFO:tasks.workunit.client.0.smithi076.stdout:6/979: fdatasync d1/f11 0 2024-03-23T15:45:16.081 INFO:tasks.workunit.client.0.smithi076.stdout:6/980: symlink d1/dd/l15c 0 2024-03-23T15:45:16.083 INFO:tasks.workunit.client.0.smithi076.stdout:6/981: mkdir d1/dd/d2a/d15d 0 2024-03-23T15:45:16.083 INFO:tasks.workunit.client.0.smithi076.stdout:6/982: fdatasync d1/dd/d2a/dfd/d113/db3/d8d/db8/fc0 0 2024-03-23T15:45:16.098 INFO:tasks.workunit.client.0.smithi076.stdout:6/983: dwrite d1/d2/f22 [0,4194304] 0 2024-03-23T15:45:16.098 INFO:tasks.workunit.client.0.smithi076.stdout:6/984: dread - d1/dd/df/db5/fc3 zero size 2024-03-23T15:45:16.098 INFO:tasks.workunit.client.0.smithi076.stdout:6/985: stat d1/dd/d2a/dfd/c14f 0 2024-03-23T15:45:16.098 INFO:tasks.workunit.client.0.smithi076.stdout:6/986: write d1/dd/df/db5/f139 [793209,15336] 0 2024-03-23T15:45:16.098 INFO:tasks.workunit.client.0.smithi076.stdout:6/987: fsync d1/dd1/ff1 0 2024-03-23T15:45:16.289 INFO:tasks.workunit.client.0.smithi076.stdout:6/988: sync 2024-03-23T15:45:16.291 INFO:tasks.workunit.client.0.smithi076.stdout:6/989: truncate d1/d81/f9a 2930648 0 2024-03-23T15:45:16.305 INFO:tasks.workunit.client.0.smithi076.stdout:6/990: dwrite d1/dd/d2a/dfd/d113/db3/d8d/db8/fc0 [0,4194304] 0 2024-03-23T15:45:16.307 INFO:tasks.workunit.client.0.smithi076.stdout:6/991: rename d1/d4/cc1 to d1/c15e 0 2024-03-23T15:45:16.308 INFO:tasks.workunit.client.0.smithi076.stdout:6/992: symlink d1/dd/df/dac/l15f 0 2024-03-23T15:45:16.310 INFO:tasks.workunit.client.0.smithi076.stdout:6/993: rename d1/dd/df/fb2 to d1/dd/f160 0 2024-03-23T15:45:16.310 INFO:tasks.workunit.client.0.smithi076.stdout:6/994: write d1/d81/fcb [481958,109226] 0 2024-03-23T15:45:16.319 INFO:tasks.workunit.client.0.smithi076.stdout:6/995: dread d1/dd1/ff1 [0,4194304] 0 2024-03-23T15:45:16.328 INFO:tasks.workunit.client.0.smithi076.stdout:6/996: link d1/ce1 d1/dd/df/dac/dec/c161 0 2024-03-23T15:45:16.344 INFO:tasks.workunit.client.0.smithi076.stdout:6/997: dwrite d1/dd/d2a/dfd/d113/db3/d8d/f9e [0,4194304] 0 2024-03-23T15:45:16.347 INFO:tasks.workunit.client.0.smithi076.stdout:6/998: mkdir d1/dd/d2a/dfd/d113/db3/d8d/d60/d162 0 2024-03-23T15:45:16.348 INFO:tasks.workunit.client.0.smithi076.stdout:6/999: symlink d1/d2/dc8/l163 0 2024-03-23T15:45:16.351 INFO:tasks.workunit.client.0.smithi076.stderr:+ rm -rf -- ./tmp.qNXWRIvOo3 2024-03-23T15:45:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:17 smithi192 ceph-mon[36653]: pgmap v61: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 29 MiB/s wr, 311 op/s 2024-03-23T15:45:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:17 smithi076 ceph-mon[32176]: pgmap v61: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 29 MiB/s wr, 311 op/s 2024-03-23T15:45:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:17 smithi183 ceph-mon[36296]: pgmap v61: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 29 MiB/s wr, 311 op/s 2024-03-23T15:45:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:19 smithi192 ceph-mon[36653]: pgmap v62: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 24 MiB/s wr, 310 op/s 2024-03-23T15:45:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:19 smithi076 ceph-mon[32176]: pgmap v62: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 24 MiB/s wr, 310 op/s 2024-03-23T15:45:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:19 smithi183 ceph-mon[36296]: pgmap v62: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 24 MiB/s wr, 310 op/s 2024-03-23T15:45:21.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:21 smithi192 ceph-mon[36653]: pgmap v63: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 948 KiB/s wr, 244 op/s 2024-03-23T15:45:21.923 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:21 smithi183 ceph-mon[36296]: pgmap v63: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 948 KiB/s wr, 244 op/s 2024-03-23T15:45:21.924 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:21 smithi076 ceph-mon[32176]: pgmap v63: 97 pgs: 97 active+clean; 2.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 948 KiB/s wr, 244 op/s 2024-03-23T15:45:23.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:23 smithi192 ceph-mon[36653]: pgmap v64: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 286 op/s 2024-03-23T15:45:23.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:23 smithi076 ceph-mon[32176]: pgmap v64: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 286 op/s 2024-03-23T15:45:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:23 smithi183 ceph-mon[36296]: pgmap v64: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 286 op/s 2024-03-23T15:45:25.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:24 smithi192 ceph-mon[36653]: pgmap v65: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 51 op/s 2024-03-23T15:45:25.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:24 smithi076 ceph-mon[32176]: pgmap v65: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 51 op/s 2024-03-23T15:45:25.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:24 smithi183 ceph-mon[36296]: pgmap v65: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 51 op/s 2024-03-23T15:45:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 -- 172.21.15.76:0/1711448242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 msgr2=0x7f7594100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 --2- 172.21.15.76:0/1711448242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594100cc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f757c009900 tx=0x7f757c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:45:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 -- 172.21.15.76:0/1711448242 shutdown_connections 2024-03-23T15:45:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 --2- 172.21.15.76:0/1711448242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7594101200 0x7f759410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 --2- 172.21.15.76:0/1711448242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594100cc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 --2- 172.21.15.76:0/1711448242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75941066a0 0x7f7594106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 -- 172.21.15.76:0/1711448242 >> 172.21.15.76:0/1711448242 conn(0x7f7594077470 msgr2=0x7f7594077880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 -- 172.21.15.76:0/1711448242 shutdown_connections 2024-03-23T15:45:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.995+0000 7f7598fdd640 1 -- 172.21.15.76:0/1711448242 wait complete. 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 Processor -- start 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 -- start start 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7594101200 0x7f7594199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75941066a0 0x7f75941a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7594199870 con 0x7f7594100860 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75941a4090 con 0x7f75941066a0 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.996+0000 7f7598fdd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f75941a41d0 con 0x7f7594101200 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37320/0 (socket says 172.21.15.76:37320) 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 -- 172.21.15.76:0/2254250602 learned_addr learned my addr 172.21.15.76:0/2254250602 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f758bfff640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7594101200 0x7f7594199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f7592d9d640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75941066a0 0x7f75941a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 -- 172.21.15.76:0/2254250602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7594101200 msgr2=0x7f7594199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7594101200 0x7f7594199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 -- 172.21.15.76:0/2254250602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75941066a0 msgr2=0x7f75941a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75941066a0 0x7f75941a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 -- 172.21.15.76:0/2254250602 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c009560 con 0x7f7594100860 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f759259c640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594198ad0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9f0 tx=0x7f758400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f758b7fe640 1 -- 172.21.15.76:0/2254250602 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584009720 con 0x7f7594100860 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75941a4430 con 0x7f7594100860 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f758b7fe640 1 -- 172.21.15.76:0/2254250602 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758400cd80 con 0x7f7594100860 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f758b7fe640 1 -- 172.21.15.76:0/2254250602 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584010690 con 0x7f7594100860 2024-03-23T15:45:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:26.997+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a4910 con 0x7f7594100860 2024-03-23T15:45:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.000+0000 7f758b7fe640 1 -- 172.21.15.76:0/2254250602 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f75840046c0 con 0x7f7594100860 2024-03-23T15:45:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.000+0000 7f759bbb7740 1 Processor -- start 2024-03-23T15:45:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.000+0000 7f759bbb7740 1 -- start start 2024-03-23T15:45:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.000+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039b5e0 0x55a67039b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039bf00 0x55a6703a4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6703a4cd0 0x55a6703a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759bbb7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6703561b0 con 0x55a67039bf00 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759bbb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a670308f80 con 0x55a6703a4cd0 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759bbb7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a6703064b0 con 0x55a67039b5e0 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f758b7fe640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f75800a22d0 0x7f75800a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039bf00 0x55a6703a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f7592d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039b5e0 0x55a67039b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039bf00 0x55a6703a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37326/0 (socket says 172.21.15.76:37326) 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 -- 172.21.15.76:0/206919312 learned_addr learned my addr 172.21.15.76:0/206919312 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f7592d9d640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f75800a22d0 0x7f75800a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f758b7fe640 1 -- 172.21.15.76:0/2254250602 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7584014070 con 0x7f7594100860 2024-03-23T15:45:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f758bfff640 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6703a4cd0 0x55a6703a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 -- 172.21.15.76:0/206919312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039b5e0 msgr2=0x55a67039b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039b5e0 0x55a67039b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 -- 172.21.15.76:0/206919312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6703a4cd0 msgr2=0x55a6703a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6703a4cd0 0x55a6703a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 -- 172.21.15.76:0/206919312 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a670297810 con 0x55a67039bf00 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f7592d9d640 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039b5e0 0x55a67039b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f759259c640 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039bf00 0x55a6703a4790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f758409aaa0 tx=0x7f758406ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4944d6aee7c2ed57 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f7592d9d640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f75800a22d0 0x7f75800a4790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f758c009fd0 tx=0x7f758c009500 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f7588ff9640 1 -- 172.21.15.76:0/206919312 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758409a700 con 0x55a67039bf00 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f7588ff9640 1 -- 172.21.15.76:0/206919312 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758409fe10 con 0x55a67039bf00 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.001+0000 7f7588ff9640 1 -- 172.21.15.76:0/206919312 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758409b240 con 0x55a67039bf00 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 -- 172.21.15.76:0/206919312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039bf00 msgr2=0x55a6703a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039bf00 0x55a6703a4790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f758409aaa0 tx=0x7f758406ded0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 -- 172.21.15.76:0/206919312 shutdown_connections 2024-03-23T15:45:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6703a4cd0 0x55a6703a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039bf00 0x55a6703a4790 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 --2- 172.21.15.76:0/206919312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039b5e0 0x55a67039b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 -- 172.21.15.76:0/206919312 >> 172.21.15.76:0/206919312 conn(0x55a670395f00 msgr2=0x55a6703962e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 -- 172.21.15.76:0/206919312 shutdown_connections 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.002+0000 7f759bbb7740 1 -- 172.21.15.76:0/206919312 wait complete. 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 -- start start 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 Processor -- start 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a6703a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a6703a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a670301c90 con 0x55a6703a4cd0 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6702a6c50 con 0x55a67039b5e0 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a66ffd5250 con 0x55a67039bf00 2024-03-23T15:45:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759bbb7740 10 client.0 _ll_register_callbacks cb 0x55a6702f5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a6703a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a6703a2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37328/0 (socket says 172.21.15.76:37328) 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f758bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a6703a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 -- 172.21.15.76:0/1637940313 learned_addr learned my addr 172.21.15.76:0/1637940313 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 -- 172.21.15.76:0/1637940313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 msgr2=0x55a6703a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a6703a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 -- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 msgr2=0x55a6703a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 -- 172.21.15.76:0/1637940313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a4910 con 0x55a6703a4cd0 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f759259c640 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7592d9d640 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a6703a2810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f758c0115c0 tx=0x7f758c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da9b1bd8923e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.003+0000 7f7579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f7579ffb640 1 -- 172.21.15.76:0/1637940313 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c020db0 con 0x55a6703a4cd0 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f7579ffb640 1 -- 172.21.15.76:0/1637940313 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f758c01e410 con 0x55a6703a4cd0 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a670326790 con 0x55a6703a4cd0 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f7579ffb640 1 -- 172.21.15.76:0/1637940313 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f758c01e5d0 con 0x55a6703a4cd0 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 10 client.16002 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a670300310 con 0x55a6703a4cd0 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f7579ffb640 1 -- 172.21.15.76:0/1637940313 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a670326790 con 0x55a6703a4cd0 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f7579ffb640 1 -- 172.21.15.76:0/1637940313 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f758c02a780 con 0x55a6703a4cd0 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 10 client.16002 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 20 client.16002 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a6703b4d10 0x55a6703b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 4 client.16002 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a670001600 con 0x55a6703b4d10 2024-03-23T15:45:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.004+0000 7f759259c640 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a6703b4d10 0x55a6703b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.005+0000 7f7579ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.005+0000 7f759259c640 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a6703b4d10 0x55a6703b7150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cb6fadf4f7b8f5 server_cookie=5c0e0c1a4c42742f in_seq=0 out_seq=0 2024-03-23T15:45:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f7579ffb640 1 -- 172.21.15.76:0/1637940313 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a670001600 con 0x55a6703b4d10 2024-03-23T15:45:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f7579ffb640 10 client.16002 handle_command_reply: tid=0 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 client.16002 shutdown 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 msgr2=0x55a6703a2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a6703a2810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f758c0115c0 tx=0x7f758c004ab0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 shutdown_connections 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a6703a2810 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a6703a22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a6703b4d10 0x55a6703b7150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 --2- 172.21.15.76:0/1637940313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f7579ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 >> 172.21.15.76:0/1637940313 conn(0x55a670395f00 msgr2=0x55a670397c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 shutdown_connections 2024-03-23T15:45:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 1 -- 172.21.15.76:0/1637940313 wait complete. 2024-03-23T15:45:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.072+0000 7f759bbb7740 20 client.16002 trim_cache size 0 max 16384 2024-03-23T15:45:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 Processor -- start 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 -- start start 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039b5e0 0x55a6703b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039bf00 0x55a6703a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6703a4cd0 0x55a670394d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a670287660 con 0x55a67039b5e0 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a670001630 con 0x55a67039bf00 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759bbb7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a6703943f0 con 0x55a6703a4cd0 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.086+0000 7f759259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039b5e0 0x55a6703b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039b5e0 0x55a6703b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37342/0 (socket says 172.21.15.76:37342) 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 -- 172.21.15.76:0/3546271424 learned_addr learned my addr 172.21.15.76:0/3546271424 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f7592d9d640 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6703a4cd0 0x55a670394d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f758bfff640 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039bf00 0x55a6703a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 -- 172.21.15.76:0/3546271424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6703a4cd0 msgr2=0x55a670394d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6703a4cd0 0x55a670394d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 -- 172.21.15.76:0/3546271424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039bf00 msgr2=0x55a6703a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039bf00 0x55a6703a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 -- 172.21.15.76:0/3546271424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67029c810 con 0x55a67039b5e0 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f758bfff640 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039bf00 0x55a6703a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f759259c640 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039b5e0 0x55a6703b03a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f758409ad30 tx=0x7f7584005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf760adebfcc3b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f757bfff640 1 -- 172.21.15.76:0/3546271424 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758409b520 con 0x55a67039b5e0 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f757bfff640 1 -- 172.21.15.76:0/3546271424 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758409b070 con 0x55a67039b5e0 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.087+0000 7f757bfff640 1 -- 172.21.15.76:0/3546271424 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758409b240 con 0x55a67039b5e0 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- 172.21.15.76:0/3546271424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039b5e0 msgr2=0x55a6703b03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039b5e0 0x55a6703b03a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f758409ad30 tx=0x7f7584005f10 comp rx=0 tx=0).stop 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- 172.21.15.76:0/3546271424 shutdown_connections 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6703a4cd0 0x55a670394d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039bf00 0x55a6703a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3546271424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a67039b5e0 0x55a6703b03a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- 172.21.15.76:0/3546271424 >> 172.21.15.76:0/3546271424 conn(0x55a670395f00 msgr2=0x55a670394610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- 172.21.15.76:0/3546271424 shutdown_connections 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- 172.21.15.76:0/3546271424 wait complete. 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- start start 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 Processor -- start 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a67039e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a67039e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a6703ad0b0 con 0x55a6703a4cd0 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6703ad220 con 0x55a67039b5e0 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a67039ed40 con 0x55a67039bf00 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.088+0000 7f759bbb7740 10 client.0 _ll_register_callbacks cb 0x7f758c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a67039e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a67039e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37348/0 (socket says 172.21.15.76:37348) 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 -- 172.21.15.76:0/3555681719 learned_addr learned my addr 172.21.15.76:0/3555681719 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f759259c640 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f758bfff640 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a67039e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 -- 172.21.15.76:0/3555681719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 msgr2=0x55a67039e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a67039e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 -- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 msgr2=0x55a6703ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 -- 172.21.15.76:0/3555681719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a670297810 con 0x55a6703a4cd0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7592d9d640 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a67039e780 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f758c02ab30 tx=0x7f758c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5548001c7ee0c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7579ffb640 1 -- 172.21.15.76:0/3555681719 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c0207f0 con 0x55a6703a4cd0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7579ffb640 1 -- 172.21.15.76:0/3555681719 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f758c003b80 con 0x55a6703a4cd0 2024-03-23T15:45:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6700521b0 con 0x55a6703a4cd0 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7579ffb640 1 -- 172.21.15.76:0/3555681719 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f758c003d40 con 0x55a6703a4cd0 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f759bbb7740 10 client.16008 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a6702c25a0 con 0x55a6703a4cd0 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7579ffb640 1 -- 172.21.15.76:0/3555681719 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c048080 con 0x55a6703a4cd0 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.089+0000 7f7579ffb640 1 -- 172.21.15.76:0/3555681719 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f758c004500 con 0x55a6703a4cd0 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759bbb7740 10 client.16008 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759bbb7740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759bbb7740 20 client.16008 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a67041b3a0 0x55a67041d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759bbb7740 4 client.16008 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a67003e550 con 0x55a67041b3a0 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759259c640 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a67041b3a0 0x55a67041d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f7579ffb640 10 client.16008 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.090+0000 7f759259c640 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a67041b3a0 0x55a67041d7c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee1977efdbf8ee1 server_cookie=15f98ba20c969b05 in_seq=0 out_seq=0 2024-03-23T15:45:27.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:26 smithi192 ceph-mon[36653]: pgmap v66: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 73 op/s 2024-03-23T15:45:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:45:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:26 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:26 smithi076 ceph-mon[32176]: pgmap v66: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 73 op/s 2024-03-23T15:45:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:45:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:26 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:26 smithi183 ceph-mon[36296]: pgmap v66: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 73 op/s 2024-03-23T15:45:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:45:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:26 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f7579ffb640 1 -- 172.21.15.76:0/3555681719 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a67003e550 con 0x55a67041b3a0 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f7579ffb640 10 client.16008 handle_command_reply: tid=0 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 client.16008 shutdown 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 msgr2=0x55a67039e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a67039e780 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f758c02ab30 tx=0x7f758c009b00 comp rx=0 tx=0).stop 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 shutdown_connections 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a6703a4cd0 0x55a67039e780 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a67039bf00 0x55a67039e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a67041b3a0 0x55a67041d7c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 --2- 172.21.15.76:0/3555681719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a67039b5e0 0x55a6703ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f7579ffb640 0 client.16008 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 >> 172.21.15.76:0/3555681719 conn(0x55a670395f00 msgr2=0x55a670394ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 shutdown_connections 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 1 -- 172.21.15.76:0/3555681719 wait complete. 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.405+0000 7f759bbb7740 20 client.16008 trim_cache size 0 max 16384 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:45:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:45:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:45:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f75800a22d0 msgr2=0x7f75800a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f75800a22d0 0x7f75800a4790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f758c009fd0 tx=0x7f758c009500 comp rx=0 tx=0).stop 2024-03-23T15:45:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 msgr2=0x7f7594198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594198ad0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f758400e9f0 tx=0x7f758400edf0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 shutdown_connections 2024-03-23T15:45:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f75800a22d0 0x7f75800a4790 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75941066a0 0x7f75941a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7594101200 0x7f7594199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 --2- 172.21.15.76:0/2254250602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7594100860 0x7f7594198ad0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 >> 172.21.15.76:0/2254250602 conn(0x7f7594077470 msgr2=0x7f75940ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 shutdown_connections 2024-03-23T15:45:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:27.406+0000 7f7598fdd640 1 -- 172.21.15.76:0/2254250602 wait complete. 2024-03-23T15:45:28.284 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:27 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:45:28.284 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:27 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:45:28.285 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:27 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:28.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:27 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:45:28.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:27 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:45:28.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:27 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:28.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:27 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:45:28.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:27 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:45:28.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:27 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:45:29.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:29 smithi192 ceph-mon[36653]: pgmap v67: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 82 op/s 2024-03-23T15:45:29.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:29 smithi076 ceph-mon[32176]: pgmap v67: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 82 op/s 2024-03-23T15:45:29.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:29 smithi183 ceph-mon[36296]: pgmap v67: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 82 op/s 2024-03-23T15:45:31.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:31 smithi192 ceph-mon[36653]: pgmap v68: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-03-23T15:45:31.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:31 smithi076 ceph-mon[32176]: pgmap v68: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-03-23T15:45:31.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:31 smithi183 ceph-mon[36296]: pgmap v68: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 77 op/s 2024-03-23T15:45:33.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:33 smithi192 ceph-mon[36653]: pgmap v69: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-03-23T15:45:33.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:33 smithi076 ceph-mon[32176]: pgmap v69: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-03-23T15:45:33.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:33 smithi183 ceph-mon[36296]: pgmap v69: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-03-23T15:45:35.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:35 smithi192 ceph-mon[36653]: pgmap v70: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 458 KiB/s wr, 137 op/s 2024-03-23T15:45:35.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:35 smithi076 ceph-mon[32176]: pgmap v70: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 458 KiB/s wr, 137 op/s 2024-03-23T15:45:35.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:35 smithi183 ceph-mon[36296]: pgmap v70: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 458 KiB/s wr, 137 op/s 2024-03-23T15:45:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:37 smithi192 ceph-mon[36653]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 679 KiB/s wr, 166 op/s 2024-03-23T15:45:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:37 smithi076 ceph-mon[32176]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 679 KiB/s wr, 166 op/s 2024-03-23T15:45:37.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:37 smithi183 ceph-mon[36296]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 679 KiB/s wr, 166 op/s 2024-03-23T15:45:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:39 smithi192 ceph-mon[36653]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 520 KiB/s wr, 163 op/s 2024-03-23T15:45:39.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:39 smithi076 ceph-mon[32176]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 520 KiB/s wr, 163 op/s 2024-03-23T15:45:39.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:39 smithi183 ceph-mon[36296]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 520 KiB/s wr, 163 op/s 2024-03-23T15:45:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 -- 172.21.15.76:0/2941003852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc100860 msgr2=0x7f21bc100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 --2- 172.21.15.76:0/2941003852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc100860 0x7f21bc100ce0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009900 tx=0x7f21ac02bc30 comp rx=0 tx=0).stop 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 -- 172.21.15.76:0/2941003852 shutdown_connections 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 --2- 172.21.15.76:0/2941003852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21bc101220 0x7f21bc10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 --2- 172.21.15.76:0/2941003852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc100860 0x7f21bc100ce0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 --2- 172.21.15.76:0/2941003852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc1055b0 0x7f21bc105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 -- 172.21.15.76:0/2941003852 >> 172.21.15.76:0/2941003852 conn(0x7f21bc077460 msgr2=0x7f21bc077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 -- 172.21.15.76:0/2941003852 shutdown_connections 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 -- 172.21.15.76:0/2941003852 wait complete. 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 Processor -- start 2024-03-23T15:45:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.509+0000 7f21c4615640 1 -- start start 2024-03-23T15:45:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c4615640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc100860 0x7f21bc197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c4615640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21bc101220 0x7f21bc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c4615640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc1055b0 0x7f21bc19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c4615640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc19cbf0 con 0x7f21bc1055b0 2024-03-23T15:45:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c4615640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc19cd60 con 0x7f21bc101220 2024-03-23T15:45:40.514 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c4615640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc19cf20 con 0x7f21bc100860 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc1055b0 0x7f21bc19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c23b1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc100860 0x7f21bc197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc1055b0 0x7f21bc19c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53884/0 (socket says 172.21.15.76:53884) 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c1bb0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21bc101220 0x7f21bc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 -- 172.21.15.76:0/1933761472 learned_addr learned my addr 172.21.15.76:0/1933761472 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 -- 172.21.15.76:0/1933761472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc100860 msgr2=0x7f21bc197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc100860 0x7f21bc197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 -- 172.21.15.76:0/1933761472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21bc101220 msgr2=0x7f21bc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21bc101220 0x7f21bc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 -- 172.21.15.76:0/1933761472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ac009560 con 0x7f21bc1055b0 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21bc101220 0x7f21bc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.510+0000 7f21c2bb2640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc1055b0 0x7f21bc19c5f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f21b400ea60 tx=0x7f21b400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.511+0000 7f21b37fe640 1 -- 172.21.15.76:0/1933761472 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b400ce40 con 0x7f21bc1055b0 2024-03-23T15:45:40.515 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.511+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc19d200 con 0x7f21bc1055b0 2024-03-23T15:45:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.511+0000 7f21b37fe640 1 -- 172.21.15.76:0/1933761472 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21b4005020 con 0x7f21bc1055b0 2024-03-23T15:45:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.511+0000 7f21b37fe640 1 -- 172.21.15.76:0/1933761472 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4010690 con 0x7f21bc1055b0 2024-03-23T15:45:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.511+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1a4a80 con 0x7f21bc1055b0 2024-03-23T15:45:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.513+0000 7f21b37fe640 1 -- 172.21.15.76:0/1933761472 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f21b4005500 con 0x7f21bc1055b0 2024-03-23T15:45:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.513+0000 7f21c71f7740 1 Processor -- start 2024-03-23T15:45:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c71f7740 1 -- start start 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca98f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca998790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca998cd0 0x564bca99b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c71f7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bca94a1b0 con 0x564bca98ff00 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c71f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bca8fcf80 con 0x564bca998cd0 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c71f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564bca8fa4b0 con 0x564bca98f5e0 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21b37fe640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f21a80a22d0 0x7f21a80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c23b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca998790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c2bb2640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f21a80a22d0 0x7f21a80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c1bb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca98f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c23b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca998790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53888/0 (socket says 172.21.15.76:53888) 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c23b1640 1 -- 172.21.15.76:0/3827344193 learned_addr learned my addr 172.21.15.76:0/3827344193 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca998cd0 0x564bca99b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.514+0000 7f21b37fe640 1 -- 172.21.15.76:0/1933761472 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f21b4014070 con 0x7f21bc1055b0 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c1bb0640 1 -- 172.21.15.76:0/3827344193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca998cd0 msgr2=0x564bca99b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca998cd0 0x564bca99b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c1bb0640 1 -- 172.21.15.76:0/3827344193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 msgr2=0x564bca998790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca998790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c1bb0640 1 -- 172.21.15.76:0/3827344193 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bca88b810 con 0x564bca98f5e0 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c23b1640 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca998790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c2bb2640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f21a80a22d0 0x7f21a80a4790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f21b409af10 tx=0x7f21b406cf40 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca98f9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ec6ffb09a8759 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21b0bfe640 1 -- 172.21.15.76:0/3827344193 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac004030 con 0x564bca98f5e0 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21b0bfe640 1 -- 172.21.15.76:0/3827344193 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ac0035f0 con 0x564bca98f5e0 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.515+0000 7f21b0bfe640 1 -- 172.21.15.76:0/3827344193 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac005050 con 0x564bca98f5e0 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 -- 172.21.15.76:0/3827344193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 msgr2=0x564bca98f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca98f9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac005b70 comp rx=0 tx=0).stop 2024-03-23T15:45:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 -- 172.21.15.76:0/3827344193 shutdown_connections 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca998cd0 0x564bca99b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca998790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3827344193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca98f9c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 -- 172.21.15.76:0/3827344193 >> 172.21.15.76:0/3827344193 conn(0x564bca989f00 msgr2=0x564bca98a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 -- 172.21.15.76:0/3827344193 shutdown_connections 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 -- 172.21.15.76:0/3827344193 wait complete. 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 -- start start 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.516+0000 7f21c71f7740 1 Processor -- start 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca995da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca9962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca998cd0 0x564bca996820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bca8f5c90 con 0x564bca998cd0 2024-03-23T15:45:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bca89ac50 con 0x564bca98ff00 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564bca5c9250 con 0x564bca98f5e0 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 10 client.0 _ll_register_callbacks cb 0x564bca8e9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca998cd0 0x564bca996820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c23b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca9962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca998cd0 0x564bca996820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53892/0 (socket says 172.21.15.76:53892) 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 -- 172.21.15.76:0/3895142740 learned_addr learned my addr 172.21.15.76:0/3895142740 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca995da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 -- 172.21.15.76:0/3895142740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 msgr2=0x564bca995da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca995da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 -- 172.21.15.76:0/3895142740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 msgr2=0x564bca9962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca9962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 -- 172.21.15.76:0/3895142740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc068d90 con 0x564bca998cd0 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca998cd0 0x564bca996820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f21ac02c170 tx=0x7f21ac003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740a7eb9e9e38f0a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21a5ffb640 1 -- 172.21.15.76:0/3895142740 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac003b70 con 0x564bca998cd0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564bca91a790 con 0x564bca998cd0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.517+0000 7f21a5ffb640 1 -- 172.21.15.76:0/3895142740 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f21ac003cd0 con 0x564bca998cd0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21a5ffb640 1 -- 172.21.15.76:0/3895142740 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21ac0044e0 con 0x564bca998cd0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 10 client.16020 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bca8f4310 con 0x564bca998cd0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21a5ffb640 1 -- 172.21.15.76:0/3895142740 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564bca91a790 con 0x564bca998cd0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21a5ffb640 1 -- 172.21.15.76:0/3895142740 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f21ac041500 con 0x564bca998cd0 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 10 client.16020 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 20 client.16020 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bca9a88c0 0x564bca9aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 4 client.16020 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bca5f5600 con 0x564bca9a88c0 2024-03-23T15:45:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.518+0000 7f21c23b1640 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bca9a88c0 0x564bca9aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.519+0000 7f21a5ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.519+0000 7f21c23b1640 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bca9a88c0 0x564bca9aace0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6ae73b693d0ad3 server_cookie=9c1cbbe5d4776e29 in_seq=0 out_seq=0 2024-03-23T15:45:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.532+0000 7f21a5ffb640 1 -- 172.21.15.76:0/3895142740 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564bca5f5600 con 0x564bca9a88c0 2024-03-23T15:45:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.532+0000 7f21a5ffb640 10 client.16020 handle_command_reply: tid=0 2024-03-23T15:45:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.532+0000 7f21c71f7740 1 client.16020 shutdown 2024-03-23T15:45:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca998cd0 msgr2=0x564bca996820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca998cd0 0x564bca996820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f21ac02c170 tx=0x7f21ac003f30 comp rx=0 tx=0).stop 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 shutdown_connections 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca998cd0 0x564bca996820 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bca9a88c0 0x564bca9aace0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca9962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 --2- 172.21.15.76:0/3895142740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca98f5e0 0x564bca995da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21a5ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 >> 172.21.15.76:0/3895142740 conn(0x564bca989f00 msgr2=0x564bca98bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 shutdown_connections 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 1 -- 172.21.15.76:0/3895142740 wait complete. 2024-03-23T15:45:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.533+0000 7f21c71f7740 20 client.16020 trim_cache size 0 max 16384 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 Processor -- start 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 -- start start 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98f5e0 0x564bca9a4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca996640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca988d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bca87b660 con 0x564bca98f5e0 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bca98f490 con 0x564bca98ff00 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c71f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564bca98aad0 con 0x564bca998cd0 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98f5e0 0x564bca9a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98f5e0 0x564bca9a4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53904/0 (socket says 172.21.15.76:53904) 2024-03-23T15:45:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 -- 172.21.15.76:0/1640907901 learned_addr learned my addr 172.21.15.76:0/1640907901 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c23b1640 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca996640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca988d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 -- 172.21.15.76:0/1640907901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 msgr2=0x564bca988d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca988d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 -- 172.21.15.76:0/1640907901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 msgr2=0x564bca996640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.549+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca996640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.550+0000 7f21c1bb0640 1 -- 172.21.15.76:0/1640907901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bca890810 con 0x564bca98f5e0 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.550+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98f5e0 0x564bca9a4340 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0418b0 tx=0x7f21ac02c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8143907ff07b537d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.550+0000 7f21a7fff640 1 -- 172.21.15.76:0/1640907901 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac004120 con 0x564bca98f5e0 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.550+0000 7f21a7fff640 1 -- 172.21.15.76:0/1640907901 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ac056030 con 0x564bca98f5e0 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.550+0000 7f21a7fff640 1 -- 172.21.15.76:0/1640907901 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0416f0 con 0x564bca98f5e0 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- 172.21.15.76:0/1640907901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98f5e0 msgr2=0x564bca9a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98f5e0 0x564bca9a4340 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0418b0 tx=0x7f21ac02c5f0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- 172.21.15.76:0/1640907901 shutdown_connections 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca988d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98ff00 0x564bca996640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1640907901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98f5e0 0x564bca9a4340 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- 172.21.15.76:0/1640907901 >> 172.21.15.76:0/1640907901 conn(0x564bca989f00 msgr2=0x564bca988610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- 172.21.15.76:0/1640907901 shutdown_connections 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- 172.21.15.76:0/1640907901 wait complete. 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- start start 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 Processor -- start 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98f5e0 0x564bca9a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca9921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca992710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564bca9a1050 con 0x564bca98ff00 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bca9a11c0 con 0x564bca98f5e0 2024-03-23T15:45:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564bca992c50 con 0x564bca998cd0 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c71f7740 10 client.0 _ll_register_callbacks cb 0x7f21b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c23b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca9921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c23b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca9921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53920/0 (socket says 172.21.15.76:53920) 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.551+0000 7f21c23b1640 1 -- 172.21.15.76:0/1287921292 learned_addr learned my addr 172.21.15.76:0/1287921292 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98f5e0 0x564bca9a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca992710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c23b1640 1 -- 172.21.15.76:0/1287921292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 msgr2=0x564bca992710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c23b1640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca992710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c23b1640 1 -- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98f5e0 msgr2=0x564bca9a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c23b1640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98f5e0 0x564bca9a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c23b1640 1 -- 172.21.15.76:0/1287921292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bca8f4310 con 0x564bca98ff00 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca992710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:45:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c23b1640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca9921d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f21b8004700 tx=0x7f21b80205a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118e8251ffe1f55d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21a5ffb640 1 -- 172.21.15.76:0/1287921292 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8017030 con 0x564bca98ff00 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21a5ffb640 1 -- 172.21.15.76:0/1287921292 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f21b8010cf0 con 0x564bca98ff00 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ac0035f0 con 0x564bca98ff00 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21a5ffb640 1 -- 172.21.15.76:0/1287921292 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21b8004030 con 0x564bca98ff00 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c71f7740 10 client.16026 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bca8b65a0 con 0x564bca98ff00 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.552+0000 7f21a5ffb640 1 -- 172.21.15.76:0/1287921292 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ac0035f0 con 0x564bca98ff00 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21a5ffb640 1 -- 172.21.15.76:0/1287921292 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f21b8028410 con 0x564bca98ff00 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c71f7740 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c71f7740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c71f7740 20 client.16026 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bcaa13370 0x564bcaa15790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c71f7740 4 client.16026 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:45:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bca6461b0 con 0x564bcaa13370 2024-03-23T15:45:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bcaa13370 0x564bcaa15790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21a5ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.553+0000 7f21c1bb0640 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bcaa13370 0x564bcaa15790 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46aeac9250cde74d server_cookie=21c8bdb5fd1b9e90 in_seq=0 out_seq=0 2024-03-23T15:45:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.948+0000 7f21a5ffb640 1 -- 172.21.15.76:0/1287921292 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bca6461b0 con 0x564bcaa13370 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.948+0000 7f21a5ffb640 10 client.16026 handle_command_reply: tid=0 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 client.16026 shutdown 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 msgr2=0x564bca9921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca9921d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f21b8004700 tx=0x7f21b80205a0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 shutdown_connections 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564bca998cd0 0x564bca992710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564bca98ff00 0x564bca9921d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564bcaa13370 0x564bcaa15790 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 --2- 172.21.15.76:0/1287921292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bca98f5e0 0x564bca9a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21a5ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 >> 172.21.15.76:0/1287921292 conn(0x564bca989f00 msgr2=0x564bca988ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 shutdown_connections 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 1 -- 172.21.15.76:0/1287921292 wait complete. 2024-03-23T15:45:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c71f7740 20 client.16026 trim_cache size 0 max 16384 2024-03-23T15:45:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:45:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:45:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:45:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f21a80a22d0 msgr2=0x7f21a80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c4615640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f21a80a22d0 0x7f21a80a4790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f21b409af10 tx=0x7f21b406cf40 comp rx=0 tx=0).stop 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc1055b0 msgr2=0x7f21bc19c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.949+0000 7f21c4615640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc1055b0 0x7f21bc19c5f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f21b400ea60 tx=0x7f21b400ef20 comp rx=0 tx=0).stop 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 shutdown_connections 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f21a80a22d0 0x7f21a80a4790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21bc1055b0 0x7f21bc19c5f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21bc101220 0x7f21bc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 --2- 172.21.15.76:0/1933761472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21bc100860 0x7f21bc197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 >> 172.21.15.76:0/1933761472 conn(0x7f21bc077460 msgr2=0x7f21bc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 shutdown_connections 2024-03-23T15:45:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:40.950+0000 7f21c4615640 1 -- 172.21.15.76:0/1933761472 wait complete. 2024-03-23T15:45:41.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:41 smithi192 ceph-mon[36653]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 513 KiB/s wr, 149 op/s 2024-03-23T15:45:41.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:41 smithi076 ceph-mon[32176]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 513 KiB/s wr, 149 op/s 2024-03-23T15:45:41.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:41 smithi183 ceph-mon[36296]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 513 KiB/s wr, 149 op/s 2024-03-23T15:45:43.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:43 smithi192 ceph-mon[36653]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 810 KiB/s wr, 230 op/s 2024-03-23T15:45:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:43 smithi076 ceph-mon[32176]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 810 KiB/s wr, 230 op/s 2024-03-23T15:45:43.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:43 smithi183 ceph-mon[36296]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 810 KiB/s wr, 230 op/s 2024-03-23T15:45:45.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:45 smithi192 ceph-mon[36653]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 535 KiB/s wr, 129 op/s 2024-03-23T15:45:45.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:45 smithi076 ceph-mon[32176]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 535 KiB/s wr, 129 op/s 2024-03-23T15:45:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:45 smithi183 ceph-mon[36296]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 535 KiB/s wr, 129 op/s 2024-03-23T15:45:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:47 smithi192 ceph-mon[36653]: pgmap v76: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 719 KiB/s wr, 169 op/s 2024-03-23T15:45:47.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:47 smithi076 ceph-mon[32176]: pgmap v76: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 719 KiB/s wr, 169 op/s 2024-03-23T15:45:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:47 smithi183 ceph-mon[36296]: pgmap v76: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 719 KiB/s wr, 169 op/s 2024-03-23T15:45:49.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:49 smithi192 ceph-mon[36653]: pgmap v77: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 500 KiB/s wr, 146 op/s 2024-03-23T15:45:49.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:49 smithi076 ceph-mon[32176]: pgmap v77: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 500 KiB/s wr, 146 op/s 2024-03-23T15:45:49.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:49 smithi183 ceph-mon[36296]: pgmap v77: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 500 KiB/s wr, 146 op/s 2024-03-23T15:45:51.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:51 smithi192 ceph-mon[36653]: pgmap v78: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 483 KiB/s wr, 127 op/s 2024-03-23T15:45:51.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:51 smithi076 ceph-mon[32176]: pgmap v78: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 483 KiB/s wr, 127 op/s 2024-03-23T15:45:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:51 smithi183 ceph-mon[36296]: pgmap v78: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 483 KiB/s wr, 127 op/s 2024-03-23T15:45:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:53 smithi192 ceph-mon[36653]: pgmap v79: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 653 KiB/s wr, 159 op/s 2024-03-23T15:45:53.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:53 smithi076 ceph-mon[32176]: pgmap v79: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 653 KiB/s wr, 159 op/s 2024-03-23T15:45:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:53 smithi183 ceph-mon[36296]: pgmap v79: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 653 KiB/s wr, 159 op/s 2024-03-23T15:45:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 -- 172.21.15.76:0/96986730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 msgr2=0x7f9e54107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/96986730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54107ab0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009930 tx=0x7f9e5002bc30 comp rx=0 tx=0).stop 2024-03-23T15:45:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 -- 172.21.15.76:0/96986730 shutdown_connections 2024-03-23T15:45:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/96986730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e54102080 0x7f9e5410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/96986730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e541016c0 0x7f9e54101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/96986730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54107ab0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 -- 172.21.15.76:0/96986730 >> 172.21.15.76:0/96986730 conn(0x7f9e540fd1d0 msgr2=0x7f9e540ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 -- 172.21.15.76:0/96986730 shutdown_connections 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.048+0000 7f9e5b95e640 1 -- 172.21.15.76:0/96986730 wait complete. 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 Processor -- start 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 -- start start 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e541016c0 0x7f9e5419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e54102080 0x7f9e5419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54198150 con 0x7f9e541076d0 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541982c0 con 0x7f9e54102080 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e5b95e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54198480 con 0x7f9e541016c0 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e59efb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e596fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e541016c0 0x7f9e5419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.059 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e59efb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43574/0 (socket says 172.21.15.76:43574) 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e59efb640 1 -- 172.21.15.76:0/1320031092 learned_addr learned my addr 172.21.15.76:0/1320031092 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.049+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e54102080 0x7f9e5419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e59efb640 1 -- 172.21.15.76:0/1320031092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e541016c0 msgr2=0x7f9e5419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e59efb640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e541016c0 0x7f9e5419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e59efb640 1 -- 172.21.15.76:0/1320031092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e54102080 msgr2=0x7f9e5419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e59efb640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e54102080 0x7f9e5419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e59efb640 1 -- 172.21.15.76:0/1320031092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c009840 con 0x7f9e541076d0 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e54102080 0x7f9e5419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e59efb640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54197b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00efd0 tx=0x7f9e4c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e4a7fc640 1 -- 172.21.15.76:0/1320031092 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c009b40 con 0x7f9e541076d0 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e4a7fc640 1 -- 172.21.15.76:0/1320031092 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c003b70 con 0x7f9e541076d0 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e50009560 con 0x7f9e541076d0 2024-03-23T15:45:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e4a7fc640 1 -- 172.21.15.76:0/1320031092 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0108f0 con 0x7f9e541076d0 2024-03-23T15:45:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.050+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e541a4810 con 0x7f9e541076d0 2024-03-23T15:45:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.052+0000 7f9e4a7fc640 1 -- 172.21.15.76:0/1320031092 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f9e4c003ce0 con 0x7f9e541076d0 2024-03-23T15:45:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 Processor -- start 2024-03-23T15:45:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 -- start start 2024-03-23T15:45:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b45e0 0x564e3d5b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 0x564e3d5c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e4a7fc640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9e400a22d0 0x7f9e400a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e3d56f1b0 con 0x564e3d5b4f00 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e3d521f80 con 0x564e3d5bdcd0 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564e3d51f4b0 con 0x564e3d5b45e0 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e596fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e596fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43582/0 (socket says 172.21.15.76:43582) 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e596fa640 1 -- 172.21.15.76:0/1581025730 learned_addr learned my addr 172.21.15.76:0/1581025730 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b45e0 0x564e3d5b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e4a7fc640 1 -- 172.21.15.76:0/1320031092 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9e4c014070 con 0x7f9e541076d0 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 0x564e3d5c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.053+0000 7f9e59efb640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9e400a22d0 0x7f9e400a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e596fa640 1 -- 172.21.15.76:0/1581025730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b45e0 msgr2=0x564e3d5b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e596fa640 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b45e0 0x564e3d5b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e596fa640 1 -- 172.21.15.76:0/1581025730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 msgr2=0x564e3d5c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e596fa640 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 0x564e3d5c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e596fa640 1 -- 172.21.15.76:0/1581025730 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e3d4b0810 con 0x564e3d5b4f00 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e59efb640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9e400a22d0 0x7f9e400a4790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0cbd60 tx=0x7f9e4c06cf60 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e596fa640 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bd790 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9e50036040 tx=0x7f9e5002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b195fba91524fd33 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e3f7fe640 1 -- 172.21.15.76:0/1581025730 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50004800 con 0x564e3d5b4f00 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e3f7fe640 1 -- 172.21.15.76:0/1581025730 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e50004960 con 0x564e3d5b4f00 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.054+0000 7f9e3f7fe640 1 -- 172.21.15.76:0/1581025730 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50005030 con 0x564e3d5b4f00 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1581025730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 msgr2=0x564e3d5bd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bd790 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9e50036040 tx=0x7f9e5002c5d0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1581025730 shutdown_connections 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 0x564e3d5c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bd790 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1581025730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b45e0 0x564e3d5b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1581025730 >> 172.21.15.76:0/1581025730 conn(0x564e3d5aef00 msgr2=0x564e3d5af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1581025730 shutdown_connections 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1581025730 wait complete. 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 -- start start 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 Processor -- start 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.055+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5bdcd0 0x564e3d5bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e3d51ac90 con 0x564e3d5b4f00 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e3d4bfc50 con 0x564e3d5b45e0 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564e3d1ee250 con 0x564e3d5bdcd0 2024-03-23T15:45:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 10 client.0 _ll_register_callbacks cb 0x564e3d50e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43596/0 (socket says 172.21.15.76:43596) 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 -- 172.21.15.76:0/1249096066 learned_addr learned my addr 172.21.15.76:0/1249096066 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5bdcd0 0x564e3d5bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 -- 172.21.15.76:0/1249096066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5bdcd0 msgr2=0x564e3d5bb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5bdcd0 0x564e3d5bb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 -- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 msgr2=0x564e3d5bad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 -- 172.21.15.76:0/1249096066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5410ee90 con 0x564e3d5b4f00 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5bdcd0 0x564e3d5bb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e596fa640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bb2a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9e5002c170 tx=0x7f9e5002c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6e42c1580b108c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:45:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e3cff9640 1 -- 172.21.15.76:0/1249096066 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50005200 con 0x564e3d5b4f00 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e3cff9640 1 -- 172.21.15.76:0/1249096066 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9e5004d070 con 0x564e3d5b4f00 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e3d53f790 con 0x564e3d5b4f00 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e3cff9640 1 -- 172.21.15.76:0/1249096066 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e50004ad0 con 0x564e3d5b4f00 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 10 client.16038 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.056+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e3d519310 con 0x564e3d5b4f00 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e3cff9640 1 -- 172.21.15.76:0/1249096066 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564e3d53f790 con 0x564e3d5b4f00 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e3cff9640 1 -- 172.21.15.76:0/1249096066 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e5003c050 con 0x564e3d5b4f00 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e5e61e740 10 client.16038 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e5e61e740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e5e61e740 20 client.16038 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d5cd890 0x564e3d5cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e5e61e740 4 client.16038 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e3d21a600 con 0x564e3d5cd890 2024-03-23T15:45:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d5cd890 0x564e3d5cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e3cff9640 10 client.16038 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.057+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d5cd890 0x564e3d5cfcb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9d88efeea0282b server_cookie=59f7cd03cf8bb20a in_seq=0 out_seq=0 2024-03-23T15:45:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.114+0000 7f9e3cff9640 1 -- 172.21.15.76:0/1249096066 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564e3d21a600 con 0x564e3d5cd890 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.114+0000 7f9e3cff9640 10 client.16038 handle_command_reply: tid=0 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 client.16038 shutdown 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 msgr2=0x564e3d5bb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bb2a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9e5002c170 tx=0x7f9e5002c3b0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 shutdown_connections 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5bdcd0 0x564e3d5bb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b4f00 0x564e3d5bb2a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d5cd890 0x564e3d5cfcb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/1249096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e3cff9640 0 client.16038 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 >> 172.21.15.76:0/1249096066 conn(0x564e3d5aef00 msgr2=0x564e3d5b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 shutdown_connections 2024-03-23T15:45:54.119 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 1 -- 172.21.15.76:0/1249096066 wait complete. 2024-03-23T15:45:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.115+0000 7f9e5e61e740 20 client.16038 trim_cache size 0 max 16384 2024-03-23T15:45:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 Processor -- start 2024-03-23T15:45:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 -- start start 2024-03-23T15:45:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5bdcd0 0x564e3d5add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e3d4a0660 con 0x564e3d5bdcd0 2024-03-23T15:45:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e3d5a9160 con 0x564e3d5b45e0 2024-03-23T15:45:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.135+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564e3d5aa6c0 con 0x564e3d5b4f00 2024-03-23T15:45:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5bdcd0 0x564e3d5add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5bdcd0 0x564e3d5add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43604/0 (socket says 172.21.15.76:43604) 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 -- 172.21.15.76:0/195034360 learned_addr learned my addr 172.21.15.76:0/195034360 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 -- 172.21.15.76:0/195034360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 msgr2=0x564e3d5c9340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5c9340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 -- 172.21.15.76:0/195034360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 msgr2=0x564e3d5bd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 -- 172.21.15.76:0/195034360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e3d4b5810 con 0x564e3d5bdcd0 2024-03-23T15:45:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.136+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5bdcd0 0x564e3d5add20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9e44013010 tx=0x7f9e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d4151fcc214885 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.137+0000 7f9e3effd640 1 -- 172.21.15.76:0/195034360 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44011050 con 0x564e3d5bdcd0 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.137+0000 7f9e3effd640 1 -- 172.21.15.76:0/195034360 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e440111b0 con 0x564e3d5bdcd0 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.137+0000 7f9e3effd640 1 -- 172.21.15.76:0/195034360 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44004a00 con 0x564e3d5bdcd0 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.138+0000 7f9e5e61e740 1 -- 172.21.15.76:0/195034360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5bdcd0 msgr2=0x564e3d5add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.138+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5bdcd0 0x564e3d5add20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9e44013010 tx=0x7f9e440046c0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.138+0000 7f9e5e61e740 1 -- 172.21.15.76:0/195034360 shutdown_connections 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.138+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5bdcd0 0x564e3d5add20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.138+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.138+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/195034360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5b45e0 0x564e3d5bd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.138+0000 7f9e5e61e740 1 -- 172.21.15.76:0/195034360 >> 172.21.15.76:0/195034360 conn(0x564e3d5aef00 msgr2=0x564e3d5ad610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.139+0000 7f9e5e61e740 1 -- 172.21.15.76:0/195034360 shutdown_connections 2024-03-23T15:45:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.139+0000 7f9e5e61e740 1 -- 172.21.15.76:0/195034360 wait complete. 2024-03-23T15:45:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.139+0000 7f9e5e61e740 1 -- start start 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.139+0000 7f9e5e61e740 1 Processor -- start 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b45e0 0x564e3d5b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e5e61e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 0x564e3d5c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564e3d5b8540 con 0x564e3d5b45e0 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e3d5c4ea0 con 0x564e3d5bdcd0 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e5e61e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564e3d5c5060 con 0x564e3d5b4f00 2024-03-23T15:45:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e5e61e740 10 client.0 _ll_register_callbacks cb 0x7f9e44011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e58ef9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b45e0 0x564e3d5b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e58ef9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b45e0 0x564e3d5b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43616/0 (socket says 172.21.15.76:43616) 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e58ef9640 1 -- 172.21.15.76:0/3969468026 learned_addr learned my addr 172.21.15.76:0/3969468026 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.140+0000 7f9e596fa640 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e596fa640 1 -- 172.21.15.76:0/3969468026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 msgr2=0x564e3d5c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e596fa640 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 0x564e3d5c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e596fa640 1 -- 172.21.15.76:0/3969468026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b45e0 msgr2=0x564e3d5b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e596fa640 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b45e0 0x564e3d5b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e596fa640 1 -- 172.21.15.76:0/3969468026 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e44019040 con 0x564e3d5b4f00 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e596fa640 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5b7e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x564e3d5bbe20 tx=0x7f9e500048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bd2cf352af906a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-23T15:45:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e3cff9640 1 -- 172.21.15.76:0/3969468026 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50004450 con 0x564e3d5b4f00 2024-03-23T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e3cff9640 1 -- 172.21.15.76:0/3969468026 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9e50000e40 con 0x564e3d5b4f00 2024-03-23T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.141+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e50004960 con 0x564e3d5b4f00 2024-03-23T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e3cff9640 1 -- 172.21.15.76:0/3969468026 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e50004ad0 con 0x564e3d5b4f00 2024-03-23T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 10 client.25822 fetch_fsmap learned FSMap version 27 2024-03-23T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564e3d4db5a0 con 0x564e3d5b4f00 2024-03-23T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e3cff9640 1 -- 172.21.15.76:0/3969468026 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e50004960 con 0x564e3d5b4f00 2024-03-23T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e3cff9640 1 -- 172.21.15.76:0/3969468026 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e5003c040 con 0x564e3d5b4f00 2024-03-23T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 10 client.25822 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 20 client.25822 populate_metadata read hostname 'smithi076' 2024-03-23T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d63c950 0x564e3d63ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 4 client.25822 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.142+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e3d26b1b0 con 0x564e3d63c950 2024-03-23T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.147+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d63c950 0x564e3d63ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:45:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.149+0000 7f9e3cff9640 10 client.25822 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.149+0000 7f9e58ef9640 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d63c950 0x564e3d63ed70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48eb617386e99748 server_cookie=9411c59ae543d33b in_seq=0 out_seq=0 2024-03-23T15:45:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e3cff9640 1 -- 172.21.15.76:0/3969468026 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e3d26b1b0 con 0x564e3d63c950 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e3cff9640 10 client.25822 handle_command_reply: tid=0 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 client.25822 shutdown 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 msgr2=0x564e3d5b7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5b7e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x564e3d5bbe20 tx=0x7f9e500048f0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 shutdown_connections 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564e3d63c950 0x564e3d63ed70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e3d5bdcd0 0x564e3d5c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e3cff9640 0 client.25822 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e3d5b4f00 0x564e3d5b7e70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 --2- 172.21.15.76:0/3969468026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564e3d5b45e0 0x564e3d5b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 >> 172.21.15.76:0/3969468026 conn(0x564e3d5aef00 msgr2=0x564e3d5adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 shutdown_connections 2024-03-23T15:45:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 1 -- 172.21.15.76:0/3969468026 wait complete. 2024-03-23T15:45:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.630+0000 7f9e5e61e740 20 client.25822 trim_cache size 0 max 16384 2024-03-23T15:45:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:45:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:45:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:45:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9e400a22d0 msgr2=0x7f9e400a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9e400a22d0 0x7f9e400a4790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0cbd60 tx=0x7f9e4c06cf60 comp rx=0 tx=0).stop 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 msgr2=0x7f9e54197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54197b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00efd0 tx=0x7f9e4c00c6a0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 shutdown_connections 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9e400a22d0 0x7f9e400a4790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e541076d0 0x7f9e54197b00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e54102080 0x7f9e5419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 --2- 172.21.15.76:0/1320031092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e541016c0 0x7f9e5419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:45:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 >> 172.21.15.76:0/1320031092 conn(0x7f9e540fd1d0 msgr2=0x7f9e540fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:45:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 shutdown_connections 2024-03-23T15:45:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:45:54.631+0000 7f9e5b95e640 1 -- 172.21.15.76:0/1320031092 wait complete. 2024-03-23T15:45:55.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:55 smithi192 ceph-mon[36653]: pgmap v80: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 356 KiB/s wr, 77 op/s 2024-03-23T15:45:55.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:55 smithi076 ceph-mon[32176]: pgmap v80: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 356 KiB/s wr, 77 op/s 2024-03-23T15:45:55.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:55 smithi183 ceph-mon[36296]: pgmap v80: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 356 KiB/s wr, 77 op/s 2024-03-23T15:45:57.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:57 smithi192 ceph-mon[36653]: pgmap v81: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 450 KiB/s wr, 166 op/s 2024-03-23T15:45:57.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:57 smithi076 ceph-mon[32176]: pgmap v81: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 450 KiB/s wr, 166 op/s 2024-03-23T15:45:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:57 smithi183 ceph-mon[36296]: pgmap v81: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 450 KiB/s wr, 166 op/s 2024-03-23T15:45:59.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:45:59 smithi192 ceph-mon[36653]: pgmap v82: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 428 KiB/s wr, 146 op/s 2024-03-23T15:45:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:45:59 smithi076 ceph-mon[32176]: pgmap v82: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 428 KiB/s wr, 146 op/s 2024-03-23T15:45:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:45:59 smithi183 ceph-mon[36296]: pgmap v82: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 428 KiB/s wr, 146 op/s 2024-03-23T15:46:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:01 smithi192 ceph-mon[36653]: pgmap v83: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 425 KiB/s wr, 139 op/s 2024-03-23T15:46:01.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:01 smithi076 ceph-mon[32176]: pgmap v83: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 425 KiB/s wr, 139 op/s 2024-03-23T15:46:01.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:01 smithi183 ceph-mon[36296]: pgmap v83: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 425 KiB/s wr, 139 op/s 2024-03-23T15:46:03.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:03 smithi192 ceph-mon[36653]: pgmap v84: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-03-23T15:46:03.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:03 smithi076 ceph-mon[32176]: pgmap v84: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-03-23T15:46:03.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:03 smithi183 ceph-mon[36296]: pgmap v84: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-03-23T15:46:05.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:05 smithi192 ceph-mon[36653]: pgmap v85: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-03-23T15:46:05.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:05 smithi076 ceph-mon[32176]: pgmap v85: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-03-23T15:46:05.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:05 smithi183 ceph-mon[36296]: pgmap v85: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-03-23T15:46:06.256 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T15:46:06.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-23T15:46:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.726+0000 7f26bad25640 1 -- 172.21.15.76:0/2917852774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4106690 msgr2=0x7f26b4106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.726+0000 7f26bad25640 1 --2- 172.21.15.76:0/2917852774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4106690 0x7f26b4106a70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009900 tx=0x7f26a402bc30 comp rx=0 tx=0).stop 2024-03-23T15:46:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- 172.21.15.76:0/2917852774 shutdown_connections 2024-03-23T15:46:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 --2- 172.21.15.76:0/2917852774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4101200 0x7f26b410d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 --2- 172.21.15.76:0/2917852774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26b4100860 0x7f26b4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 --2- 172.21.15.76:0/2917852774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4106690 0x7f26b4106a70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- 172.21.15.76:0/2917852774 >> 172.21.15.76:0/2917852774 conn(0x7f26b4077470 msgr2=0x7f26b4077880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- 172.21.15.76:0/2917852774 shutdown_connections 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- 172.21.15.76:0/2917852774 wait complete. 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 Processor -- start 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- start start 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4100860 0x7f26b419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4101200 0x7f26b419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26b4106690 0x7f26b4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4198140 con 0x7f26b4100860 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41982b0 con 0x7f26b4101200 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.727+0000 7f26bad25640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4198470 con 0x7f26b4106690 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4100860 0x7f26b419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4100860 0x7f26b419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43764/0 (socket says 172.21.15.76:43764) 2024-03-23T15:46:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 -- 172.21.15.76:0/2323280187 learned_addr learned my addr 172.21.15.76:0/2323280187 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b3fff640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4101200 0x7f26b419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b92c2640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26b4106690 0x7f26b4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 -- 172.21.15.76:0/2323280187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26b4106690 msgr2=0x7f26b4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26b4106690 0x7f26b4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 -- 172.21.15.76:0/2323280187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4101200 msgr2=0x7f26b419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4101200 0x7f26b419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 -- 172.21.15.76:0/2323280187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a4009560 con 0x7f26b4100860 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4100860 0x7f26b419d8f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005940 tx=0x7f26a402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b1ffb640 1 -- 172.21.15.76:0/2323280187 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4004350 con 0x7f26b4100860 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b1ffb640 1 -- 172.21.15.76:0/2323280187 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a40044b0 con 0x7f26b4100860 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b41986f0 con 0x7f26b4100860 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.728+0000 7f26b1ffb640 1 -- 172.21.15.76:0/2323280187 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4003d30 con 0x7f26b4100860 2024-03-23T15:46:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.729+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a4990 con 0x7f26b4100860 2024-03-23T15:46:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26b1ffb640 1 -- 172.21.15.76:0/2323280187 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f26a4004df0 con 0x7f26b4100860 2024-03-23T15:46:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 Processor -- start 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 -- start start 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26b1ffb640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f269c0a22d0 0x7f269c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec22cd0 0x56507ec250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56507ebd41b0 con 0x56507ec19f00 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56507eb86f80 con 0x56507ec195e0 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.731+0000 7f26bd9de740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56507eb844b0 con 0x56507ec22cd0 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b92c2640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f269c0a22d0 0x7f269c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec22790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43772/0 (socket says 172.21.15.76:43772) 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b8ac1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 -- 172.21.15.76:0/2684913489 learned_addr learned my addr 172.21.15.76:0/2684913489 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b1ffb640 1 -- 172.21.15.76:0/2323280187 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f26a40f0030 con 0x7f26b4100860 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b92c2640 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec22cd0 0x56507ec250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 -- 172.21.15.76:0/2684913489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec22cd0 msgr2=0x56507ec250c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec22cd0 0x56507ec250c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 -- 172.21.15.76:0/2684913489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 msgr2=0x56507ec199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 -- 172.21.15.76:0/2684913489 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507eb15810 con 0x56507ec19f00 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b92c2640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f269c0a22d0 0x7f269c0a4790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26ac005fd0 tx=0x7f26ac009dd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f26b3fff640 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec22790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009c60 tx=0x7f26a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38474ef8c206fc01 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f269affd640 1 -- 172.21.15.76:0/2684913489 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8004230 con 0x56507ec19f00 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f269affd640 1 -- 172.21.15.76:0/2684913489 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a8009e10 con 0x56507ec19f00 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.732+0000 7f269affd640 1 -- 172.21.15.76:0/2684913489 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8019620 con 0x56507ec19f00 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 -- 172.21.15.76:0/2684913489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 msgr2=0x56507ec22790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec22790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009c60 tx=0x7f26a8011810 comp rx=0 tx=0).stop 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 -- 172.21.15.76:0/2684913489 shutdown_connections 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec22cd0 0x56507ec250c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec22790 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 --2- 172.21.15.76:0/2684913489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 -- 172.21.15.76:0/2684913489 >> 172.21.15.76:0/2684913489 conn(0x56507ec13f00 msgr2=0x56507ec142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 -- 172.21.15.76:0/2684913489 shutdown_connections 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 -- 172.21.15.76:0/2684913489 wait complete. 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 -- start start 2024-03-23T15:46:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.733+0000 7f26bd9de740 1 Processor -- start 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec195e0 0x56507ec1fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec20230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec20770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26bd9de740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56507eb7fc90 con 0x56507ec19f00 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26bd9de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56507eb24c50 con 0x56507ec22cd0 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26bd9de740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56507e853250 con 0x56507ec195e0 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26bd9de740 10 client.0 _ll_register_callbacks cb 0x56507eb73290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec20230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec20230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43776/0 (socket says 172.21.15.76:43776) 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 -- 172.21.15.76:0/3782879846 learned_addr learned my addr 172.21.15.76:0/3782879846 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b92c2640 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec20770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec195e0 0x56507ec1fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 -- 172.21.15.76:0/3782879846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec195e0 msgr2=0x56507ec1fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec195e0 0x56507ec1fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 -- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 msgr2=0x56507ec20770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec20770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 -- 172.21.15.76:0/3782879846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a8009890 con 0x56507ec19f00 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f26b3fff640 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec20230 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f26a8005890 tx=0x7f26a8003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef253418aa6d6005 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.734+0000 7f2693fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f2693fff640 1 -- 172.21.15.76:0/3782879846 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80041d0 con 0x56507ec19f00 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f2693fff640 1 -- 172.21.15.76:0/3782879846 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f26a8004330 con 0x56507ec19f00 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56507eba4790 con 0x56507ec19f00 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f2693fff640 1 -- 172.21.15.76:0/3782879846 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26a8005040 con 0x56507ec19f00 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 10 client.16056 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56507eb7e310 con 0x56507ec19f00 2024-03-23T15:46:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f2693fff640 1 -- 172.21.15.76:0/3782879846 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56507eba4790 con 0x56507ec19f00 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f2693fff640 1 -- 172.21.15.76:0/3782879846 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f26a8004cc0 con 0x56507ec19f00 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 10 client.16056 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 20 client.16056 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec32ba0 0x56507ec34fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 4 client.16056 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56507e87f600 con 0x56507ec32ba0 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.735+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec32ba0 0x56507ec34fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.736+0000 7f2693fff640 10 client.16056 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.736+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec32ba0 0x56507ec34fc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a399a44a697e1064 server_cookie=58064eeaac3671d2 in_seq=0 out_seq=0 2024-03-23T15:46:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.779+0000 7f2693fff640 1 -- 172.21.15.76:0/3782879846 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56507e87f600 con 0x56507ec32ba0 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.779+0000 7f2693fff640 10 client.16056 handle_command_reply: tid=0 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.779+0000 7f26bd9de740 1 client.16056 shutdown 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 msgr2=0x56507ec20230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec20230 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f26a8005890 tx=0x7f26a8003e20 comp rx=0 tx=0).stop 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 shutdown_connections 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec32ba0 0x56507ec34fc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec20770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec19f00 0x56507ec20230 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f2693fff640 0 client.16056 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3782879846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec195e0 0x56507ec1fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 >> 172.21.15.76:0/3782879846 conn(0x56507ec13f00 msgr2=0x56507ec15c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 shutdown_connections 2024-03-23T15:46:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 1 -- 172.21.15.76:0/3782879846 wait complete. 2024-03-23T15:46:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.780+0000 7f26bd9de740 20 client.16056 trim_cache size 0 max 16384 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 Processor -- start 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 -- start start 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec195e0 0x56507ec22420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec2e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec1a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56507eb05660 con 0x56507ec195e0 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56507ec0f6c0 con 0x56507ec22cd0 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26bd9de740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56507ec150c0 con 0x56507ec19f00 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26b8ac1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec195e0 0x56507ec22420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec2e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26b8ac1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec195e0 0x56507ec22420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43786/0 (socket says 172.21.15.76:43786) 2024-03-23T15:46:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26b8ac1640 1 -- 172.21.15.76:0/3854011195 learned_addr learned my addr 172.21.15.76:0/3854011195 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26b3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec2e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:38062/0 (socket says 172.21.15.76:38062) 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.798+0000 7f26b92c2640 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec1a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f26b8ac1640 1 -- 172.21.15.76:0/3854011195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 msgr2=0x56507ec2e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec2e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f26b8ac1640 1 -- 172.21.15.76:0/3854011195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 msgr2=0x56507ec1a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec1a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f26b8ac1640 1 -- 172.21.15.76:0/3854011195 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507eb1a810 con 0x56507ec195e0 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec195e0 0x56507ec22420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f26a40edd80 tx=0x7f26a404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9927aef997c963 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f269a7fc640 1 -- 172.21.15.76:0/3854011195 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40035f0 con 0x56507ec195e0 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f269a7fc640 1 -- 172.21.15.76:0/3854011195 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a40bcb30 con 0x56507ec195e0 2024-03-23T15:46:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.799+0000 7f269a7fc640 1 -- 172.21.15.76:0/3854011195 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40bae30 con 0x56507ec195e0 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 -- 172.21.15.76:0/3854011195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec195e0 msgr2=0x56507ec22420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec195e0 0x56507ec22420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f26a40edd80 tx=0x7f26a404d000 comp rx=0 tx=0).stop 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 -- 172.21.15.76:0/3854011195 shutdown_connections 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec22cd0 0x56507ec1a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec2e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3854011195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec195e0 0x56507ec22420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 -- 172.21.15.76:0/3854011195 >> 172.21.15.76:0/3854011195 conn(0x56507ec13f00 msgr2=0x56507ec142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 -- 172.21.15.76:0/3854011195 shutdown_connections 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 -- 172.21.15.76:0/3854011195 wait complete. 2024-03-23T15:46:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 -- start start 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.800+0000 7f26bd9de740 1 Processor -- start 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec26450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec26990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26bd9de740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec22cd0 0x56507ec26ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26bd9de740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56507ec97f60 con 0x56507ec22cd0 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26bd9de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56507ec980d0 con 0x56507ec195e0 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26bd9de740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56507ec98290 con 0x56507ec19f00 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26bd9de740 10 client.0 _ll_register_callbacks cb 0x7f26ac013700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec22cd0 0x56507ec26ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec22cd0 0x56507ec26ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43790/0 (socket says 172.21.15.76:43790) 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 -- 172.21.15.76:0/3070002876 learned_addr learned my addr 172.21.15.76:0/3070002876 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec26450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b3fff640 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec26990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 -- 172.21.15.76:0/3070002876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 msgr2=0x56507ec26990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec26990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 -- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 msgr2=0x56507ec26450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec26450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 -- 172.21.15.76:0/3070002876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41986f0 con 0x56507ec22cd0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f26b92c2640 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec22cd0 0x56507ec26ed0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26ac011870 tx=0x7f26ac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d012156866320fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.801+0000 7f2693fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f2693fff640 1 -- 172.21.15.76:0/3070002876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac026040 con 0x56507ec22cd0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f2693fff640 1 -- 172.21.15.76:0/3070002876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f26ac013020 con 0x56507ec22cd0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a8009e10 con 0x56507ec22cd0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f2693fff640 1 -- 172.21.15.76:0/3070002876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26ac0131e0 con 0x56507ec22cd0 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 10 client.16062 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56507eb405a0 con 0x56507ec22cd0 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f2693fff640 1 -- 172.21.15.76:0/3070002876 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a8009e10 con 0x56507ec22cd0 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f2693fff640 1 -- 172.21.15.76:0/3070002876 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f26ac021410 con 0x56507ec22cd0 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 10 client.16062 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 20 client.16062 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec99be0 0x56507ec9c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 4 client.16062 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56507e8d01b0 con 0x56507ec99be0 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.802+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec99be0 0x56507ec9c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.803+0000 7f2693fff640 10 client.16062 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:07.803+0000 7f26b8ac1640 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec99be0 0x56507ec9c000 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db47d88ba697ade3 server_cookie=a84e2df4a8202001 in_seq=0 out_seq=0 2024-03-23T15:46:07.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:07 smithi192 ceph-mon[36653]: pgmap v86: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-03-23T15:46:07.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:07 smithi076 ceph-mon[32176]: pgmap v86: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-03-23T15:46:07.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:07 smithi183 ceph-mon[36296]: pgmap v86: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-03-23T15:46:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f2693fff640 1 -- 172.21.15.76:0/3070002876 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56507e8d01b0 con 0x56507ec99be0 2024-03-23T15:46:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f2693fff640 10 client.16062 handle_command_reply: tid=0 2024-03-23T15:46:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 client.16062 shutdown 2024-03-23T15:46:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec22cd0 msgr2=0x56507ec26ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec22cd0 0x56507ec26ed0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26ac011870 tx=0x7f26ac01d000 comp rx=0 tx=0).stop 2024-03-23T15:46:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 shutdown_connections 2024-03-23T15:46:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56507ec22cd0 0x56507ec26ed0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56507ec19f00 0x56507ec26990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x56507ec99be0 0x56507ec9c000 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 --2- 172.21.15.76:0/3070002876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507ec195e0 0x56507ec26450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f2693fff640 0 client.16062 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 >> 172.21.15.76:0/3070002876 conn(0x56507ec13f00 msgr2=0x56507ec1a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 shutdown_connections 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 1 -- 172.21.15.76:0/3070002876 wait complete. 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.551+0000 7f26bd9de740 20 client.16062 trim_cache size 0 max 16384 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:46:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:46:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:46:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:46:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f269c0a22d0 msgr2=0x7f269c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f269c0a22d0 0x7f269c0a4790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26ac005fd0 tx=0x7f26ac009dd0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4100860 msgr2=0x7f26b419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4100860 0x7f26b419d8f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f26a4005940 tx=0x7f26a402c670 comp rx=0 tx=0).stop 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 shutdown_connections 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f269c0a22d0 0x7f269c0a4790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26b4106690 0x7f26b4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b4101200 0x7f26b419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 --2- 172.21.15.76:0/2323280187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26b4100860 0x7f26b419d8f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 >> 172.21.15.76:0/2323280187 conn(0x7f26b4077470 msgr2=0x7f26b40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 shutdown_connections 2024-03-23T15:46:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:08.552+0000 7f26bad25640 1 -- 172.21.15.76:0/2323280187 wait complete. 2024-03-23T15:46:09.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:09 smithi192 ceph-mon[36653]: pgmap v87: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 61 op/s 2024-03-23T15:46:09.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:09 smithi076 ceph-mon[32176]: pgmap v87: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 61 op/s 2024-03-23T15:46:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:09 smithi183 ceph-mon[36296]: pgmap v87: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 61 op/s 2024-03-23T15:46:11.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:11 smithi192 ceph-mon[36653]: pgmap v88: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 42 op/s 2024-03-23T15:46:11.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:11 smithi076 ceph-mon[32176]: pgmap v88: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 42 op/s 2024-03-23T15:46:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:11 smithi183 ceph-mon[36296]: pgmap v88: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 42 op/s 2024-03-23T15:46:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:13 smithi192 ceph-mon[36653]: pgmap v89: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 345 op/s 2024-03-23T15:46:13.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:13 smithi076 ceph-mon[32176]: pgmap v89: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 345 op/s 2024-03-23T15:46:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:13 smithi183 ceph-mon[36296]: pgmap v89: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 345 op/s 2024-03-23T15:46:15.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:15 smithi192 ceph-mon[36653]: pgmap v90: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 938 KiB/s wr, 309 op/s 2024-03-23T15:46:15.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:15 smithi076 ceph-mon[32176]: pgmap v90: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 938 KiB/s wr, 309 op/s 2024-03-23T15:46:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:15 smithi183 ceph-mon[36296]: pgmap v90: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 938 KiB/s wr, 309 op/s 2024-03-23T15:46:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:17 smithi192 ceph-mon[36653]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 353 op/s 2024-03-23T15:46:17.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:17 smithi076 ceph-mon[32176]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 353 op/s 2024-03-23T15:46:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:17 smithi183 ceph-mon[36296]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 353 op/s 2024-03-23T15:46:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:19 smithi192 ceph-mon[36653]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 897 KiB/s wr, 346 op/s 2024-03-23T15:46:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:19 smithi076 ceph-mon[32176]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 897 KiB/s wr, 346 op/s 2024-03-23T15:46:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:19 smithi183 ceph-mon[36296]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 897 KiB/s wr, 346 op/s 2024-03-23T15:46:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.649+0000 7f20bc864640 1 -- 172.21.15.76:0/3793171915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b81016c0 msgr2=0x7f20b8101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.649+0000 7f20bc864640 1 --2- 172.21.15.76:0/3793171915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b81016c0 0x7f20b8101b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009930 tx=0x7f20a002bc30 comp rx=0 tx=0).stop 2024-03-23T15:46:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.649+0000 7f20bc864640 1 -- 172.21.15.76:0/3793171915 shutdown_connections 2024-03-23T15:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.649+0000 7f20bc864640 1 --2- 172.21.15.76:0/3793171915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b8102080 0x7f20b810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.649+0000 7f20bc864640 1 --2- 172.21.15.76:0/3793171915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b81016c0 0x7f20b8101b40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.649+0000 7f20bc864640 1 --2- 172.21.15.76:0/3793171915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20b81076d0 0x7f20b8107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.649+0000 7f20bc864640 1 -- 172.21.15.76:0/3793171915 >> 172.21.15.76:0/3793171915 conn(0x7f20b80fd190 msgr2=0x7f20b80ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.650+0000 7f20bc864640 1 -- 172.21.15.76:0/3793171915 shutdown_connections 2024-03-23T15:46:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.650+0000 7f20bc864640 1 -- 172.21.15.76:0/3793171915 wait complete. 2024-03-23T15:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.650+0000 7f20bc864640 1 Processor -- start 2024-03-23T15:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.650+0000 7f20bc864640 1 -- start start 2024-03-23T15:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20bc864640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b81016c0 0x7f20b819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20bc864640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b8102080 0x7f20b819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20bc864640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20b81076d0 0x7f20b8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20bc864640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f20b8198110 con 0x7f20b8102080 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20bc864640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20b8198280 con 0x7f20b81076d0 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20bc864640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20b8198440 con 0x7f20b81016c0 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b8102080 0x7f20b819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b8102080 0x7f20b819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39086/0 (socket says 172.21.15.76:39086) 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20b659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b81016c0 0x7f20b819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20b81076d0 0x7f20b8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 -- 172.21.15.76:0/1423926928 learned_addr learned my addr 172.21.15.76:0/1423926928 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 -- 172.21.15.76:0/1423926928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b81016c0 msgr2=0x7f20b819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b81016c0 0x7f20b819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 -- 172.21.15.76:0/1423926928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20b81076d0 msgr2=0x7f20b8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20b81076d0 0x7f20b8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 -- 172.21.15.76:0/1423926928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a0009560 con 0x7f20b8102080 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20b659c640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b81016c0 0x7f20b819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20b81076d0 0x7f20b8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20affff640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b8102080 0x7f20b819de50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009230 tx=0x7f20a002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20af7fe640 1 -- 172.21.15.76:0/1423926928 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0004350 con 0x7f20b8102080 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20af7fe640 1 -- 172.21.15.76:0/1423926928 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a00044b0 con 0x7f20b8102080 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.651+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b81986c0 con 0x7f20b8102080 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.652+0000 7f20af7fe640 1 -- 172.21.15.76:0/1423926928 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0003d00 con 0x7f20b8102080 2024-03-23T15:46:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.652+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b81a49b0 con 0x7f20b8102080 2024-03-23T15:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.654+0000 7f20af7fe640 1 -- 172.21.15.76:0/1423926928 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f20a0004df0 con 0x7f20b8102080 2024-03-23T15:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.654+0000 7f20bf442740 1 Processor -- start 2024-03-23T15:46:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.654+0000 7f20bf442740 1 -- start start 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20af7fe640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f20a40a22d0 0x7f20a40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20bf442740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059da95e0 0x557059da99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20bf442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20bf442740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059db2cd0 0x557059db50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20bf442740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557059d641b0 con 0x557059da95e0 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20bf442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557059d16f80 con 0x557059da9f00 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20bf442740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557059d144b0 con 0x557059db2cd0 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f20a40a22d0 0x7f20a40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059da95e0 0x557059da99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059da95e0 0x557059da99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39096/0 (socket says 172.21.15.76:39096) 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 -- 172.21.15.76:0/729878162 learned_addr learned my addr 172.21.15.76:0/729878162 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20af7fe640 1 -- 172.21.15.76:0/1423926928 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f20a00effd0 con 0x7f20b8102080 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059db2cd0 0x557059db50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20affff640 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 -- 172.21.15.76:0/729878162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059db2cd0 msgr2=0x557059db50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059db2cd0 0x557059db50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 -- 172.21.15.76:0/729878162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 msgr2=0x557059db2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 -- 172.21.15.76:0/729878162 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557059ca5810 con 0x557059da95e0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f20a40a22d0 0x7f20a40a4790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20b0005fd0 tx=0x7f20b0013660 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.655+0000 7f20b659c640 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059da95e0 0x557059da99c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f20a8011470 tx=0x7f20a8011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34028992d522b624 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20acff9640 1 -- 172.21.15.76:0/729878162 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8004020 con 0x557059da95e0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20acff9640 1 -- 172.21.15.76:0/729878162 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a80035f0 con 0x557059da95e0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20acff9640 1 -- 172.21.15.76:0/729878162 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a800f7a0 con 0x557059da95e0 2024-03-23T15:46:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 -- 172.21.15.76:0/729878162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059da95e0 msgr2=0x557059da99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059da95e0 0x557059da99c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f20a8011470 tx=0x7f20a8011870 comp rx=0 tx=0).stop 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 -- 172.21.15.76:0/729878162 shutdown_connections 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059db2cd0 0x557059db50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 --2- 172.21.15.76:0/729878162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059da95e0 0x557059da99c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 -- 172.21.15.76:0/729878162 >> 172.21.15.76:0/729878162 conn(0x557059da3f00 msgr2=0x557059da42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 -- 172.21.15.76:0/729878162 shutdown_connections 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.656+0000 7f20bf442740 1 -- 172.21.15.76:0/729878162 wait complete. 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 -- start start 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 Processor -- start 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dafd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557059d0fc90 con 0x557059db2cd0 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557059cb4c50 con 0x557059da9f00 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5570599e3250 con 0x557059da95e0 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20bf442740 10 client.0 _ll_register_callbacks cb 0x557059d03290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20b659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dafd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20affff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39106/0 (socket says 172.21.15.76:39106) 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20b6d9d640 1 -- 172.21.15.76:0/53049608 learned_addr learned my addr 172.21.15.76:0/53049608 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.657+0000 7f20b659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dafd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:46142/0 (socket says 172.21.15.76:46142) 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20b6d9d640 1 -- 172.21.15.76:0/53049608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 msgr2=0x557059dafd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dafd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20b6d9d640 1 -- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 msgr2=0x557059db0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.667 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20b6d9d640 1 -- 172.21.15.76:0/53049608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a8009c40 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20affff640 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20b659c640 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dafd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db0790 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f20b0003bd0 tx=0x7f20b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85b2762ef83d5e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f209dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f209dffb640 1 -- 172.21.15.76:0/53049608 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0004680 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f209dffb640 1 -- 172.21.15.76:0/53049608 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f20b00047e0 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557059d34790 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f209dffb640 1 -- 172.21.15.76:0/53049608 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20b0004b20 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20bf442740 10 client.16074 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557059d0e310 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f209dffb640 1 -- 172.21.15.76:0/53049608 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557059d34790 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f209dffb640 1 -- 172.21.15.76:0/53049608 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f20b0033480 con 0x557059db2cd0 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20bf442740 10 client.16074 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20bf442740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.658+0000 7f20bf442740 20 client.16074 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.659+0000 7f20bf442740 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059dc2c80 0x557059dc50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.659+0000 7f20bf442740 4 client.16074 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.659+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557059a0f600 con 0x557059dc2c80 2024-03-23T15:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.659+0000 7f20b659c640 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059dc2c80 0x557059dc50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.659+0000 7f209dffb640 10 client.16074 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:21.669 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.659+0000 7f20b659c640 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059dc2c80 0x557059dc50a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56b26b025ca44a server_cookie=82cb6351e29fbbc3 in_seq=0 out_seq=0 2024-03-23T15:46:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.676+0000 7f209dffb640 1 -- 172.21.15.76:0/53049608 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557059a0f600 con 0x557059dc2c80 2024-03-23T15:46:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.676+0000 7f209dffb640 10 client.16074 handle_command_reply: tid=0 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.676+0000 7f20bf442740 1 client.16074 shutdown 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 msgr2=0x557059db0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db0790 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f20b0003bd0 tx=0x7f20b0004ab0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 shutdown_connections 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db0790 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059dc2c80 0x557059dc50a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 --2- 172.21.15.76:0/53049608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dafd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 >> 172.21.15.76:0/53049608 conn(0x557059da3f00 msgr2=0x557059da5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f209dffb640 0 client.16074 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 shutdown_connections 2024-03-23T15:46:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 1 -- 172.21.15.76:0/53049608 wait complete. 2024-03-23T15:46:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.677+0000 7f20bf442740 20 client.16074 trim_cache size 0 max 16384 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 Processor -- start 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 -- start start 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dbe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059da2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557059c95660 con 0x557059db2cd0 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557059d34790 con 0x557059da9f00 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20bf442740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557059da23f0 con 0x557059da95e0 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059da2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20b659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dbe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059da2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39114/0 (socket says 172.21.15.76:39114) 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20b6d9d640 1 -- 172.21.15.76:0/3460730642 learned_addr learned my addr 172.21.15.76:0/3460730642 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.692+0000 7f20affff640 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f20b659c640 1 -- 172.21.15.76:0/3460730642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 msgr2=0x557059db05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f20b659c640 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f20b659c640 1 -- 172.21.15.76:0/3460730642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 msgr2=0x557059da2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f20b659c640 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059da2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f20b659c640 1 -- 172.21.15.76:0/3460730642 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557059caa810 con 0x557059da95e0 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059da2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f20b659c640 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dbe3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20a801b040 tx=0x7f20a8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c730363cbfb3ffcb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f209ffff640 1 -- 172.21.15.76:0/3460730642 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8003a70 con 0x557059da95e0 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f209ffff640 1 -- 172.21.15.76:0/3460730642 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a8005520 con 0x557059da95e0 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.693+0000 7f209ffff640 1 -- 172.21.15.76:0/3460730642 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8004910 con 0x557059da95e0 2024-03-23T15:46:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- 172.21.15.76:0/3460730642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 msgr2=0x557059dbe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dbe3a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20a801b040 tx=0x7f20a8009310 comp rx=0 tx=0).stop 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- 172.21.15.76:0/3460730642 shutdown_connections 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059da2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da9f00 0x557059db05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 --2- 172.21.15.76:0/3460730642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da95e0 0x557059dbe3a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- 172.21.15.76:0/3460730642 >> 172.21.15.76:0/3460730642 conn(0x557059da3f00 msgr2=0x557059da2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- 172.21.15.76:0/3460730642 shutdown_connections 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- 172.21.15.76:0/3460730642 wait complete. 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- start start 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 Processor -- start 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da95e0 0x557059db61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da9f00 0x557059db6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557059dbb1c0 con 0x557059db2cd0 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557059e27fe0 con 0x557059da95e0 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557059e28120 con 0x557059da9f00 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20bf442740 10 client.0 _ll_register_callbacks cb 0x7f20b00137f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.694+0000 7f20b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39124/0 (socket says 172.21.15.76:39124) 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 -- 172.21.15.76:0/1479506791 learned_addr learned my addr 172.21.15.76:0/1479506791 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da95e0 0x557059db61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20affff640 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da9f00 0x557059db6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 -- 172.21.15.76:0/1479506791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da9f00 msgr2=0x557059db6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da9f00 0x557059db6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 -- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da95e0 msgr2=0x557059db61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da95e0 0x557059db61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 -- 172.21.15.76:0/1479506791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557059d0e310 con 0x557059db2cd0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b659c640 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da95e0 0x557059db61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20b6d9d640 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db6c70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f20b0033a00 tx=0x7f20b00336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f611fda856045ac0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f209dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f209dffb640 1 -- 172.21.15.76:0/1479506791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0033c90 con 0x557059db2cd0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f209dffb640 1 -- 172.21.15.76:0/1479506791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f20b0009e40 con 0x557059db2cd0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a80035f0 con 0x557059db2cd0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f209dffb640 1 -- 172.21.15.76:0/1479506791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20b0004680 con 0x557059db2cd0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20bf442740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557059ca5810 con 0x557059db2cd0 2024-03-23T15:46:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.695+0000 7f209dffb640 1 -- 172.21.15.76:0/1479506791 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a80035f0 con 0x557059db2cd0 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f209dffb640 1 -- 172.21.15.76:0/1479506791 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f20b00213f0 con 0x557059db2cd0 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20bf442740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20bf442740 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20bf442740 20 client.16080 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20bf442740 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059e29970 0x557059e2bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20bf442740 4 client.16080 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557059a601b0 con 0x557059e29970 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20b659c640 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059e29970 0x557059e2bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f209dffb640 10 client.16080 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:21.696+0000 7f20b659c640 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059e29970 0x557059e2bd90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18fd669be4c1d861 server_cookie=89bb88f0474abe32 in_seq=0 out_seq=0 2024-03-23T15:46:21.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:21 smithi192 ceph-mon[36653]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 897 KiB/s wr, 346 op/s 2024-03-23T15:46:21.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:21 smithi076 ceph-mon[32176]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 897 KiB/s wr, 346 op/s 2024-03-23T15:46:21.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:21 smithi183 ceph-mon[36296]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 897 KiB/s wr, 346 op/s 2024-03-23T15:46:22.093 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f209dffb640 1 -- 172.21.15.76:0/1479506791 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557059a601b0 con 0x557059e29970 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f209dffb640 10 client.16080 handle_command_reply: tid=0 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 client.16080 shutdown 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 msgr2=0x557059db6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db6c70 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f20b0033a00 tx=0x7f20b00336b0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 shutdown_connections 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557059db2cd0 0x557059db6c70 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557059da9f00 0x557059db6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x557059e29970 0x557059e2bd90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f209dffb640 0 client.16080 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 --2- 172.21.15.76:0/1479506791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557059da95e0 0x557059db61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.090+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 >> 172.21.15.76:0/1479506791 conn(0x557059da3f00 msgr2=0x557059da2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 shutdown_connections 2024-03-23T15:46:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bf442740 1 -- 172.21.15.76:0/1479506791 wait complete. 2024-03-23T15:46:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bf442740 20 client.16080 trim_cache size 0 max 16384 2024-03-23T15:46:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:46:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:46:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:46:22.095 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:46:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f20a40a22d0 msgr2=0x7f20a40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f20a40a22d0 0x7f20a40a4790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20b0005fd0 tx=0x7f20b0013660 comp rx=0 tx=0).stop 2024-03-23T15:46:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b8102080 msgr2=0x7f20b819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:22.096 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b8102080 0x7f20b819de50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009230 tx=0x7f20a002c650 comp rx=0 tx=0).stop 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 shutdown_connections 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f20a40a22d0 0x7f20a40a4790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20b81076d0 0x7f20b8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20b8102080 0x7f20b819de50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 --2- 172.21.15.76:0/1423926928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b81016c0 0x7f20b819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 >> 172.21.15.76:0/1423926928 conn(0x7f20b80fd190 msgr2=0x7f20b80fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 shutdown_connections 2024-03-23T15:46:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:22.091+0000 7f20bc864640 1 -- 172.21.15.76:0/1423926928 wait complete. 2024-03-23T15:46:23.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:23 smithi192 ceph-mon[36653]: pgmap v94: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.2 MiB/s wr, 400 op/s 2024-03-23T15:46:23.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:23 smithi076 ceph-mon[32176]: pgmap v94: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.2 MiB/s wr, 400 op/s 2024-03-23T15:46:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:23 smithi183 ceph-mon[36296]: pgmap v94: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.2 MiB/s wr, 400 op/s 2024-03-23T15:46:25.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:25 smithi192 ceph-mon[36653]: pgmap v95: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 547 KiB/s wr, 97 op/s 2024-03-23T15:46:25.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:25 smithi076 ceph-mon[32176]: pgmap v95: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 547 KiB/s wr, 97 op/s 2024-03-23T15:46:25.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:25 smithi183 ceph-mon[36296]: pgmap v95: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 547 KiB/s wr, 97 op/s 2024-03-23T15:46:27.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:27 smithi192 ceph-mon[36653]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 899 KiB/s wr, 225 op/s 2024-03-23T15:46:27.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:27 smithi076 ceph-mon[32176]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 899 KiB/s wr, 225 op/s 2024-03-23T15:46:27.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:27 smithi183 ceph-mon[36296]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 899 KiB/s wr, 225 op/s 2024-03-23T15:46:28.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:28 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:46:28.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:28 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:46:28.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:28 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:46:28.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:28 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:46:28.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:28 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:46:28.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:28 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:46:28.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:28 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:46:28.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:28 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:46:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:28 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:46:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:28 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:46:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:28 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:46:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:28 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:46:29.785 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:29 smithi183 ceph-mon[36296]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 639 KiB/s wr, 181 op/s 2024-03-23T15:46:29.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:29 smithi192 ceph-mon[36653]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 639 KiB/s wr, 181 op/s 2024-03-23T15:46:29.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:29 smithi076 ceph-mon[32176]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 639 KiB/s wr, 181 op/s 2024-03-23T15:46:31.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:31 smithi192 ceph-mon[36653]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 639 KiB/s wr, 181 op/s 2024-03-23T15:46:31.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:31 smithi076 ceph-mon[32176]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 639 KiB/s wr, 181 op/s 2024-03-23T15:46:31.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:31 smithi183 ceph-mon[36296]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 639 KiB/s wr, 181 op/s 2024-03-23T15:46:33.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:33 smithi192 ceph-mon[36653]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 928 KiB/s wr, 240 op/s 2024-03-23T15:46:33.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:33 smithi076 ceph-mon[32176]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 928 KiB/s wr, 240 op/s 2024-03-23T15:46:33.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:33 smithi183 ceph-mon[36296]: pgmap v99: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 928 KiB/s wr, 240 op/s 2024-03-23T15:46:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 -- 172.21.15.76:0/590834526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 msgr2=0x7f31bc10d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/590834526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 0x7f31bc10d440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009c10 tx=0x7f31b402bef0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 -- 172.21.15.76:0/590834526 shutdown_connections 2024-03-23T15:46:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/590834526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 0x7f31bc10d440 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/590834526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31bc100860 0x7f31bc100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/590834526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc1055b0 0x7f31bc105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 -- 172.21.15.76:0/590834526 >> 172.21.15.76:0/590834526 conn(0x7f31bc077460 msgr2=0x7f31bc077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 -- 172.21.15.76:0/590834526 shutdown_connections 2024-03-23T15:46:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.189+0000 7f31c2bb5640 1 -- 172.21.15.76:0/590834526 wait complete. 2024-03-23T15:46:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 Processor -- start 2024-03-23T15:46:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 -- start start 2024-03-23T15:46:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc100860 0x7f31bc19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 0x7f31bc19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31bc1055b0 0x7f31bc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1982a0 con 0x7f31bc100860 2024-03-23T15:46:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc198410 con 0x7f31bc1055b0 2024-03-23T15:46:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c2bb5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1985d0 con 0x7f31bc101220 2024-03-23T15:46:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c0951640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc100860 0x7f31bc19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c0951640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc100860 0x7f31bc19da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50662/0 (socket says 172.21.15.76:50662) 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c0951640 1 -- 172.21.15.76:0/211605048 learned_addr learned my addr 172.21.15.76:0/211605048 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31bbfff640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 0x7f31bc19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.190+0000 7f31c1152640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31bc1055b0 0x7f31bc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c0951640 1 -- 172.21.15.76:0/211605048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 msgr2=0x7f31bc19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c0951640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 0x7f31bc19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c0951640 1 -- 172.21.15.76:0/211605048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31bc1055b0 msgr2=0x7f31bc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c0951640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31bc1055b0 0x7f31bc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c0951640 1 -- 172.21.15.76:0/211605048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b0009560 con 0x7f31bc100860 2024-03-23T15:46:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31bbfff640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 0x7f31bc19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c0951640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc100860 0x7f31bc19da50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f31b000e9c0 tx=0x7f31b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31b9ffb640 1 -- 172.21.15.76:0/211605048 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b000cde0 con 0x7f31bc100860 2024-03-23T15:46:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b4009840 con 0x7f31bc100860 2024-03-23T15:46:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31b9ffb640 1 -- 172.21.15.76:0/211605048 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b00035f0 con 0x7f31bc100860 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31b9ffb640 1 -- 172.21.15.76:0/211605048 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0010650 con 0x7f31bc100860 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.191+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1a49d0 con 0x7f31bc100860 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.193+0000 7f31b9ffb640 1 -- 172.21.15.76:0/211605048 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f31b0003ad0 con 0x7f31bc100860 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 Processor -- start 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31b9ffb640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f31ac0a22d0 0x7f31ac0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 -- start start 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c1152640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f31ac0a22d0 0x7f31ac0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568f5e0 0x562d9568f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568ff00 0x562d95698790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31b9ffb640 1 -- 172.21.15.76:0/211605048 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f31b0014070 con 0x7f31bc100860 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 0x562d9569b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562d9564a1b0 con 0x562d95698cd0 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d955fcf80 con 0x562d9568f5e0 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c579a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d955fa4b0 con 0x562d9568ff00 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c0951640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568f5e0 0x562d9568f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c1152640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 0x562d9569b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31bbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568ff00 0x562d95698790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c0951640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568f5e0 0x562d9568f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:44456/0 (socket says 172.21.15.76:44456) 2024-03-23T15:46:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.194+0000 7f31c0951640 1 -- 172.21.15.76:0/125150252 learned_addr learned my addr 172.21.15.76:0/125150252 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31bbfff640 1 -- 172.21.15.76:0/125150252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568f5e0 msgr2=0x562d9568f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31bbfff640 1 --2- 172.21.15.76:0/125150252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568f5e0 0x562d9568f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31bbfff640 1 -- 172.21.15.76:0/125150252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 msgr2=0x562d9569b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31bbfff640 1 --2- 172.21.15.76:0/125150252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 0x562d9569b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31bbfff640 1 -- 172.21.15.76:0/125150252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9558b810 con 0x562d9568ff00 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31c1152640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f31ac0a22d0 0x7f31ac0a4790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006010 tx=0x7f31b4033ad0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31bbfff640 1 --2- 172.21.15.76:0/125150252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568ff00 0x562d95698790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009fa0 tx=0x7f31a80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1cea0c101ffedf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31a6ffd640 1 -- 172.21.15.76:0/125150252 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8004310 con 0x562d9568ff00 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31a6ffd640 1 -- 172.21.15.76:0/125150252 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a8004470 con 0x562d9568ff00 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.195+0000 7f31a6ffd640 1 -- 172.21.15.76:0/125150252 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a80035f0 con 0x562d9568ff00 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 -- 172.21.15.76:0/125150252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568ff00 msgr2=0x562d95698790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 --2- 172.21.15.76:0/125150252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568ff00 0x562d95698790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009fa0 tx=0x7f31a80118d0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 -- 172.21.15.76:0/125150252 shutdown_connections 2024-03-23T15:46:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 --2- 172.21.15.76:0/125150252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 0x562d9569b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 --2- 172.21.15.76:0/125150252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568ff00 0x562d95698790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 --2- 172.21.15.76:0/125150252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568f5e0 0x562d9568f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 -- 172.21.15.76:0/125150252 >> 172.21.15.76:0/125150252 conn(0x562d95689f00 msgr2=0x562d9568a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 -- 172.21.15.76:0/125150252 shutdown_connections 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 -- 172.21.15.76:0/125150252 wait complete. 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 -- start start 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.196+0000 7f31c579a740 1 Processor -- start 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568f5e0 0x562d95695d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d95696280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d95698cd0 0x562d956967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562d955f5c90 con 0x562d9568f5e0 2024-03-23T15:46:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d9559ac50 con 0x562d9568ff00 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d952c9250 con 0x562d95698cd0 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568f5e0 0x562d95695d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 10 client.0 _ll_register_callbacks cb 0x562d955e9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568f5e0 0x562d95695d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50688/0 (socket says 172.21.15.76:50688) 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 -- 172.21.15.76:0/2091732739 learned_addr learned my addr 172.21.15.76:0/2091732739 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31bbfff640 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d95696280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c1152640 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d95698cd0 0x562d956967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 -- 172.21.15.76:0/2091732739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d95698cd0 msgr2=0x562d956967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d95698cd0 0x562d956967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 -- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 msgr2=0x562d95696280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d95696280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 -- 172.21.15.76:0/2091732739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a8009c00 con 0x562d9568f5e0 2024-03-23T15:46:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c0951640 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568f5e0 0x562d95695d40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c6a0 tx=0x7f31b009b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2cc31e80cb5fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f319bfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:46:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f319bfff640 1 -- 172.21.15.76:0/2091732739 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00a3650 con 0x562d9568f5e0 2024-03-23T15:46:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f319bfff640 1 -- 172.21.15.76:0/2091732739 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f31b0097520 con 0x562d9568f5e0 2024-03-23T15:46:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.197+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d9561a790 con 0x562d9568f5e0 2024-03-23T15:46:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f319bfff640 1 -- 172.21.15.76:0/2091732739 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f31b00976e0 con 0x562d9568f5e0 2024-03-23T15:46:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 10 client.16092 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d955f4310 con 0x562d9568f5e0 2024-03-23T15:46:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f319bfff640 1 -- 172.21.15.76:0/2091732739 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562d9561a790 con 0x562d9568f5e0 2024-03-23T15:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f319bfff640 1 -- 172.21.15.76:0/2091732739 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f31b001f060 con 0x562d9568f5e0 2024-03-23T15:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 10 client.16092 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 20 client.16092 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d956a8ca0 0x562d956ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 4 client.16092 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d952f5600 con 0x562d956a8ca0 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.198+0000 7f31bbfff640 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d956a8ca0 0x562d956ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.199+0000 7f319bfff640 10 client.16092 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.199+0000 7f31bbfff640 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d956a8ca0 0x562d956ab0e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5261395110705f83 server_cookie=de1d7b334a946bb6 in_seq=0 out_seq=0 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f319bfff640 1 -- 172.21.15.76:0/2091732739 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562d952f5600 con 0x562d956a8ca0 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f319bfff640 10 client.16092 handle_command_reply: tid=0 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 client.16092 shutdown 2024-03-23T15:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568f5e0 msgr2=0x562d95695d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568f5e0 0x562d95695d40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c6a0 tx=0x7f31b009b5b0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 shutdown_connections 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d95698cd0 0x562d956967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d956a8ca0 0x562d956ab0e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d95696280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f319bfff640 0 client.16092 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 --2- 172.21.15.76:0/2091732739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568f5e0 0x562d95695d40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 >> 172.21.15.76:0/2091732739 conn(0x562d95689f00 msgr2=0x562d9568bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 shutdown_connections 2024-03-23T15:46:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 1 -- 172.21.15.76:0/2091732739 wait complete. 2024-03-23T15:46:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.200+0000 7f31c579a740 20 client.16092 trim_cache size 0 max 16384 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.215+0000 7f31c579a740 1 Processor -- start 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.215+0000 7f31c579a740 1 -- start start 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c579a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c579a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d956b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c579a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 0x562d956965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c579a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562d9557b660 con 0x562d95698cd0 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c579a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d9561a7c0 con 0x562d9568ff00 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c579a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d9568aad0 con 0x562d9568f5e0 2024-03-23T15:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:50022/0 (socket says 172.21.15.76:50022) 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 -- 172.21.15.76:0/1951541002 learned_addr learned my addr 172.21.15.76:0/1951541002 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31bbfff640 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d956b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 -- 172.21.15.76:0/1951541002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 msgr2=0x562d956b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d956b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 -- 172.21.15.76:0/1951541002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 msgr2=0x562d956965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 0x562d956965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 -- 172.21.15.76:0/1951541002 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d95590810 con 0x562d9568f5e0 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.216+0000 7f31c0951640 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a43a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009530 tx=0x7f31b00993c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b10ce566c74531b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31a67fc640 1 -- 172.21.15.76:0/1951541002 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b009ac80 con 0x562d9568f5e0 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31a67fc640 1 -- 172.21.15.76:0/1951541002 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b00a3650 con 0x562d9568f5e0 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31a67fc640 1 -- 172.21.15.76:0/1951541002 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00b3be0 con 0x562d9568f5e0 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 -- 172.21.15.76:0/1951541002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 msgr2=0x562d956a43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a43a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009530 tx=0x7f31b00993c0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 -- 172.21.15.76:0/1951541002 shutdown_connections 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d95698cd0 0x562d956965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9568ff00 0x562d956b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 --2- 172.21.15.76:0/1951541002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a43a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 -- 172.21.15.76:0/1951541002 >> 172.21.15.76:0/1951541002 conn(0x562d95689f00 msgr2=0x562d95688ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 -- 172.21.15.76:0/1951541002 shutdown_connections 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.217+0000 7f31c579a740 1 -- 172.21.15.76:0/1951541002 wait complete. 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 -- start start 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 Processor -- start 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568ff00 0x562d9570a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d95698cd0 0x562d9570a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562d9570abe0 con 0x562d9568ff00 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d95707610 con 0x562d95698cd0 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562d957077d0 con 0x562d9568f5e0 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c579a740 10 client.0 _ll_register_callbacks cb 0x7f31b009a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31bbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568ff00 0x562d9570a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31bbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568ff00 0x562d9570a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50714/0 (socket says 172.21.15.76:50714) 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31bbfff640 1 -- 172.21.15.76:0/31253346 learned_addr learned my addr 172.21.15.76:0/31253346 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c0951640 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.218+0000 7f31c1152640 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d95698cd0 0x562d9570a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31bbfff640 1 -- 172.21.15.76:0/31253346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 msgr2=0x562d956a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31bbfff640 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31bbfff640 1 -- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d95698cd0 msgr2=0x562d9570a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31bbfff640 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d95698cd0 0x562d9570a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31bbfff640 1 -- 172.21.15.76:0/31253346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d955f4310 con 0x562d9568ff00 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31bbfff640 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568ff00 0x562d9570a160 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f31a8005fd0 tx=0x7f31a8004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37d141ea376d5bd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f319bfff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f319bfff640 1 -- 172.21.15.76:0/31253346 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a802b070 con 0x562d9568ff00 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f319bfff640 1 -- 172.21.15.76:0/31253346 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f31a8004270 con 0x562d9568ff00 2024-03-23T15:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a8004470 con 0x562d9568ff00 2024-03-23T15:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f319bfff640 1 -- 172.21.15.76:0/31253346 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f31a80047c0 con 0x562d9568ff00 2024-03-23T15:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31c579a740 10 client.16095 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d955b65a0 con 0x562d9568ff00 2024-03-23T15:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.219+0000 7f319bfff640 1 -- 172.21.15.76:0/31253346 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a803c080 con 0x562d9568ff00 2024-03-23T15:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f319bfff640 1 -- 172.21.15.76:0/31253346 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f31a8036030 con 0x562d9568ff00 2024-03-23T15:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c579a740 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c579a740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c579a740 20 client.16095 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c579a740 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d95713c60 0x562d95716080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c579a740 4 client.16095 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d953461b0 con 0x562d95713c60 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c0951640 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d95713c60 0x562d95716080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f319bfff640 10 client.16095 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.220+0000 7f31c0951640 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d95713c60 0x562d95716080 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db4aa1a2946324a server_cookie=c2ffbf603cdeca42 in_seq=0 out_seq=0 2024-03-23T15:46:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f319bfff640 1 -- 172.21.15.76:0/31253346 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d953461b0 con 0x562d95713c60 2024-03-23T15:46:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f319bfff640 10 client.16095 handle_command_reply: tid=0 2024-03-23T15:46:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 client.16095 shutdown 2024-03-23T15:46:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568ff00 msgr2=0x562d9570a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568ff00 0x562d9570a160 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f31a8005fd0 tx=0x7f31a8004090 comp rx=0 tx=0).stop 2024-03-23T15:46:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 shutdown_connections 2024-03-23T15:46:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x562d95713c60 0x562d95716080 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d95698cd0 0x562d9570a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562d9568ff00 0x562d9570a160 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f319bfff640 0 client.16095 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 --2- 172.21.15.76:0/31253346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562d9568f5e0 0x562d956a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 >> 172.21.15.76:0/31253346 conn(0x562d95689f00 msgr2=0x562d9568b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 shutdown_connections 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 1 -- 172.21.15.76:0/31253346 wait complete. 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.587+0000 7f31c579a740 20 client.16095 trim_cache size 0 max 16384 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:46:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:46:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f31ac0a22d0 msgr2=0x7f31ac0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f31ac0a22d0 0x7f31ac0a4790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006010 tx=0x7f31b4033ad0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc100860 msgr2=0x7f31bc19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc100860 0x7f31bc19da50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f31b000e9c0 tx=0x7f31b000ee80 comp rx=0 tx=0).stop 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 shutdown_connections 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f31ac0a22d0 0x7f31ac0a4790 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31bc1055b0 0x7f31bc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc101220 0x7f31bc19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 --2- 172.21.15.76:0/211605048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31bc100860 0x7f31bc19da50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 >> 172.21.15.76:0/211605048 conn(0x7f31bc077460 msgr2=0x7f31bc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 shutdown_connections 2024-03-23T15:46:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:35.588+0000 7f31c2bb5640 1 -- 172.21.15.76:0/211605048 wait complete. 2024-03-23T15:46:35.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:35 smithi192 ceph-mon[36653]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 642 KiB/s wr, 187 op/s 2024-03-23T15:46:35.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:35 smithi076 ceph-mon[32176]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 642 KiB/s wr, 187 op/s 2024-03-23T15:46:35.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:35 smithi183 ceph-mon[36296]: pgmap v100: 97 pgs: 97 active+clean; 1.9 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 642 KiB/s wr, 187 op/s 2024-03-23T15:46:37.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:37 smithi192 ceph-mon[36653]: pgmap v101: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.0 MiB/s wr, 337 op/s 2024-03-23T15:46:37.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:37 smithi076 ceph-mon[32176]: pgmap v101: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.0 MiB/s wr, 337 op/s 2024-03-23T15:46:37.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:37 smithi183 ceph-mon[36296]: pgmap v101: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.0 MiB/s wr, 337 op/s 2024-03-23T15:46:39.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:39 smithi192 ceph-mon[36653]: pgmap v102: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 682 KiB/s wr, 208 op/s 2024-03-23T15:46:39.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:39 smithi076 ceph-mon[32176]: pgmap v102: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 682 KiB/s wr, 208 op/s 2024-03-23T15:46:39.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:39 smithi183 ceph-mon[36296]: pgmap v102: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 682 KiB/s wr, 208 op/s 2024-03-23T15:46:41.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:41 smithi192 ceph-mon[36653]: pgmap v103: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 682 KiB/s wr, 208 op/s 2024-03-23T15:46:41.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:41 smithi076 ceph-mon[32176]: pgmap v103: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 682 KiB/s wr, 208 op/s 2024-03-23T15:46:41.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:41 smithi183 ceph-mon[36296]: pgmap v103: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 682 KiB/s wr, 208 op/s 2024-03-23T15:46:43.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:43 smithi192 ceph-mon[36653]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-03-23T15:46:44.583 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:43 smithi183 ceph-mon[36296]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-03-23T15:46:44.584 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:43 smithi076 ceph-mon[32176]: pgmap v104: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-03-23T15:46:45.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:45 smithi192 ceph-mon[36653]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 856 KiB/s wr, 238 op/s 2024-03-23T15:46:45.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:45 smithi076 ceph-mon[32176]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 856 KiB/s wr, 238 op/s 2024-03-23T15:46:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:45 smithi183 ceph-mon[36296]: pgmap v105: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 856 KiB/s wr, 238 op/s 2024-03-23T15:46:47.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:47 smithi192 ceph-mon[36653]: pgmap v106: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-03-23T15:46:47.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:47 smithi076 ceph-mon[32176]: pgmap v106: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-03-23T15:46:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:47 smithi183 ceph-mon[36296]: pgmap v106: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-03-23T15:46:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 -- 172.21.15.76:0/2676897870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c107f70 msgr2=0x7faa3c108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 --2- 172.21.15.76:0/2676897870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c107f70 0x7faa3c108350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa38009900 tx=0x7faa3802bc30 comp rx=0 tx=0).stop 2024-03-23T15:46:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 -- 172.21.15.76:0/2676897870 shutdown_connections 2024-03-23T15:46:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 --2- 172.21.15.76:0/2676897870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3c10d040 0x7faa3c114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 --2- 172.21.15.76:0/2676897870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa3c108920 0x7faa3c10c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 --2- 172.21.15.76:0/2676897870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c107f70 0x7faa3c108350 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 -- 172.21.15.76:0/2676897870 >> 172.21.15.76:0/2676897870 conn(0x7faa3c077440 msgr2=0x7faa3c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.687+0000 7faa44483640 1 -- 172.21.15.76:0/2676897870 shutdown_connections 2024-03-23T15:46:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.688+0000 7faa44483640 1 -- 172.21.15.76:0/2676897870 wait complete. 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.688+0000 7faa44483640 1 Processor -- start 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.688+0000 7faa44483640 1 -- start start 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa44483640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3c107f70 0x7faa3c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa44483640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa3c10d040 0x7faa3c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa44483640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c1a2b00 0x7faa3c1a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa44483640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1a34f0 con 0x7faa3c10d040 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa44483640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1a3660 con 0x7faa3c107f70 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa44483640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1a37d0 con 0x7faa3c1a2b00 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa41a1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa3c10d040 0x7faa3c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa42a20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c1a2b00 0x7faa3c1a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa42a20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c1a2b00 0x7faa3c1a2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:37000/0 (socket says 172.21.15.76:37000) 2024-03-23T15:46:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa42a20640 1 -- 172.21.15.76:0/23442454 learned_addr learned my addr 172.21.15.76:0/23442454 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa41a1e640 1 -- 172.21.15.76:0/23442454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c1a2b00 msgr2=0x7faa3c1a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa41a1e640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c1a2b00 0x7faa3c1a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa41a1e640 1 -- 172.21.15.76:0/23442454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3c107f70 msgr2=0x7faa3c19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.689+0000 7faa41a1e640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3c107f70 0x7faa3c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.690+0000 7faa41a1e640 1 -- 172.21.15.76:0/23442454 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa38009560 con 0x7faa3c10d040 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.690+0000 7faa41a1e640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa3c10d040 0x7faa3c19e7d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00e9c0 tx=0x7faa2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.690+0000 7faa337fe640 1 -- 172.21.15.76:0/23442454 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00ccc0 con 0x7faa3c10d040 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.690+0000 7faa337fe640 1 -- 172.21.15.76:0/23442454 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2c00ce20 con 0x7faa3c10d040 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.690+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3c1aadb0 con 0x7faa3c10d040 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.690+0000 7faa337fe640 1 -- 172.21.15.76:0/23442454 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c018880 con 0x7faa3c10d040 2024-03-23T15:46:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.690+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c1ab250 con 0x7faa3c10d040 2024-03-23T15:46:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.693+0000 7faa337fe640 1 -- 172.21.15.76:0/23442454 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7faa2c0189e0 con 0x7faa3c10d040 2024-03-23T15:46:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.694+0000 7faa4705b740 1 Processor -- start 2024-03-23T15:46:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.694+0000 7faa4705b740 1 -- start start 2024-03-23T15:46:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4705b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4705b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578aff00 0x55f7578b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4705b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 0x55f7578bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4705b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f75786a1b0 con 0x55f7578aff00 2024-03-23T15:46:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4705b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f75781cf80 con 0x55f7578b8cd0 2024-03-23T15:46:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4705b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f75781a4b0 con 0x55f7578af5e0 2024-03-23T15:46:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4221f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578aff00 0x55f7578b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa42a20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa42a20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:37010/0 (socket says 172.21.15.76:37010) 2024-03-23T15:46:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4221f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578aff00 0x55f7578b8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56246/0 (socket says 172.21.15.76:56246) 2024-03-23T15:46:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa42a20640 1 -- 172.21.15.76:0/1018362931 learned_addr learned my addr 172.21.15.76:0/1018362931 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4221f640 1 -- 172.21.15.76:0/1018362931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 msgr2=0x55f7578af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4221f640 1 --2- 172.21.15.76:0/1018362931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa337fe640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7faa280a2350 0x7faa280a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4221f640 1 -- 172.21.15.76:0/1018362931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 msgr2=0x55f7578bb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4221f640 1 --2- 172.21.15.76:0/1018362931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 0x55f7578bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.695+0000 7faa4221f640 1 -- 172.21.15.76:0/1018362931 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7577ab810 con 0x55f7578aff00 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.696+0000 7faa337fe640 1 -- 172.21.15.76:0/23442454 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7faa2c014070 con 0x7faa3c10d040 2024-03-23T15:46:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.696+0000 7faa42a20640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7faa280a2350 0x7faa280a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.696+0000 7faa4221f640 1 --2- 172.21.15.76:0/1018362931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578aff00 0x55f7578b8790 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faa38009230 tx=0x7faa3802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df6a8fd21c85b3b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.696+0000 7faa30bfe640 1 -- 172.21.15.76:0/1018362931 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3802dea0 con 0x55f7578aff00 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.696+0000 7faa30bfe640 1 -- 172.21.15.76:0/1018362931 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3802c810 con 0x55f7578aff00 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.696+0000 7faa30bfe640 1 -- 172.21.15.76:0/1018362931 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38004df0 con 0x55f7578aff00 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.697+0000 7faa4705b740 1 -- 172.21.15.76:0/1018362931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578aff00 msgr2=0x55f7578b8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.697+0000 7faa4705b740 1 --2- 172.21.15.76:0/1018362931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578aff00 0x55f7578b8790 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faa38009230 tx=0x7faa3802c5d0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.698+0000 7faa42a20640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7faa280a2350 0x7faa280a4810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa34011ac0 tx=0x7faa34004ab0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.698+0000 7faa4705b740 1 -- 172.21.15.76:0/1018362931 shutdown_connections 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.698+0000 7faa4705b740 1 --2- 172.21.15.76:0/1018362931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 0x55f7578bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.698+0000 7faa4705b740 1 --2- 172.21.15.76:0/1018362931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578aff00 0x55f7578b8790 secure :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faa38009230 tx=0x7faa3802c5d0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.698+0000 7faa4705b740 1 --2- 172.21.15.76:0/1018362931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.698+0000 7faa4705b740 1 -- 172.21.15.76:0/1018362931 >> 172.21.15.76:0/1018362931 conn(0x55f7578a9f00 msgr2=0x55f7578aa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.698+0000 7faa4705b740 1 -- 172.21.15.76:0/1018362931 shutdown_connections 2024-03-23T15:46:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.699+0000 7faa4705b740 1 -- 172.21.15.76:0/1018362931 wait complete. 2024-03-23T15:46:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.699+0000 7faa4705b740 1 -- start start 2024-03-23T15:46:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.699+0000 7faa4705b740 1 Processor -- start 2024-03-23T15:46:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa4705b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578af5e0 0x55f7578b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa4705b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa4705b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578b68f0 0x55f7578c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa4705b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f757815c90 con 0x55f7578b8cd0 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa4705b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7577bac50 con 0x55f7578af5e0 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa4705b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7574e9250 con 0x55f7578b68f0 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa4705b740 10 client.0 _ll_register_callbacks cb 0x55f757809290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa42a20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578b68f0 0x55f7578c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa42a20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578b68f0 0x55f7578c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:37016/0 (socket says 172.21.15.76:37016) 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa42a20640 1 -- 172.21.15.76:0/2192003423 learned_addr learned my addr 172.21.15.76:0/2192003423 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.700+0000 7faa41a1e640 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa42a20640 1 -- 172.21.15.76:0/2192003423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578af5e0 msgr2=0x55f7578b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa42a20640 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578af5e0 0x55f7578b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa42a20640 1 -- 172.21.15.76:0/2192003423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 msgr2=0x55f7578b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa42a20640 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa42a20640 1 -- 172.21.15.76:0/2192003423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c115420 con 0x55f7578b68f0 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa42a20640 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578b68f0 0x55f7578c49d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa3400fe70 tx=0x7faa34004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a774ee7ebdda6506 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-03-23T15:46:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa25ffb640 1 -- 172.21.15.76:0/2192003423 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34029410 con 0x55f7578b68f0 2024-03-23T15:46:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa25ffb640 1 -- 172.21.15.76:0/2192003423 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7faa34029570 con 0x55f7578b68f0 2024-03-23T15:46:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f75783a790 con 0x55f7578b68f0 2024-03-23T15:46:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa25ffb640 1 -- 172.21.15.76:0/2192003423 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa34029730 con 0x55f7578b68f0 2024-03-23T15:46:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa4705b740 10 client.25888 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.701+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f757814310 con 0x55f7578b68f0 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa25ffb640 1 -- 172.21.15.76:0/2192003423 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa34036020 con 0x55f7578b68f0 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa25ffb640 1 -- 172.21.15.76:0/2192003423 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7faa3401e910 con 0x55f7578b68f0 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa4705b740 10 client.25888 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa4705b740 10 client.25888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa4705b740 20 client.25888 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa4705b740 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7578c9250 0x55f7578cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa4705b740 4 client.25888 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:46:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.702+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f757515600 con 0x55f7578c9250 2024-03-23T15:46:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.703+0000 7faa4221f640 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7578c9250 0x55f7578cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.704+0000 7faa4221f640 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7578c9250 0x55f7578cb6b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dcf007c2b1750b9 server_cookie=e3ebb91881dec682 in_seq=0 out_seq=0 2024-03-23T15:46:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.704+0000 7faa25ffb640 10 client.25888 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.776+0000 7faa25ffb640 1 -- 172.21.15.76:0/2192003423 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f757515600 con 0x55f7578c9250 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.776+0000 7faa25ffb640 10 client.25888 handle_command_reply: tid=0 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.776+0000 7faa4705b740 1 client.25888 shutdown 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578b68f0 msgr2=0x55f7578c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578b68f0 0x55f7578c49d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faa3400fe70 tx=0x7faa34004a40 comp rx=0 tx=0).stop 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 shutdown_connections 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578b68f0 0x55f7578c49d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f7578c9250 0x55f7578cb6b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 --2- 172.21.15.76:0/2192003423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578af5e0 0x55f7578b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa25ffb640 0 client.25888 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 >> 172.21.15.76:0/2192003423 conn(0x55f7578a9f00 msgr2=0x55f7578abc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 shutdown_connections 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 1 -- 172.21.15.76:0/2192003423 wait complete. 2024-03-23T15:46:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.777+0000 7faa4705b740 20 client.25888 trim_cache size 0 max 16384 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 Processor -- start 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 -- start start 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b68f0 0x55f7578d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f75779b660 con 0x55f7578b8cd0 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7578afd40 con 0x55f7578b68f0 2024-03-23T15:46:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4705b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7578b43c0 con 0x55f7578af5e0 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56264/0 (socket says 172.21.15.76:56264) 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa4221f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 -- 172.21.15.76:0/3053109160 learned_addr learned my addr 172.21.15.76:0/3053109160 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa41a1e640 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b68f0 0x55f7578d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 -- 172.21.15.76:0/3053109160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 msgr2=0x55f7578aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 -- 172.21.15.76:0/3053109160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b68f0 msgr2=0x55f7578d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b68f0 0x55f7578d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 -- 172.21.15.76:0/3053109160 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7577b0810 con 0x55f7578b8cd0 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.792+0000 7faa42a20640 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b4740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faa340049f0 tx=0x7faa34009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282f02d3ac09b827 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa27fff640 1 -- 172.21.15.76:0/3053109160 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3401e5f0 con 0x55f7578b8cd0 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa27fff640 1 -- 172.21.15.76:0/3053109160 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa34036030 con 0x55f7578b8cd0 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa27fff640 1 -- 172.21.15.76:0/3053109160 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34029e00 con 0x55f7578b8cd0 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 -- 172.21.15.76:0/3053109160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 msgr2=0x55f7578b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b4740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faa340049f0 tx=0x7faa34009de0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 -- 172.21.15.76:0/3053109160 shutdown_connections 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b8cd0 0x55f7578b4740 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b68f0 0x55f7578d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 --2- 172.21.15.76:0/3053109160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 -- 172.21.15.76:0/3053109160 >> 172.21.15.76:0/3053109160 conn(0x55f7578a9f00 msgr2=0x55f7578a4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 -- 172.21.15.76:0/3053109160 shutdown_connections 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.793+0000 7faa4705b740 1 -- 172.21.15.76:0/3053109160 wait complete. 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 -- start start 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 Processor -- start 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b68f0 0x55f7578bbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 0x55f7578bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7578bcb40 con 0x55f7578b68f0 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7578bccb0 con 0x55f7578b8cd0 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f75792e5f0 con 0x55f7578af5e0 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4705b740 10 client.0 _ll_register_callbacks cb 0x7faa3400fd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b68f0 0x55f7578bbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b68f0 0x55f7578bbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56280/0 (socket says 172.21.15.76:56280) 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 -- 172.21.15.76:0/2730721507 learned_addr learned my addr 172.21.15.76:0/2730721507 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4221f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa42a20640 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 0x55f7578bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 -- 172.21.15.76:0/2730721507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 msgr2=0x55f7578c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 -- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 msgr2=0x55f7578bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 0x55f7578bc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa41a1e640 1 -- 172.21.15.76:0/2730721507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f757814310 con 0x55f7578b68f0 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.794+0000 7faa4221f640 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:46:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa41a1e640 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b68f0 0x55f7578bbf30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0cbd10 tx=0x7faa2c09b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76431d208784a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa25ffb640 1 -- 172.21.15.76:0/2730721507 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c09f050 con 0x55f7578b68f0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa25ffb640 1 -- 172.21.15.76:0/2730721507 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7faa2c0a36a0 con 0x55f7578b68f0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa3802c810 con 0x55f7578b68f0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa25ffb640 1 -- 172.21.15.76:0/2730721507 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa2c09a350 con 0x55f7578b68f0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 10 client.16113 fetch_fsmap learned FSMap version 27 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7577d65a0 con 0x55f7578b68f0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa25ffb640 1 -- 172.21.15.76:0/2730721507 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3802c810 con 0x55f7578b68f0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa25ffb640 1 -- 172.21.15.76:0/2730721507 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7faa2c079d20 con 0x55f7578b68f0 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 10 client.16113 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 20 client.16113 populate_metadata read hostname 'smithi076' 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f75792fd60 0x55f757932180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:46:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 4 client.16113 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:46:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7575661b0 con 0x55f75792fd60 2024-03-23T15:46:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.795+0000 7faa4221f640 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f75792fd60 0x55f757932180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:46:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.796+0000 7faa25ffb640 10 client.16113 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:48.796+0000 7faa4221f640 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f75792fd60 0x55f757932180 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af07c87f9421f7 server_cookie=1f3434933bff0ecc in_seq=0 out_seq=0 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa25ffb640 1 -- 172.21.15.76:0/2730721507 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7575661b0 con 0x55f75792fd60 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa25ffb640 10 client.16113 handle_command_reply: tid=0 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 client.16113 shutdown 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b68f0 msgr2=0x55f7578bbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b68f0 0x55f7578bbf30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0cbd10 tx=0x7faa2c09b600 comp rx=0 tx=0).stop 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 shutdown_connections 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55f75792fd60 0x55f757932180 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7578b8cd0 0x55f7578bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa25ffb640 0 client.16113 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7578b68f0 0x55f7578bbf30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 --2- 172.21.15.76:0/2730721507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7578af5e0 0x55f7578c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 >> 172.21.15.76:0/2730721507 conn(0x55f7578a9f00 msgr2=0x55f7578aa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 shutdown_connections 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 1 -- 172.21.15.76:0/2730721507 wait complete. 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.216+0000 7faa4705b740 20 client.16113 trim_cache size 0 max 16384 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:46:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:46:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:46:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7faa280a2350 msgr2=0x7faa280a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7faa280a2350 0x7faa280a4810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa34011ac0 tx=0x7faa34004ab0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa3c10d040 msgr2=0x7faa3c19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa3c10d040 0x7faa3c19e7d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00e9c0 tx=0x7faa2c00ee80 comp rx=0 tx=0).stop 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 shutdown_connections 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7faa280a2350 0x7faa280a4810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa3c1a2b00 0x7faa3c1a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa3c10d040 0x7faa3c19e7d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 --2- 172.21.15.76:0/23442454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa3c107f70 0x7faa3c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 >> 172.21.15.76:0/23442454 conn(0x7faa3c077440 msgr2=0x7faa3c077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 shutdown_connections 2024-03-23T15:46:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:46:49.217+0000 7faa44483640 1 -- 172.21.15.76:0/23442454 wait complete. 2024-03-23T15:46:49.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:49 smithi192 ceph-mon[36653]: pgmap v107: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 685 KiB/s wr, 132 op/s 2024-03-23T15:46:49.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:49 smithi076 ceph-mon[32176]: pgmap v107: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 685 KiB/s wr, 132 op/s 2024-03-23T15:46:49.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:49 smithi183 ceph-mon[36296]: pgmap v107: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 685 KiB/s wr, 132 op/s 2024-03-23T15:46:51.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:51 smithi192 ceph-mon[36653]: pgmap v108: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 685 KiB/s wr, 132 op/s 2024-03-23T15:46:51.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:51 smithi076 ceph-mon[32176]: pgmap v108: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 685 KiB/s wr, 132 op/s 2024-03-23T15:46:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:51 smithi183 ceph-mon[36296]: pgmap v108: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 685 KiB/s wr, 132 op/s 2024-03-23T15:46:53.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:53 smithi192 ceph-mon[36653]: pgmap v109: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 283 op/s 2024-03-23T15:46:53.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:53 smithi076 ceph-mon[32176]: pgmap v109: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 283 op/s 2024-03-23T15:46:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:53 smithi183 ceph-mon[36296]: pgmap v109: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 283 op/s 2024-03-23T15:46:55.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:55 smithi192 ceph-mon[36653]: pgmap v110: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 765 KiB/s wr, 194 op/s 2024-03-23T15:46:55.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:55 smithi076 ceph-mon[32176]: pgmap v110: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 765 KiB/s wr, 194 op/s 2024-03-23T15:46:55.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:55 smithi183 ceph-mon[36296]: pgmap v110: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 765 KiB/s wr, 194 op/s 2024-03-23T15:46:57.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:57 smithi192 ceph-mon[36653]: pgmap v111: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 285 op/s 2024-03-23T15:46:57.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:57 smithi076 ceph-mon[32176]: pgmap v111: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 285 op/s 2024-03-23T15:46:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:57 smithi183 ceph-mon[36296]: pgmap v111: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 285 op/s 2024-03-23T15:46:59.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:46:59 smithi192 ceph-mon[36653]: pgmap v112: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-03-23T15:46:59.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:46:59 smithi076 ceph-mon[32176]: pgmap v112: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-03-23T15:46:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:46:59 smithi183 ceph-mon[36296]: pgmap v112: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-03-23T15:47:01.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:01 smithi192 ceph-mon[36653]: pgmap v113: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-03-23T15:47:01.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:01 smithi076 ceph-mon[32176]: pgmap v113: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-03-23T15:47:01.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:01 smithi183 ceph-mon[36296]: pgmap v113: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-03-23T15:47:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 -- 172.21.15.76:0/627288416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801016e0 msgr2=0x7f8680101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 --2- 172.21.15.76:0/627288416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801016e0 0x7f8680101b60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8670009930 tx=0x7f867002bc30 comp rx=0 tx=0).stop 2024-03-23T15:47:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 -- 172.21.15.76:0/627288416 shutdown_connections 2024-03-23T15:47:02.308 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 --2- 172.21.15.76:0/627288416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86801020a0 0x7f868010e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 --2- 172.21.15.76:0/627288416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801016e0 0x7f8680101b60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 --2- 172.21.15.76:0/627288416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86801076f0 0x7f8680107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 -- 172.21.15.76:0/627288416 >> 172.21.15.76:0/627288416 conn(0x7f86800fd210 msgr2=0x7f86800ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 -- 172.21.15.76:0/627288416 shutdown_connections 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 -- 172.21.15.76:0/627288416 wait complete. 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 Processor -- start 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.303+0000 7f8687d5e640 1 -- start start 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86801016e0 0x7f868019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801020a0 0x7f868019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86801076f0 0x7f8680197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8680198130 con 0x7f86801020a0 2024-03-23T15:47:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86801982a0 con 0x7f86801016e0 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8680198460 con 0x7f86801076f0 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801020a0 0x7f868019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801020a0 0x7f868019de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57278/0 (socket says 172.21.15.76:57278) 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 -- 172.21.15.76:0/1890739399 learned_addr learned my addr 172.21.15.76:0/1890739399 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8685afa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86801016e0 0x7f868019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86862fb640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86801076f0 0x7f8680197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 -- 172.21.15.76:0/1890739399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86801076f0 msgr2=0x7f8680197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86801076f0 0x7f8680197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 -- 172.21.15.76:0/1890739399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86801016e0 msgr2=0x7f868019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86801016e0 0x7f868019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 -- 172.21.15.76:0/1890739399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8670009560 con 0x7f86801020a0 2024-03-23T15:47:02.310 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f86852f9640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801020a0 0x7f868019de70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8670009230 tx=0x7f867002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8676ffd640 1 -- 172.21.15.76:0/1890739399 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670004350 con 0x7f86801020a0 2024-03-23T15:47:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86801986e0 con 0x7f86801020a0 2024-03-23T15:47:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8676ffd640 1 -- 172.21.15.76:0/1890739399 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86700044b0 con 0x7f86801020a0 2024-03-23T15:47:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8676ffd640 1 -- 172.21.15.76:0/1890739399 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670003d00 con 0x7f86801020a0 2024-03-23T15:47:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.304+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86801a49d0 con 0x7f86801020a0 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.306+0000 7f8676ffd640 1 -- 172.21.15.76:0/1890739399 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f8670004df0 con 0x7f86801020a0 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 Processor -- start 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 -- start start 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f8676ffd640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f866c0a22d0 0x7f866c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a9c60891b0 con 0x55a9c60ce5e0 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9c603bf80 con 0x55a9c60d7cd0 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.307+0000 7f868aa1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a9c60394b0 con 0x55a9c60cef00 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f86852f9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57294/0 (socket says 172.21.15.76:57294) 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f86862fb640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f866c0a22d0 0x7f866c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 -- 172.21.15.76:0/3979082515 learned_addr learned my addr 172.21.15.76:0/3979082515 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8676ffd640 1 -- 172.21.15.76:0/1890739399 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f86700eff50 con 0x7f86801020a0 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f86862fb640 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 -- 172.21.15.76:0/3979082515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 msgr2=0x55a9c60d7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 -- 172.21.15.76:0/3979082515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 msgr2=0x55a9c60da0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60da0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 -- 172.21.15.76:0/3979082515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c5fca810 con 0x55a9c60ce5e0 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f8685afa640 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60ce9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f867c009c70 tx=0x7f867c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ddb698b08989ef server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f86862fb640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f866c0a22d0 0x7f866c0a4790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8678005fd0 tx=0x7f8678009f10 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f866bfff640 1 -- 172.21.15.76:0/3979082515 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c004230 con 0x55a9c60ce5e0 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f866bfff640 1 -- 172.21.15.76:0/3979082515 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c004390 con 0x55a9c60ce5e0 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.308+0000 7f866bfff640 1 -- 172.21.15.76:0/3979082515 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0196d0 con 0x55a9c60ce5e0 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 -- 172.21.15.76:0/3979082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 msgr2=0x55a9c60ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60ce9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f867c009c70 tx=0x7f867c011810 comp rx=0 tx=0).stop 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 -- 172.21.15.76:0/3979082515 shutdown_connections 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60da0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3979082515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60ce9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 -- 172.21.15.76:0/3979082515 >> 172.21.15.76:0/3979082515 conn(0x55a9c60c8f00 msgr2=0x55a9c60c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 -- 172.21.15.76:0/3979082515 shutdown_connections 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 -- 172.21.15.76:0/3979082515 wait complete. 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.309+0000 7f868aa1e740 1 -- start start 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 1 Processor -- start 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a9c6034c90 con 0x55a9c60d7cd0 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9c5fd9c50 con 0x55a9c60ce5e0 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a9c5d08250 con 0x55a9c60cef00 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f868aa1e740 10 client.0 _ll_register_callbacks cb 0x55a9c6028290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f8685afa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57308/0 (socket says 172.21.15.76:57308) 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 -- 172.21.15.76:0/3264995853 learned_addr learned my addr 172.21.15.76:0/3264995853 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86852f9640 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 -- 172.21.15.76:0/3264995853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 msgr2=0x55a9c60d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 -- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 msgr2=0x55a9c60d4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f86862fb640 1 -- 172.21.15.76:0/3264995853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c0098d0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.310+0000 7f8685afa640 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f86862fb640 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f867801e550 tx=0x7f867801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b9c3f7acfaf503 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f86697fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f86697fa640 1 -- 172.21.15.76:0/3264995853 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678016040 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f86697fa640 1 -- 172.21.15.76:0/3264995853 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f86780206c0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9c6059790 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f86697fa640 1 -- 172.21.15.76:0/3264995853 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8678004b20 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 10 client.16125 fetch_fsmap learned FSMap version 27 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9c6033310 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f86697fa640 1 -- 172.21.15.76:0/3264995853 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a9c6059790 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f86697fa640 1 -- 172.21.15.76:0/3264995853 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8678030070 con 0x55a9c60d7cd0 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 10 client.16125 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:47:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:47:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 20 client.16125 populate_metadata read hostname 'smithi076' 2024-03-23T15:47:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c60e7cc0 0x55a9c60ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 4 client.16125 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:47:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9c5d34600 con 0x55a9c60e7cc0 2024-03-23T15:47:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.311+0000 7f8685afa640 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c60e7cc0 0x55a9c60ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.312+0000 7f86697fa640 10 client.16125 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.312+0000 7f8685afa640 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c60e7cc0 0x55a9c60ea0e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74ec5ecdc0d891ba server_cookie=f3f3efa1d826f733 in_seq=0 out_seq=0 2024-03-23T15:47:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f86697fa640 1 -- 172.21.15.76:0/3264995853 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a9c5d34600 con 0x55a9c60e7cc0 2024-03-23T15:47:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f86697fa640 10 client.16125 handle_command_reply: tid=0 2024-03-23T15:47:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 client.16125 shutdown 2024-03-23T15:47:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 msgr2=0x55a9c60d5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f867801e550 tx=0x7f867801d000 comp rx=0 tx=0).stop 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 shutdown_connections 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5710 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c60e7cc0 0x55a9c60ea0e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 --2- 172.21.15.76:0/3264995853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f86697fa640 0 client.16125 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 >> 172.21.15.76:0/3264995853 conn(0x55a9c60c8f00 msgr2=0x55a9c60cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 shutdown_connections 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 1 -- 172.21.15.76:0/3264995853 wait complete. 2024-03-23T15:47:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.358+0000 7f868aa1e740 20 client.16125 trim_cache size 0 max 16384 2024-03-23T15:47:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 Processor -- start 2024-03-23T15:47:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 -- start start 2024-03-23T15:47:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60dfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60e33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a9c5fba660 con 0x55a9c60ce5e0 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9c60c73f0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f868aa1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a9c60e45f0 con 0x55a9c60cef00 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60dfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60dfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57322/0 (socket says 172.21.15.76:57322) 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 -- 172.21.15.76:0/343283435 learned_addr learned my addr 172.21.15.76:0/343283435 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f86852f9640 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60e33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f86862fb640 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 -- 172.21.15.76:0/343283435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 msgr2=0x55a9c60e33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60e33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 -- 172.21.15.76:0/343283435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 msgr2=0x55a9c60d5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 -- 172.21.15.76:0/343283435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9c5fcf810 con 0x55a9c60ce5e0 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f86852f9640 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60e33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.374+0000 7f8685afa640 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60dfdb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f867c004360 tx=0x7f867c0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ef599f340f09b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f866b7fe640 1 -- 172.21.15.76:0/343283435 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c009e20 con 0x55a9c60ce5e0 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f866b7fe640 1 -- 172.21.15.76:0/343283435 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c003ad0 con 0x55a9c60ce5e0 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f866b7fe640 1 -- 172.21.15.76:0/343283435 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0047e0 con 0x55a9c60ce5e0 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 -- 172.21.15.76:0/343283435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 msgr2=0x55a9c60dfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60dfdb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f867c004360 tx=0x7f867c0097e0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 -- 172.21.15.76:0/343283435 shutdown_connections 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60e33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 --2- 172.21.15.76:0/343283435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60ce5e0 0x55a9c60dfdb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 -- 172.21.15.76:0/343283435 >> 172.21.15.76:0/343283435 conn(0x55a9c60c8f00 msgr2=0x55a9c60c7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 -- 172.21.15.76:0/343283435 shutdown_connections 2024-03-23T15:47:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.375+0000 7f868aa1e740 1 -- 172.21.15.76:0/343283435 wait complete. 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 -- start start 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 Processor -- start 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a9c60d2070 con 0x55a9c60d7cd0 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9c60d21e0 con 0x55a9c60ce5e0 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a9c60d23a0 con 0x55a9c60cef00 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f868aa1e740 10 client.0 _ll_register_callbacks cb 0x7f8678009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57332/0 (socket says 172.21.15.76:57332) 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 -- 172.21.15.76:0/4142320374 learned_addr learned my addr 172.21.15.76:0/4142320374 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86852f9640 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f8685afa640 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 -- 172.21.15.76:0/4142320374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 msgr2=0x55a9c60d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 -- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 msgr2=0x55a9c60d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.376+0000 7f86862fb640 1 -- 172.21.15.76:0/4142320374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c01e040 con 0x55a9c60d7cd0 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f86862fb640 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d19a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f867801efd0 tx=0x7f86780118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe0517c208ab640 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f86697fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f86697fa640 1 -- 172.21.15.76:0/4142320374 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678016070 con 0x55a9c60d7cd0 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f86697fa640 1 -- 172.21.15.76:0/4142320374 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8678004bf0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9c5d851b0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f86697fa640 1 -- 172.21.15.76:0/4142320374 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8678004db0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9c5ff55a0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f86697fa640 1 -- 172.21.15.76:0/4142320374 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a9c5d851b0 con 0x55a9c60d7cd0 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f86697fa640 1 -- 172.21.15.76:0/4142320374 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8678038070 con 0x55a9c60d7cd0 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 20 client.16131 populate_metadata read hostname 'smithi076' 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c6152cd0 0x55a9c61550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 4 client.16131 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9c5d71550 con 0x55a9c6152cd0 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.377+0000 7f8685afa640 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c6152cd0 0x55a9c61550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.378+0000 7f86697fa640 10 client.16131 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.378+0000 7f8685afa640 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c6152cd0 0x55a9c61550f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3252c601e740f97f server_cookie=87dc32e3f3b3b8cf in_seq=0 out_seq=0 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f86697fa640 1 -- 172.21.15.76:0/4142320374 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9c5d71550 con 0x55a9c6152cd0 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f86697fa640 10 client.16131 handle_command_reply: tid=0 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 client.16131 shutdown 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 msgr2=0x55a9c60d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d19a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f867801efd0 tx=0x7f86780118d0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 shutdown_connections 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a9c60d7cd0 0x55a9c60d19a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a9c60cef00 0x55a9c60d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55a9c6152cd0 0x55a9c61550f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 --2- 172.21.15.76:0/4142320374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9c60ce5e0 0x55a9c60d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 >> 172.21.15.76:0/4142320374 conn(0x55a9c60c8f00 msgr2=0x55a9c60c7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f86697fa640 0 client.16131 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 shutdown_connections 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.688+0000 7f868aa1e740 1 -- 172.21.15.76:0/4142320374 wait complete. 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f868aa1e740 20 client.16131 trim_cache size 0 max 16384 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:47:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:47:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f866c0a22d0 msgr2=0x7f866c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f866c0a22d0 0x7f866c0a4790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8678005fd0 tx=0x7f8678009f10 comp rx=0 tx=0).stop 2024-03-23T15:47:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801020a0 msgr2=0x7f868019de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801020a0 0x7f868019de70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8670009230 tx=0x7f867002c650 comp rx=0 tx=0).stop 2024-03-23T15:47:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 shutdown_connections 2024-03-23T15:47:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f866c0a22d0 0x7f866c0a4790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86801076f0 0x7f8680197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86801020a0 0x7f868019de70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 --2- 172.21.15.76:0/1890739399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86801016e0 0x7f868019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.689+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 >> 172.21.15.76:0/1890739399 conn(0x7f86800fd210 msgr2=0x7f86800fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.690+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 shutdown_connections 2024-03-23T15:47:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:02.690+0000 7f8687d5e640 1 -- 172.21.15.76:0/1890739399 wait complete. 2024-03-23T15:47:03.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:03 smithi192 ceph-mon[36653]: pgmap v114: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 370 op/s 2024-03-23T15:47:03.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:03 smithi076 ceph-mon[32176]: pgmap v114: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 370 op/s 2024-03-23T15:47:03.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:03 smithi183 ceph-mon[36296]: pgmap v114: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 370 op/s 2024-03-23T15:47:05.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:05 smithi192 ceph-mon[36653]: pgmap v115: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-03-23T15:47:05.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:05 smithi076 ceph-mon[32176]: pgmap v115: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-03-23T15:47:05.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:05 smithi183 ceph-mon[36296]: pgmap v115: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-03-23T15:47:07.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:07 smithi192 ceph-mon[36653]: pgmap v116: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-03-23T15:47:07.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:07 smithi076 ceph-mon[32176]: pgmap v116: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-03-23T15:47:07.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:07 smithi183 ceph-mon[36296]: pgmap v116: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-03-23T15:47:09.822 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:09 smithi192 ceph-mon[36653]: pgmap v117: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-03-23T15:47:09.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:09 smithi076 ceph-mon[32176]: pgmap v117: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-03-23T15:47:09.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:09 smithi183 ceph-mon[36296]: pgmap v117: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-03-23T15:47:11.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:11 smithi192 ceph-mon[36653]: pgmap v118: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-03-23T15:47:11.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:11 smithi076 ceph-mon[32176]: pgmap v118: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-03-23T15:47:11.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:11 smithi183 ceph-mon[36296]: pgmap v118: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 287 op/s 2024-03-23T15:47:13.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:13 smithi192 ceph-mon[36653]: pgmap v119: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 351 op/s 2024-03-23T15:47:13.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:13 smithi076 ceph-mon[32176]: pgmap v119: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 351 op/s 2024-03-23T15:47:13.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:13 smithi183 ceph-mon[36296]: pgmap v119: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 351 op/s 2024-03-23T15:47:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 -- 172.21.15.76:0/2087702537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 msgr2=0x7fe854106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/2087702537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 0x7fe854106a60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe850009900 tx=0x7fe85002bcd0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 -- 172.21.15.76:0/2087702537 shutdown_connections 2024-03-23T15:47:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/2087702537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe854100d10 0x7fe85410d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/2087702537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854106fa0 0x7fe8541005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/2087702537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 0x7fe854106a60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 -- 172.21.15.76:0/2087702537 >> 172.21.15.76:0/2087702537 conn(0x7fe854077740 msgr2=0x7fe854077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 -- 172.21.15.76:0/2087702537 shutdown_connections 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.790+0000 7fe85c5c7640 1 -- 172.21.15.76:0/2087702537 wait complete. 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 Processor -- start 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 -- start start 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854100d10 0x7fe85419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 0x7fe85419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe854106fa0 0x7fe854197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8541980f0 con 0x7fe854106680 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe854198260 con 0x7fe854106fa0 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85c5c7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe854198420 con 0x7fe854100d10 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854100d10 0x7fe85419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854100d10 0x7fe85419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:35786/0 (socket says 172.21.15.76:35786) 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 -- 172.21.15.76:0/1778193823 learned_addr learned my addr 172.21.15.76:0/1778193823 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe859b62640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 0x7fe85419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85ab64640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe854106fa0 0x7fe854197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 -- 172.21.15.76:0/1778193823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe854106fa0 msgr2=0x7fe854197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe854106fa0 0x7fe854197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 -- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 msgr2=0x7fe85419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 0x7fe85419dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe85a363640 1 -- 172.21.15.76:0/1778193823 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe850009560 con 0x7fe854100d10 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.791+0000 7fe859b62640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 0x7fe85419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.792+0000 7fe85a363640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854100d10 0x7fe85419d8a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe850036040 tx=0x7fe85002c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.792+0000 7fe84b7fe640 1 -- 172.21.15.76:0/1778193823 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850004350 con 0x7fe854100d10 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.792+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8541986a0 con 0x7fe854100d10 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.792+0000 7fe84b7fe640 1 -- 172.21.15.76:0/1778193823 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8500044b0 con 0x7fe854100d10 2024-03-23T15:47:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.792+0000 7fe84b7fe640 1 -- 172.21.15.76:0/1778193823 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850003da0 con 0x7fe854100d10 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.792+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541a4940 con 0x7fe854100d10 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.794+0000 7fe84b7fe640 1 -- 172.21.15.76:0/1778193823 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fe850004df0 con 0x7fe854100d10 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.794+0000 7fe85f1a6740 1 Processor -- start 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85f1a6740 1 -- start start 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de64185e0 0x564de64189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de6418f00 0x564de6421790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de64240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe84b7fe640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe8400a22d0 0x7fe8400a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564de63d31b0 con 0x564de6421cd0 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564de6385f80 con 0x564de6418f00 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564de63834b0 con 0x564de64185e0 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe859b62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de64240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85a363640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de6418f00 0x564de6421790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85ab64640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe8400a22d0 0x7fe8400a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe859b62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de64240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56864/0 (socket says 172.21.15.76:56864) 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe859b62640 1 -- 172.21.15.76:0/3175280736 learned_addr learned my addr 172.21.15.76:0/3175280736 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85a363640 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de6418f00 0x564de6421790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:48952/0 (socket says 172.21.15.76:48952) 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85ab64640 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de64185e0 0x564de64189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe84b7fe640 1 -- 172.21.15.76:0/1778193823 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe8500f0660 con 0x7fe854100d10 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85a363640 1 -- 172.21.15.76:0/3175280736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de64185e0 msgr2=0x564de64189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85a363640 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de64185e0 0x564de64189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85a363640 1 -- 172.21.15.76:0/3175280736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 msgr2=0x564de64240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85a363640 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de64240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe85a363640 1 -- 172.21.15.76:0/3175280736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de6314810 con 0x564de6418f00 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.795+0000 7fe859b62640 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de64240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.796+0000 7fe85ab64640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe8400a22d0 0x7fe8400a4790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe84c005fd0 tx=0x7fe84c009f00 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.796+0000 7fe85a363640 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de6418f00 0x564de6421790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8500edec0 tx=0x7fe850088790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf9ed3ccea79ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.796+0000 7fe848bfe640 1 -- 172.21.15.76:0/3175280736 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500bcb50 con 0x564de6418f00 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.796+0000 7fe848bfe640 1 -- 172.21.15.76:0/3175280736 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8500bd8f0 con 0x564de6418f00 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.796+0000 7fe848bfe640 1 -- 172.21.15.76:0/3175280736 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500bb4f0 con 0x564de6418f00 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3175280736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de6418f00 msgr2=0x564de6421790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de6418f00 0x564de6421790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8500edec0 tx=0x7fe850088790 comp rx=0 tx=0).stop 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3175280736 shutdown_connections 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de64240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de6418f00 0x564de6421790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3175280736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de64185e0 0x564de64189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3175280736 >> 172.21.15.76:0/3175280736 conn(0x564de6412f00 msgr2=0x564de64132e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3175280736 shutdown_connections 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3175280736 wait complete. 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- start start 2024-03-23T15:47:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 Processor -- start 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de641eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de641f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564de637ec90 con 0x564de6421cd0 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564de6323c50 con 0x564de64185e0 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564de6052250 con 0x564de6418f00 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85f1a6740 10 client.0 _ll_register_callbacks cb 0x564de6372290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85ab64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85a363640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de641eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe859b62640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de641f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85a363640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de641eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:48962/0 (socket says 172.21.15.76:48962) 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85a363640 1 -- 172.21.15.76:0/2830697572 learned_addr learned my addr 172.21.15.76:0/2830697572 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe85ab64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56874/0 (socket says 172.21.15.76:56874) 2024-03-23T15:47:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.797+0000 7fe859b62640 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de641f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.76:35816/0 (socket says 172.21.15.76:35816) 2024-03-23T15:47:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85a363640 1 -- 172.21.15.76:0/2830697572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 msgr2=0x564de641f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85a363640 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de641f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85a363640 1 -- 172.21.15.76:0/2830697572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 msgr2=0x564de641f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85a363640 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85a363640 1 -- 172.21.15.76:0/2830697572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541a4940 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85ab64640 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85a363640 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de641eba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8500b9bc0 tx=0x7fe85002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996d8ad8e7b996a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe83dffb640 1 -- 172.21.15.76:0/2830697572 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500bd1f0 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564de63a3790 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe83dffb640 1 -- 172.21.15.76:0/2830697572 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe8500c1120 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe83dffb640 1 -- 172.21.15.76:0/2830697572 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8500d42e0 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85f1a6740 10 client.25907 fetch_fsmap learned FSMap version 27 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564de637d310 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.798+0000 7fe83dffb640 1 -- 172.21.15.76:0/2830697572 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564de63a3790 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe83dffb640 1 -- 172.21.15.76:0/2830697572 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe85003c070 con 0x564de64185e0 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe85f1a6740 10 client.25907 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe85f1a6740 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:47:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe85f1a6740 20 client.25907 populate_metadata read hostname 'smithi076' 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de64313d0 0x564de64337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe85f1a6740 4 client.25907 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de607e600 con 0x564de64313d0 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe859b62640 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de64313d0 0x564de64337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe859b62640 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de64313d0 0x564de64337e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f0c03789f44eb3 server_cookie=b5ff4aab3f041ff3 in_seq=0 out_seq=0 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.799+0000 7fe83dffb640 10 client.25907 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe83dffb640 1 -- 172.21.15.76:0/2830697572 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564de607e600 con 0x564de64313d0 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe83dffb640 10 client.25907 handle_command_reply: tid=0 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 client.25907 shutdown 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 msgr2=0x564de641eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de641eba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8500b9bc0 tx=0x7fe85002bcd0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 shutdown_connections 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de641f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de64313d0 0x564de64337e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/2830697572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de641eba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe83dffb640 0 client.25907 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 >> 172.21.15.76:0/2830697572 conn(0x564de6412f00 msgr2=0x564de6414c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 shutdown_connections 2024-03-23T15:47:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 1 -- 172.21.15.76:0/2830697572 wait complete. 2024-03-23T15:47:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.800+0000 7fe85f1a6740 20 client.25907 trim_cache size 0 max 16384 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.815+0000 7fe85f1a6740 1 Processor -- start 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85f1a6740 1 -- start start 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de64212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de642d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564de6304660 con 0x564de6421cd0 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564de6431220 con 0x564de64185e0 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564de607e600 con 0x564de6418f00 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe859b62640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de642d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56878/0 (socket says 172.21.15.76:56878) 2024-03-23T15:47:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 -- 172.21.15.76:0/3566846443 learned_addr learned my addr 172.21.15.76:0/3566846443 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85a363640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de64212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 -- 172.21.15.76:0/3566846443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 msgr2=0x564de642d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de642d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 -- 172.21.15.76:0/3566846443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 msgr2=0x564de64212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de64212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 -- 172.21.15.76:0/3566846443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de6319810 con 0x564de6421cd0 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe859b62640 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de642d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85a363640 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de64212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe85ab64640 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641dd00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009fa0 tx=0x7fe84c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cb13678fa4fa1a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.816+0000 7fe83ffff640 1 -- 172.21.15.76:0/3566846443 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c016050 con 0x564de6421cd0 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe83ffff640 1 -- 172.21.15.76:0/3566846443 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84c003e00 con 0x564de6421cd0 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe83ffff640 1 -- 172.21.15.76:0/3566846443 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c005610 con 0x564de6421cd0 2024-03-23T15:47:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3566846443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 msgr2=0x564de641dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641dd00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009fa0 tx=0x7fe84c021000 comp rx=0 tx=0).stop 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3566846443 shutdown_connections 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6421cd0 0x564de641dd00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6418f00 0x564de642d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/3566846443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de64212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3566846443 >> 172.21.15.76:0/3566846443 conn(0x564de6412f00 msgr2=0x564de64132e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3566846443 shutdown_connections 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 -- 172.21.15.76:0/3566846443 wait complete. 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.817+0000 7fe85f1a6740 1 -- start start 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 1 Processor -- start 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de6429890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6418f00 0x564de641b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6421cd0 0x564de641b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564de641c050 con 0x564de6418f00 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564de641c1c0 con 0x564de64185e0 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564de64966e0 con 0x564de6421cd0 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85f1a6740 10 client.0 _ll_register_callbacks cb 0x7fe850003f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6418f00 0x564de641b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85a363640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de6429890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6418f00 0x564de641b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56894/0 (socket says 172.21.15.76:56894) 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 -- 172.21.15.76:0/964528082 learned_addr learned my addr 172.21.15.76:0/964528082 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85ab64640 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6421cd0 0x564de641b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 -- 172.21.15.76:0/964528082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6421cd0 msgr2=0x564de641b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6421cd0 0x564de641b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 -- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 msgr2=0x564de6429890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de6429890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 -- 172.21.15.76:0/964528082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de637d310 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe85a363640 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de6429890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe859b62640 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6418f00 0x564de641b410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x564de641e400 tx=0x7fe844012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4d6a589fcf191f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.818+0000 7fe83dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe83dffb640 1 -- 172.21.15.76:0/964528082 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844017040 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe83dffb640 1 -- 172.21.15.76:0/964528082 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe844004440 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8500bd8f0 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe83dffb640 1 -- 172.21.15.76:0/964528082 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe844010960 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 10 client.16149 fetch_fsmap learned FSMap version 27 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x7fe844012040 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe83dffb640 1 -- 172.21.15.76:0/964528082 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8500bd8f0 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe83dffb640 1 -- 172.21.15.76:0/964528082 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe8440035f0 con 0x564de6418f00 2024-03-23T15:47:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 10 client.16149 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 20 client.16149 populate_metadata read hostname 'smithi076' 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de6497ed0 0x564de649a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 4 client.16149 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564de60cf1b0 con 0x564de6497ed0 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.819+0000 7fe85a363640 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de6497ed0 0x564de649a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.820+0000 7fe85a363640 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de6497ed0 0x564de649a2f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb06b1e05cfa164 server_cookie=c88d59824dacc3c8 in_seq=0 out_seq=0 2024-03-23T15:47:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:15.820+0000 7fe83dffb640 10 client.16149 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:15.829 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:15 smithi192 ceph-mon[36653]: pgmap v120: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 870 KiB/s wr, 222 op/s 2024-03-23T15:47:15.829 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:15 smithi076 ceph-mon[32176]: pgmap v120: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 870 KiB/s wr, 222 op/s 2024-03-23T15:47:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:15 smithi183 ceph-mon[36296]: pgmap v120: 97 pgs: 97 active+clean; 1.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 870 KiB/s wr, 222 op/s 2024-03-23T15:47:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe83dffb640 1 -- 172.21.15.76:0/964528082 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564de60cf1b0 con 0x564de6497ed0 2024-03-23T15:47:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe83dffb640 10 client.16149 handle_command_reply: tid=0 2024-03-23T15:47:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 client.16149 shutdown 2024-03-23T15:47:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6418f00 msgr2=0x564de641b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6418f00 0x564de641b410 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x564de641e400 tx=0x7fe844012c40 comp rx=0 tx=0).stop 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 shutdown_connections 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de6421cd0 0x564de641b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564de6418f00 0x564de641b410 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x564de6497ed0 0x564de649a2f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe83dffb640 0 client.16149 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 --2- 172.21.15.76:0/964528082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564de64185e0 0x564de6429890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 >> 172.21.15.76:0/964528082 conn(0x564de6412f00 msgr2=0x564de6411610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 shutdown_connections 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 1 -- 172.21.15.76:0/964528082 wait complete. 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.080+0000 7fe85f1a6740 20 client.16149 trim_cache size 0 max 16384 2024-03-23T15:47:16.084 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:47:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:47:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:47:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:47:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe8400a22d0 msgr2=0x7fe8400a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe8400a22d0 0x7fe8400a4790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe84c005fd0 tx=0x7fe84c009f00 comp rx=0 tx=0).stop 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854100d10 msgr2=0x7fe85419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854100d10 0x7fe85419d8a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe850036040 tx=0x7fe85002c720 comp rx=0 tx=0).stop 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 shutdown_connections 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe8400a22d0 0x7fe8400a4790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe854106fa0 0x7fe854197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe854106680 0x7fe85419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.081+0000 7fe85c5c7640 1 --2- 172.21.15.76:0/1778193823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe854100d10 0x7fe85419d8a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.082+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 >> 172.21.15.76:0/1778193823 conn(0x7fe854077740 msgr2=0x7fe8540feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.082+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 shutdown_connections 2024-03-23T15:47:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:16.082+0000 7fe85c5c7640 1 -- 172.21.15.76:0/1778193823 wait complete. 2024-03-23T15:47:17.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:17 smithi192 ceph-mon[36653]: pgmap v121: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 279 op/s 2024-03-23T15:47:17.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:17 smithi076 ceph-mon[32176]: pgmap v121: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 279 op/s 2024-03-23T15:47:17.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:17 smithi183 ceph-mon[36296]: pgmap v121: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 279 op/s 2024-03-23T15:47:19.819 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:19 smithi192 ceph-mon[36653]: pgmap v122: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 678 KiB/s wr, 121 op/s 2024-03-23T15:47:19.825 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:19 smithi076 ceph-mon[32176]: pgmap v122: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 678 KiB/s wr, 121 op/s 2024-03-23T15:47:19.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:19 smithi183 ceph-mon[36296]: pgmap v122: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 678 KiB/s wr, 121 op/s 2024-03-23T15:47:21.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:21 smithi076 ceph-mon[32176]: pgmap v123: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 678 KiB/s wr, 121 op/s 2024-03-23T15:47:21.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:21 smithi183 ceph-mon[36296]: pgmap v123: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 678 KiB/s wr, 121 op/s 2024-03-23T15:47:22.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:21 smithi192 ceph-mon[36653]: pgmap v123: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 678 KiB/s wr, 121 op/s 2024-03-23T15:47:23.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:23 smithi076 ceph-mon[32176]: pgmap v124: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 336 op/s 2024-03-23T15:47:23.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:23 smithi183 ceph-mon[36296]: pgmap v124: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 336 op/s 2024-03-23T15:47:24.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:23 smithi192 ceph-mon[36653]: pgmap v124: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 336 op/s 2024-03-23T15:47:25.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:25 smithi183 ceph-mon[36296]: pgmap v125: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 958 KiB/s wr, 271 op/s 2024-03-23T15:47:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:25 smithi192 ceph-mon[36653]: pgmap v125: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 958 KiB/s wr, 271 op/s 2024-03-23T15:47:26.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:25 smithi076 ceph-mon[32176]: pgmap v125: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 958 KiB/s wr, 271 op/s 2024-03-23T15:47:27.872 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:27 smithi192 ceph-mon[36653]: pgmap v126: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 351 op/s 2024-03-23T15:47:27.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:27 smithi183 ceph-mon[36296]: pgmap v126: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 351 op/s 2024-03-23T15:47:28.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:27 smithi076 ceph-mon[32176]: pgmap v126: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 351 op/s 2024-03-23T15:47:28.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:28 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:47:29.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:28 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:47:29.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:28 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:47:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2131539297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 msgr2=0x7f8e9c100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2131539297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c100a70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009930 tx=0x7f8e8402bc30 comp rx=0 tx=0).stop 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2131539297 shutdown_connections 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2131539297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e9c100fb0 0x7f8e9c10d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2131539297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c100a70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2131539297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c106690 0x7f8e9c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2131539297 >> 172.21.15.76:0/2131539297 conn(0x7f8e9c077770 msgr2=0x7f8e9c077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2131539297 shutdown_connections 2024-03-23T15:47:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.189+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2131539297 wait complete. 2024-03-23T15:47:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 Processor -- start 2024-03-23T15:47:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 -- start start 2024-03-23T15:47:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e9c100fb0 0x7f8e9c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c19c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58808/0 (socket says 172.21.15.76:58808) 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c106690 0x7f8e9c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/2178071666 learned_addr learned my addr 172.21.15.76:0/2178071666 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c197010 con 0x7f8e9c1005f0 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c197180 con 0x7f8e9c100fb0 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c197340 con 0x7f8e9c106690 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8e93fff640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e9c100fb0 0x7f8e9c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.190+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c106690 0x7f8e9c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/2178071666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c106690 msgr2=0x7f8e9c1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c106690 0x7f8e9c1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/2178071666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e9c100fb0 msgr2=0x7f8e9c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e9c100fb0 0x7f8e9c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/2178071666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e84009560 con 0x7f8e9c1005f0 2024-03-23T15:47:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e93fff640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e9c100fb0 0x7f8e9c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c19c810 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00e9c0 tx=0x7f8e8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e98d99640 1 -- 172.21.15.76:0/2178071666 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c0096e0 con 0x7f8e9c1005f0 2024-03-23T15:47:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e98d99640 1 -- 172.21.15.76:0/2178071666 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8c00cd80 con 0x7f8e9c1005f0 2024-03-23T15:47:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c197620 con 0x7f8e9c1005f0 2024-03-23T15:47:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8e98d99640 1 -- 172.21.15.76:0/2178071666 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c010430 con 0x7f8e9c1005f0 2024-03-23T15:47:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.191+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a38b0 con 0x7f8e9c1005f0 2024-03-23T15:47:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.193+0000 7f8e98d99640 1 -- 172.21.15.76:0/2178071666 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f8e8c003680 con 0x7f8e9c1005f0 2024-03-23T15:47:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 Processor -- start 2024-03-23T15:47:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 -- start start 2024-03-23T15:47:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8e98d99640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8e880a22d0 0x7f8e880a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b9320951b0 con 0x55b9320dc7f0 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b932047f80 con 0x55b9320dd110 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b9320454b0 con 0x55b9320e5ee0 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8e880a22d0 0x7f8e880a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320dcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58816/0 (socket says 172.21.15.76:58816) 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327674865 learned_addr learned my addr 172.21.15.76:0/3327674865 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8e93fff640 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.194+0000 7f8e98d99640 1 -- 172.21.15.76:0/2178071666 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8e8c014070 con 0x7f8e9c1005f0 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327674865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 msgr2=0x55b9320e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327674865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 msgr2=0x55b9320e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327674865 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b931fd6810 con 0x55b9320dc7f0 2024-03-23T15:47:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8e880a22d0 0x7f8e880a4790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e94005c40 tx=0x7f8e94012b10 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320dcbd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c097300 tx=0x7f8e8c003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f6fe9871b05b10 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e917fa640 1 -- 172.21.15.76:0/3327674865 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c09a800 con 0x55b9320dc7f0 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e917fa640 1 -- 172.21.15.76:0/3327674865 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8c0ad8d0 con 0x55b9320dc7f0 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.195+0000 7f8e917fa640 1 -- 172.21.15.76:0/3327674865 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c09aea0 con 0x55b9320dc7f0 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327674865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 msgr2=0x55b9320dcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320dcbd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c097300 tx=0x7f8e8c003a20 comp rx=0 tx=0).stop 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327674865 shutdown_connections 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327674865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320dcbd0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327674865 >> 172.21.15.76:0/3327674865 conn(0x55b9320d7110 msgr2=0x55b9320d74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327674865 shutdown_connections 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327674865 wait complete. 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 -- start start 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.196+0000 7f8ea3d8f740 1 Processor -- start 2024-03-23T15:47:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dc7f0 0x55b9320e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320e5ee0 0x55b9320e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b932040c90 con 0x55b9320e5ee0 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b931fe5c50 con 0x55b9320dd110 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b931d14250 con 0x55b9320dc7f0 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8ea3d8f740 10 client.0 _ll_register_callbacks cb 0x55b932034290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320e5ee0 0x55b9320e3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320e5ee0 0x55b9320e3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58828/0 (socket says 172.21.15.76:58828) 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 -- 172.21.15.76:0/562260035 learned_addr learned my addr 172.21.15.76:0/562260035 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e93fff640 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dc7f0 0x55b9320e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 -- 172.21.15.76:0/562260035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dc7f0 msgr2=0x55b9320e2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dc7f0 0x55b9320e2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 -- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 msgr2=0x55b9320e34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 -- 172.21.15.76:0/562260035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a38b0 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320e5ee0 0x55b9320e3a20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8e94003ca0 tx=0x7f8e94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8391d224a6b1be8d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.197+0000 7f8e827fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8e827fc640 1 -- 172.21.15.76:0/562260035 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94027600 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8e827fc640 1 -- 172.21.15.76:0/562260035 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8e94003e10 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b932065790 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8e827fc640 1 -- 172.21.15.76:0/562260035 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e94027760 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 10 client.16161 fetch_fsmap learned FSMap version 27 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b93203f310 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8e827fc640 1 -- 172.21.15.76:0/562260035 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e94027da0 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8e827fc640 1 -- 172.21.15.76:0/562260035 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8e94025020 con 0x55b9320e5ee0 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 10 client.16161 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:47:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:47:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 20 client.16161 populate_metadata read hostname 'smithi076' 2024-03-23T15:47:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b9320f8430 0x55b9320fa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 4 client.16161 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T15:47:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.198+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b932065790 con 0x55b9320f8430 2024-03-23T15:47:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.199+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b9320f8430 0x55b9320fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.199+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b9320f8430 0x55b9320fa820 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438bd8f41673e51 server_cookie=c76f4bff9a14df0 in_seq=0 out_seq=0 2024-03-23T15:47:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.199+0000 7f8e827fc640 10 client.16161 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8e827fc640 1 -- 172.21.15.76:0/562260035 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b932065790 con 0x55b9320f8430 2024-03-23T15:47:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8e827fc640 10 client.16161 handle_command_reply: tid=0 2024-03-23T15:47:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 client.16161 shutdown 2024-03-23T15:47:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320e5ee0 msgr2=0x55b9320e3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320e5ee0 0x55b9320e3a20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8e94003ca0 tx=0x7f8e94004ab0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 shutdown_connections 2024-03-23T15:47:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320e5ee0 0x55b9320e3a20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b9320f8430 0x55b9320fa820 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dd110 0x55b9320e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/562260035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dc7f0 0x55b9320e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8e827fc640 0 client.16161 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 >> 172.21.15.76:0/562260035 conn(0x55b9320d7110 msgr2=0x55b9320d8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 shutdown_connections 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/562260035 wait complete. 2024-03-23T15:47:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.200+0000 7f8ea3d8f740 20 client.16161 trim_cache size 0 max 16384 2024-03-23T15:47:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.216+0000 7f8ea3d8f740 1 Processor -- start 2024-03-23T15:47:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.216+0000 7f8ea3d8f740 1 -- start start 2024-03-23T15:47:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dc7f0 0x55b9320e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dd110 0x55b9320f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b931fc6660 con 0x55b9320dd110 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b9320d6c40 con 0x55b9320dc7f0 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b9320d5f40 con 0x55b9320e5ee0 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dd110 0x55b9320f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dd110 0x55b9320f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58836/0 (socket says 172.21.15.76:58836) 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 -- 172.21.15.76:0/2465226054 learned_addr learned my addr 172.21.15.76:0/2465226054 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dc7f0 0x55b9320e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 -- 172.21.15.76:0/2465226054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 msgr2=0x55b9320ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 -- 172.21.15.76:0/2465226054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dc7f0 msgr2=0x55b9320e56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dc7f0 0x55b9320e56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 -- 172.21.15.76:0/2465226054 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b931fdb810 con 0x55b9320dd110 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.217+0000 7f8e93fff640 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dd110 0x55b9320f15b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009900 tx=0x7f8e8402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39330b4da78aec85 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8e90ff9640 1 -- 172.21.15.76:0/2465226054 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8402ca10 con 0x55b9320dd110 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8e90ff9640 1 -- 172.21.15.76:0/2465226054 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e84004510 con 0x55b9320dd110 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8e90ff9640 1 -- 172.21.15.76:0/2465226054 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84005520 con 0x55b9320dd110 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/2465226054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dd110 msgr2=0x55b9320f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dd110 0x55b9320f15b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009900 tx=0x7f8e8402c7e0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/2465226054 shutdown_connections 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320e5ee0 0x55b9320ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dd110 0x55b9320f15b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/2465226054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320dc7f0 0x55b9320e56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.218+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/2465226054 >> 172.21.15.76:0/2465226054 conn(0x55b9320d7110 msgr2=0x55b9320d74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/2465226054 shutdown_connections 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/2465226054 wait complete. 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 -- start start 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 Processor -- start 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dd110 0x55b9320e9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320e5ee0 0x55b9320ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b93215d4f0 con 0x55b9320dc7f0 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b93215d660 con 0x55b9320e5ee0 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b93215d820 con 0x55b9320dd110 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8ea3d8f740 10 client.0 _ll_register_callbacks cb 0x7f8e94003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320e96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58842/0 (socket says 172.21.15.76:58842) 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327406549 learned_addr learned my addr 172.21.15.76:0/3327406549 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e93fff640 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dd110 0x55b9320e9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9b59e640 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320e5ee0 0x55b9320ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327406549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dd110 msgr2=0x55b9320e9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dd110 0x55b9320e9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320e5ee0 msgr2=0x55b9320ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320e5ee0 0x55b9320ea140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.219+0000 7f8e9ad9d640 1 -- 172.21.15.76:0/3327406549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93203f310 con 0x55b9320dc7f0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e9ad9d640 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320e96c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c003630 tx=0x7f8e8c09ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b1388a352af881 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e827fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e827fc640 1 -- 172.21.15.76:0/3327406549 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c09b620 con 0x55b9320dc7f0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e827fc640 1 -- 172.21.15.76:0/3327406549 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8e8c09eff0 con 0x55b9320dc7f0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b9320e2dd0 con 0x55b9320dc7f0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e827fc640 1 -- 172.21.15.76:0/3327406549 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8e8c09a800 con 0x55b9320dc7f0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 10 client.16167 fetch_fsmap learned FSMap version 27 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x7f8e9c10ddc0 con 0x55b9320dc7f0 2024-03-23T15:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e827fc640 1 -- 172.21.15.76:0/3327406549 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b9320e2dd0 con 0x55b9320dc7f0 2024-03-23T15:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e827fc640 1 -- 172.21.15.76:0/3327406549 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8e8c01f060 con 0x55b9320dc7f0 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 10 client.16167 fetch_fsmap finished waiting for FSMap version 27 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 20 client.16167 populate_metadata read hostname 'smithi076' 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b93215ede0 0x55b932161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 4 client.16167 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 --> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8e8c0ad8d0 con 0x55b93215ede0 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.220+0000 7f8e93fff640 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b93215ede0 0x55b932161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.221+0000 7f8e827fc640 10 client.16167 ms_handle_connect on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.221+0000 7f8e93fff640 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b93215ede0 0x55b932161200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be25657619d20a4 server_cookie=6c4d67a06a68cd3b in_seq=0 out_seq=0 2024-03-23T15:47:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8e827fc640 1 -- 172.21.15.76:0/3327406549 <== mds.0 v2:172.21.15.183:6838/1020785546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8e8c0ad8d0 con 0x55b93215ede0 2024-03-23T15:47:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8e827fc640 10 client.16167 handle_command_reply: tid=0 2024-03-23T15:47:29.547 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 client.16167 shutdown 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 msgr2=0x55b9320e96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320e96c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c003630 tx=0x7f8e8c09ada0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 shutdown_connections 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:6838/1020785546,v1:172.21.15.183:6839/1020785546] conn(0x55b93215ede0 0x55b932161200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8e827fc640 0 client.16167 ms_handle_reset on v2:172.21.15.183:6838/1020785546 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9320e5ee0 0x55b9320ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b9320dd110 0x55b9320e9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 --2- 172.21.15.76:0/3327406549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9320dc7f0 0x55b9320e96c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.543+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 >> 172.21.15.76:0/3327406549 conn(0x55b9320d7110 msgr2=0x55b9320d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 shutdown_connections 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea3d8f740 1 -- 172.21.15.76:0/3327406549 wait complete. 2024-03-23T15:47:29.548 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea3d8f740 20 client.16167 trim_cache size 0 max 16384 2024-03-23T15:47:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:{ 2024-03-23T15:47:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "message": "", 2024-03-23T15:47:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout: "return_code": 0 2024-03-23T15:47:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stdout:} 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8e880a22d0 msgr2=0x7f8e880a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8e880a22d0 0x7f8e880a4790 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8e94005c40 tx=0x7f8e94012b10 comp rx=0 tx=0).stop 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 msgr2=0x7f8e9c19c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c19c810 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00e9c0 tx=0x7f8e8c00ee80 comp rx=0 tx=0).stop 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 shutdown_connections 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f8e880a22d0 0x7f8e880a4790 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e9c106690 0x7f8e9c1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e9c100fb0 0x7f8e9c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 --2- 172.21.15.76:0/2178071666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e9c1005f0 0x7f8e9c19c810 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:47:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 >> 172.21.15.76:0/2178071666 conn(0x7f8e9c077770 msgr2=0x7f8e9c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:47:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.544+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 shutdown_connections 2024-03-23T15:47:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi076.front.sepia.ceph.com.smithi076.stderr:2024-03-23T15:47:29.545+0000 7f8ea10d4640 1 -- 172.21.15.76:0/2178071666 wait complete. 2024-03-23T15:47:29.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:29 smithi183 ceph-mon[36296]: pgmap v127: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-03-23T15:47:29.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:29 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:47:29.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:29 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:47:29.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:29 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:47:30.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:29 smithi192 ceph-mon[36653]: pgmap v127: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-03-23T15:47:30.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:29 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:47:30.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:29 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:47:30.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:29 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:47:30.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:29 smithi076 ceph-mon[32176]: pgmap v127: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-03-23T15:47:30.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:29 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:47:30.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:29 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:47:30.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:29 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:47:31.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:31 smithi183 ceph-mon[36296]: pgmap v128: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-03-23T15:47:32.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:31 smithi192 ceph-mon[36653]: pgmap v128: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-03-23T15:47:32.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:31 smithi076 ceph-mon[32176]: pgmap v128: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.0 MiB/s wr, 294 op/s 2024-03-23T15:47:33.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:33 smithi183 ceph-mon[36296]: pgmap v129: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 510 op/s 2024-03-23T15:47:34.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:33 smithi192 ceph-mon[36653]: pgmap v129: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 510 op/s 2024-03-23T15:47:34.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:33 smithi076 ceph-mon[32176]: pgmap v129: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 510 op/s 2024-03-23T15:47:35.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:35 smithi183 ceph-mon[36296]: pgmap v130: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.0 MiB/s wr, 295 op/s 2024-03-23T15:47:36.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:35 smithi192 ceph-mon[36653]: pgmap v130: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.0 MiB/s wr, 295 op/s 2024-03-23T15:47:36.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:35 smithi076 ceph-mon[32176]: pgmap v130: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.0 MiB/s wr, 295 op/s 2024-03-23T15:47:37.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:37 smithi183 ceph-mon[36296]: pgmap v131: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 384 op/s 2024-03-23T15:47:38.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:37 smithi192 ceph-mon[36653]: pgmap v131: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 384 op/s 2024-03-23T15:47:38.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:37 smithi076 ceph-mon[32176]: pgmap v131: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 384 op/s 2024-03-23T15:47:39.877 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:39 smithi076 ceph-mon[32176]: pgmap v132: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-03-23T15:47:39.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:39 smithi183 ceph-mon[36296]: pgmap v132: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-03-23T15:47:40.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:39 smithi192 ceph-mon[36653]: pgmap v132: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-03-23T15:47:40.801 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-23T15:47:40.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-23T15:47:41.091 DEBUG:teuthology.parallel:result is None 2024-03-23T15:47:41.092 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-23T15:47:41.125 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-23T15:47:41.125 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-23T15:47:41.136 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:47:41.173 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-23T15:47:41.184 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-23T15:47:41.184 INFO:teuthology.task.exec:Executing custom commands... 2024-03-23T15:47:41.184 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:47:41.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-23T15:47:41.251 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-23T15:47:41.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:41 smithi183 ceph-mon[36296]: pgmap v133: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-03-23T15:47:42.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:41 smithi192 ceph-mon[36653]: pgmap v133: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-03-23T15:47:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:41 smithi076 ceph-mon[32176]: pgmap v133: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-03-23T15:47:43.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:43 smithi183 ceph-mon[36296]: pgmap v134: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 406 op/s 2024-03-23T15:47:44.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:43 smithi192 ceph-mon[36653]: pgmap v134: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 406 op/s 2024-03-23T15:47:44.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:43 smithi076 ceph-mon[32176]: pgmap v134: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 406 op/s 2024-03-23T15:47:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:45 smithi183 ceph-mon[36296]: pgmap v135: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-03-23T15:47:46.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:45 smithi192 ceph-mon[36653]: pgmap v135: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-03-23T15:47:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:45 smithi076 ceph-mon[32176]: pgmap v135: 97 pgs: 97 active+clean; 1.9 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-03-23T15:47:47.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:47 smithi183 ceph-mon[36296]: pgmap v136: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-03-23T15:47:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:47 smithi192 ceph-mon[36653]: pgmap v136: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-03-23T15:47:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:47 smithi076 ceph-mon[32176]: pgmap v136: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-03-23T15:47:49.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:49 smithi183 ceph-mon[36296]: pgmap v137: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 681 KiB/s wr, 129 op/s 2024-03-23T15:47:50.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:49 smithi192 ceph-mon[36653]: pgmap v137: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 681 KiB/s wr, 129 op/s 2024-03-23T15:47:50.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:49 smithi076 ceph-mon[32176]: pgmap v137: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 681 KiB/s wr, 129 op/s 2024-03-23T15:47:51.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:51 smithi183 ceph-mon[36296]: pgmap v138: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 681 KiB/s wr, 129 op/s 2024-03-23T15:47:52.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:51 smithi192 ceph-mon[36653]: pgmap v138: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 681 KiB/s wr, 129 op/s 2024-03-23T15:47:52.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:51 smithi076 ceph-mon[32176]: pgmap v138: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 681 KiB/s wr, 129 op/s 2024-03-23T15:47:53.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:53 smithi183 ceph-mon[36296]: pgmap v139: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 693 KiB/s wr, 134 op/s 2024-03-23T15:47:54.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:53 smithi192 ceph-mon[36653]: pgmap v139: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 693 KiB/s wr, 134 op/s 2024-03-23T15:47:54.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:53 smithi076 ceph-mon[32176]: pgmap v139: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 693 KiB/s wr, 134 op/s 2024-03-23T15:47:55.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:55 smithi183 ceph-mon[36296]: pgmap v140: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-03-23T15:47:56.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:55 smithi192 ceph-mon[36653]: pgmap v140: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-03-23T15:47:56.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:55 smithi076 ceph-mon[32176]: pgmap v140: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-03-23T15:47:57.871 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:57 smithi192 ceph-mon[36653]: pgmap v141: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 58 KiB/s wr, 35 op/s 2024-03-23T15:47:57.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:57 smithi183 ceph-mon[36296]: pgmap v141: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 58 KiB/s wr, 35 op/s 2024-03-23T15:47:58.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:57 smithi076 ceph-mon[32176]: pgmap v141: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 58 KiB/s wr, 35 op/s 2024-03-23T15:47:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:59 smithi183 ceph-mon[36296]: pgmap v142: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-03-23T15:47:59.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:47:59 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:48:00.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:59 smithi192 ceph-mon[36653]: pgmap v142: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-03-23T15:48:00.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:47:59 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:48:00.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:59 smithi076 ceph-mon[32176]: pgmap v142: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-03-23T15:48:00.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:47:59 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:48:01.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:01 smithi183 ceph-mon[36296]: pgmap v143: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-03-23T15:48:02.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:01 smithi192 ceph-mon[36653]: pgmap v143: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-03-23T15:48:02.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:01 smithi076 ceph-mon[32176]: pgmap v143: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-03-23T15:48:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:03 smithi192 ceph-mon[36653]: pgmap v144: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-03-23T15:48:04.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:03 smithi076 ceph-mon[32176]: pgmap v144: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-03-23T15:48:04.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:03 smithi183 ceph-mon[36296]: pgmap v144: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-03-23T15:48:06.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:05 smithi192 ceph-mon[36653]: pgmap v145: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:06.077 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:05 smithi076 ceph-mon[32176]: pgmap v145: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:06.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:05 smithi183 ceph-mon[36296]: pgmap v145: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:08.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:07 smithi192 ceph-mon[36653]: pgmap v146: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-23T15:48:08.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:07 smithi076 ceph-mon[32176]: pgmap v146: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-23T15:48:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:07 smithi183 ceph-mon[36296]: pgmap v146: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-23T15:48:10.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:09 smithi192 ceph-mon[36653]: pgmap v147: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:10.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:09 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:48:10.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:09 smithi076 ceph-mon[32176]: pgmap v147: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:09 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:48:10.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:09 smithi183 ceph-mon[36296]: pgmap v147: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:10.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:09 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:48:12.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:11 smithi192 ceph-mon[36653]: pgmap v148: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:12.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:11 smithi076 ceph-mon[32176]: pgmap v148: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:12.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:11 smithi183 ceph-mon[36296]: pgmap v148: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-03-23T15:48:14.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:13 smithi192 ceph-mon[36653]: pgmap v149: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-23T15:48:14.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:13 smithi076 ceph-mon[32176]: pgmap v149: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-23T15:48:14.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:13 smithi183 ceph-mon[36296]: pgmap v149: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-03-23T15:48:16.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:15 smithi192 ceph-mon[36653]: pgmap v150: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-23T15:48:16.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:15 smithi076 ceph-mon[32176]: pgmap v150: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-23T15:48:16.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:15 smithi183 ceph-mon[36296]: pgmap v150: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-23T15:48:18.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:17 smithi192 ceph-mon[36653]: pgmap v151: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:48:18.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:17 smithi076 ceph-mon[32176]: pgmap v151: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:48:18.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:17 smithi183 ceph-mon[36296]: pgmap v151: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:48:20.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:19 smithi192 ceph-mon[36653]: pgmap v152: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:48:20.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:19 smithi076 ceph-mon[32176]: pgmap v152: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:48:20.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:19 smithi183 ceph-mon[36296]: pgmap v152: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:48:22.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:21 smithi192 ceph-mon[36653]: pgmap v153: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:48:22.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:21 smithi076 ceph-mon[32176]: pgmap v153: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:48:22.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:21 smithi183 ceph-mon[36296]: pgmap v153: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:48:24.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:23 smithi192 ceph-mon[36653]: pgmap v154: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T15:48:24.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:23 smithi076 ceph-mon[32176]: pgmap v154: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T15:48:24.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:23 smithi183 ceph-mon[36296]: pgmap v154: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T15:48:26.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:25 smithi192 ceph-mon[36653]: pgmap v155: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:26.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:25 smithi076 ceph-mon[32176]: pgmap v155: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:26.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:25 smithi183 ceph-mon[36296]: pgmap v155: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:28.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:27 smithi192 ceph-mon[36653]: pgmap v156: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:28.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:27 smithi076 ceph-mon[32176]: pgmap v156: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:28.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:27 smithi183 ceph-mon[36296]: pgmap v156: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:29.936 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:29 smithi183 ceph-mon[36296]: pgmap v157: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:29.936 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:29 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:48:30.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:29 smithi192 ceph-mon[36653]: pgmap v157: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:30.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:29 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:48:30.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:29 smithi076 ceph-mon[32176]: pgmap v157: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:30.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:29 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:48:31.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:48:31.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:48:31.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:48:31.075 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:48:31.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:48:31.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:48:31.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:48:31.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:48:31.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:48:32.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:31 smithi192 ceph-mon[36653]: pgmap v158: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:32.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:31 smithi076 ceph-mon[32176]: pgmap v158: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:32.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:31 smithi183 ceph-mon[36296]: pgmap v158: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:34.069 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:33 smithi192 ceph-mon[36653]: pgmap v159: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:34.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:33 smithi076 ceph-mon[32176]: pgmap v159: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:34.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:33 smithi183 ceph-mon[36296]: pgmap v159: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:36.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:35 smithi192 ceph-mon[36653]: pgmap v160: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:36.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:35 smithi076 ceph-mon[32176]: pgmap v160: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:36.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:35 smithi183 ceph-mon[36296]: pgmap v160: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:38.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:37 smithi192 ceph-mon[36653]: pgmap v161: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:38.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:37 smithi076 ceph-mon[32176]: pgmap v161: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:37 smithi183 ceph-mon[36296]: pgmap v161: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:39.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:38 smithi192 ceph-mon[36653]: pgmap v162: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:39.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:38 smithi076 ceph-mon[32176]: pgmap v162: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:39.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:38 smithi183 ceph-mon[36296]: pgmap v162: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:41.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:40 smithi192 ceph-mon[36653]: pgmap v163: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:41.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:40 smithi076 ceph-mon[32176]: pgmap v163: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:41.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:40 smithi183 ceph-mon[36296]: pgmap v163: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:43.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:42 smithi192 ceph-mon[36653]: pgmap v164: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:43.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:42 smithi076 ceph-mon[32176]: pgmap v164: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:43.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:42 smithi183 ceph-mon[36296]: pgmap v164: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:45.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:44 smithi192 ceph-mon[36653]: pgmap v165: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:45.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:44 smithi076 ceph-mon[32176]: pgmap v165: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:45.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:44 smithi183 ceph-mon[36296]: pgmap v165: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:47.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:46 smithi192 ceph-mon[36653]: pgmap v166: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:47.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:46 smithi076 ceph-mon[32176]: pgmap v166: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:47.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:46 smithi183 ceph-mon[36296]: pgmap v166: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:49.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:48 smithi192 ceph-mon[36653]: pgmap v167: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:49.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:48 smithi076 ceph-mon[32176]: pgmap v167: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:49.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:48 smithi183 ceph-mon[36296]: pgmap v167: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:51.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:50 smithi192 ceph-mon[36653]: pgmap v168: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:51.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:50 smithi076 ceph-mon[32176]: pgmap v168: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:51.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:50 smithi183 ceph-mon[36296]: pgmap v168: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:53.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:52 smithi192 ceph-mon[36653]: pgmap v169: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:53.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:52 smithi076 ceph-mon[32176]: pgmap v169: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:52 smithi183 ceph-mon[36296]: pgmap v169: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:55.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:54 smithi192 ceph-mon[36653]: pgmap v170: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:55.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:54 smithi076 ceph-mon[32176]: pgmap v170: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:55.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:54 smithi183 ceph-mon[36296]: pgmap v170: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:57.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:56 smithi192 ceph-mon[36653]: pgmap v171: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:57.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:56 smithi076 ceph-mon[32176]: pgmap v171: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:57.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:56 smithi183 ceph-mon[36296]: pgmap v171: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:48:59.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:48:58 smithi192 ceph-mon[36653]: pgmap v172: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:59.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:48:58 smithi076 ceph-mon[32176]: pgmap v172: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:48:59.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:48:58 smithi183 ceph-mon[36296]: pgmap v172: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:00 smithi192 ceph-mon[36653]: pgmap v173: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:00 smithi076 ceph-mon[32176]: pgmap v173: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:01.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:00 smithi183 ceph-mon[36296]: pgmap v173: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:03.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:02 smithi192 ceph-mon[36653]: pgmap v174: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:02 smithi076 ceph-mon[32176]: pgmap v174: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:03.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:02 smithi183 ceph-mon[36296]: pgmap v174: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:05.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:04 smithi192 ceph-mon[36653]: pgmap v175: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-03-23T15:49:05.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:04 smithi076 ceph-mon[32176]: pgmap v175: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-03-23T15:49:05.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:04 smithi183 ceph-mon[36296]: pgmap v175: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-03-23T15:49:07.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:06 smithi192 ceph-mon[36653]: pgmap v176: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:07.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:06 smithi076 ceph-mon[32176]: pgmap v176: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:07.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:06 smithi183 ceph-mon[36296]: pgmap v176: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:09.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:08 smithi192 ceph-mon[36653]: pgmap v177: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-23T15:49:09.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:08 smithi076 ceph-mon[32176]: pgmap v177: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-23T15:49:09.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:08 smithi183 ceph-mon[36296]: pgmap v177: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-23T15:49:11.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:10 smithi192 ceph-mon[36653]: pgmap v178: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-23T15:49:11.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:10 smithi076 ceph-mon[32176]: pgmap v178: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-23T15:49:11.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:10 smithi183 ceph-mon[36296]: pgmap v178: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-23T15:49:13.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:12 smithi192 ceph-mon[36653]: pgmap v179: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:13.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:12 smithi076 ceph-mon[32176]: pgmap v179: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:13.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:12 smithi183 ceph-mon[36296]: pgmap v179: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-23T15:49:15.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:14 smithi192 ceph-mon[36653]: pgmap v180: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:49:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:14 smithi076 ceph-mon[32176]: pgmap v180: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:49:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:14 smithi183 ceph-mon[36296]: pgmap v180: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-23T15:49:17.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:16 smithi192 ceph-mon[36653]: pgmap v181: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T15:49:17.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:16 smithi076 ceph-mon[32176]: pgmap v181: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T15:49:17.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:16 smithi183 ceph-mon[36296]: pgmap v181: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T15:49:19.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:18 smithi192 ceph-mon[36653]: pgmap v182: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-23T15:49:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:18 smithi076 ceph-mon[32176]: pgmap v182: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-23T15:49:19.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:18 smithi183 ceph-mon[36296]: pgmap v182: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-23T15:49:21.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:20 smithi192 ceph-mon[36653]: pgmap v183: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-23T15:49:21.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:20 smithi076 ceph-mon[32176]: pgmap v183: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-23T15:49:21.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:20 smithi183 ceph-mon[36296]: pgmap v183: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-23T15:49:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:22 smithi192 ceph-mon[36653]: pgmap v184: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:49:23.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:22 smithi076 ceph-mon[32176]: pgmap v184: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:49:23.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:22 smithi183 ceph-mon[36296]: pgmap v184: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:49:25.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:24 smithi192 ceph-mon[36653]: pgmap v185: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:25.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:24 smithi076 ceph-mon[32176]: pgmap v185: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:25.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:24 smithi183 ceph-mon[36296]: pgmap v185: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:26 smithi192 ceph-mon[36653]: pgmap v186: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:26 smithi076 ceph-mon[32176]: pgmap v186: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:26 smithi183 ceph-mon[36296]: pgmap v186: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:29.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:28 smithi192 ceph-mon[36653]: pgmap v187: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:29.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:28 smithi076 ceph-mon[32176]: pgmap v187: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:28 smithi183 ceph-mon[36296]: pgmap v187: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:30 smithi192 ceph-mon[36653]: pgmap v188: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:49:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:49:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:49:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:49:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:30 smithi076 ceph-mon[32176]: pgmap v188: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:49:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:49:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:49:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:49:31.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:30 smithi183 ceph-mon[36296]: pgmap v188: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:49:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:49:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:49:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:49:33.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:32 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:49:33.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:32 smithi192 ceph-mon[36653]: pgmap v189: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:33.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:32 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:49:33.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:32 smithi076 ceph-mon[32176]: pgmap v189: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:33.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:32 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:49:33.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:32 smithi183 ceph-mon[36296]: pgmap v189: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:35.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:34 smithi192 ceph-mon[36653]: pgmap v190: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:35.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:34 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:49:35.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:34 smithi076 ceph-mon[32176]: pgmap v190: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:35.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:34 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:49:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:34 smithi183 ceph-mon[36296]: pgmap v190: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:35.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:34 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:49:37.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:36 smithi192 ceph-mon[36653]: pgmap v191: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:37.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:36 smithi076 ceph-mon[32176]: pgmap v191: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:37.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:36 smithi183 ceph-mon[36296]: pgmap v191: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:39.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:38 smithi192 ceph-mon[36653]: pgmap v192: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:39.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:38 smithi076 ceph-mon[32176]: pgmap v192: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:39.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:38 smithi183 ceph-mon[36296]: pgmap v192: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:41.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:40 smithi192 ceph-mon[36653]: pgmap v193: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:41.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:40 smithi076 ceph-mon[32176]: pgmap v193: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:41.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:40 smithi183 ceph-mon[36296]: pgmap v193: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:43.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:42 smithi192 ceph-mon[36653]: pgmap v194: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:43.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:42 smithi076 ceph-mon[32176]: pgmap v194: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:43.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:42 smithi183 ceph-mon[36296]: pgmap v194: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:45.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:44 smithi192 ceph-mon[36653]: pgmap v195: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:45.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:44 smithi076 ceph-mon[32176]: pgmap v195: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:45.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:44 smithi183 ceph-mon[36296]: pgmap v195: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:47.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:46 smithi192 ceph-mon[36653]: pgmap v196: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:47.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:46 smithi076 ceph-mon[32176]: pgmap v196: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:47.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:46 smithi183 ceph-mon[36296]: pgmap v196: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:49.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:48 smithi192 ceph-mon[36653]: pgmap v197: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:49.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:48 smithi076 ceph-mon[32176]: pgmap v197: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:49.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:48 smithi183 ceph-mon[36296]: pgmap v197: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:51.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:50 smithi192 ceph-mon[36653]: pgmap v198: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:50 smithi076 ceph-mon[32176]: pgmap v198: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:51.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:50 smithi183 ceph-mon[36296]: pgmap v198: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:53.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:52 smithi192 ceph-mon[36653]: pgmap v199: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:53.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:52 smithi076 ceph-mon[32176]: pgmap v199: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:52 smithi183 ceph-mon[36296]: pgmap v199: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:55.319 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:54 smithi192 ceph-mon[36653]: pgmap v200: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:55.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:54 smithi076 ceph-mon[32176]: pgmap v200: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:55.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:54 smithi183 ceph-mon[36296]: pgmap v200: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:57.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:56 smithi192 ceph-mon[36653]: pgmap v201: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:57.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:56 smithi076 ceph-mon[32176]: pgmap v201: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:57.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:56 smithi183 ceph-mon[36296]: pgmap v201: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:49:59.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:49:58 smithi192 ceph-mon[36653]: pgmap v202: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:59.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:49:58 smithi076 ceph-mon[32176]: pgmap v202: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:49:59.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:49:58 smithi183 ceph-mon[36296]: pgmap v202: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:00.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:49:59.999+0000 7fa1f8f14700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-23T15:50:00.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:49:59.999+0000 7fa1f8f14700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-23T15:50:00.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:49:59.999+0000 7fa1f8f14700 -1 log_channel(cluster) log [ERR] : mds.h(mds.2): Metadata damage detected 2024-03-23T15:50:00.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:49:59.999+0000 7fa1f8f14700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2024-03-23T15:50:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:00 smithi192 ceph-mon[36653]: pgmap v203: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:00 smithi192 ceph-mon[36653]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-23T15:50:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:00 smithi192 ceph-mon[36653]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-23T15:50:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:00 smithi192 ceph-mon[36653]: mds.h(mds.2): Metadata damage detected 2024-03-23T15:50:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:00 smithi192 ceph-mon[36653]: mds.i(mds.1): Metadata damage detected 2024-03-23T15:50:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-mon[32176]: pgmap v203: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-mon[32176]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-23T15:50:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-mon[32176]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-23T15:50:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-mon[32176]: mds.h(mds.2): Metadata damage detected 2024-03-23T15:50:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:00 smithi076 ceph-mon[32176]: mds.i(mds.1): Metadata damage detected 2024-03-23T15:50:01.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:00 smithi183 ceph-mon[36296]: pgmap v203: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:01.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:00 smithi183 ceph-mon[36296]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-23T15:50:01.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:00 smithi183 ceph-mon[36296]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-23T15:50:01.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:00 smithi183 ceph-mon[36296]: mds.h(mds.2): Metadata damage detected 2024-03-23T15:50:01.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:00 smithi183 ceph-mon[36296]: mds.i(mds.1): Metadata damage detected 2024-03-23T15:50:03.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:02 smithi192 ceph-mon[36653]: pgmap v204: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-23T15:50:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:02 smithi076 ceph-mon[32176]: pgmap v204: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-23T15:50:03.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:02 smithi183 ceph-mon[36296]: pgmap v204: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-23T15:50:05.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:04 smithi192 ceph-mon[36653]: pgmap v205: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-23T15:50:05.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:04 smithi076 ceph-mon[32176]: pgmap v205: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-23T15:50:05.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:04 smithi183 ceph-mon[36296]: pgmap v205: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-23T15:50:07.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:06 smithi192 ceph-mon[36653]: pgmap v206: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-23T15:50:07.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:06 smithi076 ceph-mon[32176]: pgmap v206: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-23T15:50:07.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:06 smithi183 ceph-mon[36296]: pgmap v206: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-03-23T15:50:09.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:08 smithi192 ceph-mon[36653]: pgmap v207: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-23T15:50:09.325 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:08 smithi076 ceph-mon[32176]: pgmap v207: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-23T15:50:09.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:08 smithi183 ceph-mon[36296]: pgmap v207: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-03-23T15:50:10.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:09 smithi192 ceph-mon[36653]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T15:50:10.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:09 smithi076 ceph-mon[32176]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T15:50:10.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:09 smithi183 ceph-mon[36296]: osdmap e89: 12 total, 12 up, 12 in 2024-03-23T15:50:11.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:10 smithi076 ceph-mon[32176]: pgmap v209: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-23T15:50:11.322 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:10 smithi192 ceph-mon[36653]: pgmap v209: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-23T15:50:11.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:10 smithi183 ceph-mon[36296]: pgmap v209: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-03-23T15:50:13.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:12 smithi192 ceph-mon[36653]: pgmap v210: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:50:13.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:12 smithi076 ceph-mon[32176]: pgmap v210: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:50:13.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:12 smithi183 ceph-mon[36296]: pgmap v210: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:50:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:14 smithi192 ceph-mon[36653]: osdmap e90: 12 total, 12 up, 12 in 2024-03-23T15:50:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:14 smithi192 ceph-mon[36653]: pgmap v212: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-23T15:50:15.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:14 smithi076 ceph-mon[32176]: osdmap e90: 12 total, 12 up, 12 in 2024-03-23T15:50:15.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:14 smithi076 ceph-mon[32176]: pgmap v212: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-23T15:50:15.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:14 smithi183 ceph-mon[36296]: osdmap e90: 12 total, 12 up, 12 in 2024-03-23T15:50:15.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:14 smithi183 ceph-mon[36296]: pgmap v212: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-23T15:50:17.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:16 smithi192 ceph-mon[36653]: pgmap v213: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:50:17.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:16 smithi076 ceph-mon[32176]: pgmap v213: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:50:17.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:16 smithi183 ceph-mon[36296]: pgmap v213: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:50:19.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:18 smithi192 ceph-mon[36653]: pgmap v214: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 228 B/s wr, 7 op/s 2024-03-23T15:50:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:18 smithi076 ceph-mon[32176]: pgmap v214: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 228 B/s wr, 7 op/s 2024-03-23T15:50:19.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:18 smithi183 ceph-mon[36296]: pgmap v214: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 228 B/s wr, 7 op/s 2024-03-23T15:50:21.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:20 smithi192 ceph-mon[36653]: pgmap v215: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:50:21.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:20 smithi076 ceph-mon[32176]: pgmap v215: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:50:21.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:20 smithi183 ceph-mon[36296]: pgmap v215: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:50:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:22 smithi192 ceph-mon[36653]: pgmap v216: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:50:23.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:22 smithi076 ceph-mon[32176]: pgmap v216: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:50:23.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:22 smithi183 ceph-mon[36296]: pgmap v216: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:50:25.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:24 smithi192 ceph-mon[36653]: pgmap v217: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 5 op/s 2024-03-23T15:50:25.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:24 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:50:25.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:24 smithi076 ceph-mon[32176]: pgmap v217: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 5 op/s 2024-03-23T15:50:25.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:24 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:50:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:24 smithi183 ceph-mon[36296]: pgmap v217: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 5 op/s 2024-03-23T15:50:25.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:24 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:50:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:26 smithi192 ceph-mon[36653]: pgmap v218: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:50:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:26 smithi192 ceph-mon[36653]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:50:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:26 smithi076 ceph-mon[32176]: pgmap v218: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:50:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:26 smithi076 ceph-mon[32176]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:50:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:26 smithi183 ceph-mon[36296]: pgmap v218: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:50:27.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:26 smithi183 ceph-mon[36296]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-23T15:50:29.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:28 smithi192 ceph-mon[36653]: pgmap v219: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:29.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:28 smithi076 ceph-mon[32176]: pgmap v219: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:28 smithi183 ceph-mon[36296]: pgmap v219: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:30 smithi192 ceph-mon[36653]: pgmap v220: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:30 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:50:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:30 smithi076 ceph-mon[32176]: pgmap v220: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:30 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:50:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:30 smithi183 ceph-mon[36296]: pgmap v220: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:30 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:50:32.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:31 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:50:32.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:31 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:50:32.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:31 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:50:32.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:31 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:50:32.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:31 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:50:32.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:31 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:50:32.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:31 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:50:32.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:31 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:50:32.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:31 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:50:34.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:33 smithi192 ceph-mon[36653]: pgmap v221: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:34.082 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:33 smithi076 ceph-mon[32176]: pgmap v221: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:34.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:33 smithi183 ceph-mon[36296]: pgmap v221: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:36.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:35 smithi192 ceph-mon[36653]: pgmap v222: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:36.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:35 smithi076 ceph-mon[32176]: pgmap v222: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:36.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:35 smithi183 ceph-mon[36296]: pgmap v222: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:38.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:37 smithi192 ceph-mon[36653]: pgmap v223: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:38.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:37 smithi076 ceph-mon[32176]: pgmap v223: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:38.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:37 smithi183 ceph-mon[36296]: pgmap v223: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:40.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:39 smithi192 ceph-mon[36653]: pgmap v224: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:40.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:39 smithi076 ceph-mon[32176]: pgmap v224: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:39 smithi183 ceph-mon[36296]: pgmap v224: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:41 smithi192 ceph-mon[36653]: pgmap v225: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:41 smithi076 ceph-mon[32176]: pgmap v225: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:42.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:41 smithi183 ceph-mon[36296]: pgmap v225: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:43.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:42 smithi192 ceph-mon[36653]: pgmap v226: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:43.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:42 smithi076 ceph-mon[32176]: pgmap v226: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:43.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:42 smithi183 ceph-mon[36296]: pgmap v226: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:45.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:44 smithi192 ceph-mon[36653]: pgmap v227: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:45.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:44 smithi076 ceph-mon[32176]: pgmap v227: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:45.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:44 smithi183 ceph-mon[36296]: pgmap v227: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:47.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:46 smithi192 ceph-mon[36653]: pgmap v228: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:47.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:46 smithi076 ceph-mon[32176]: pgmap v228: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:47.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:46 smithi183 ceph-mon[36296]: pgmap v228: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:49.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:48 smithi192 ceph-mon[36653]: pgmap v229: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:49.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:48 smithi076 ceph-mon[32176]: pgmap v229: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:49.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:48 smithi183 ceph-mon[36296]: pgmap v229: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:51.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:50 smithi192 ceph-mon[36653]: pgmap v230: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:51.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:50 smithi076 ceph-mon[32176]: pgmap v230: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:51.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:50 smithi183 ceph-mon[36296]: pgmap v230: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:53.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:52 smithi192 ceph-mon[36653]: pgmap v231: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:53.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:52 smithi076 ceph-mon[32176]: pgmap v231: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:52 smithi183 ceph-mon[36296]: pgmap v231: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:55.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:54 smithi192 ceph-mon[36653]: pgmap v232: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:55.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:54 smithi076 ceph-mon[32176]: pgmap v232: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:55.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:54 smithi183 ceph-mon[36296]: pgmap v232: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:57.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:56 smithi192 ceph-mon[36653]: pgmap v233: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:57.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:56 smithi076 ceph-mon[32176]: pgmap v233: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:57.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:56 smithi183 ceph-mon[36296]: pgmap v233: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:50:59.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:50:58 smithi192 ceph-mon[36653]: pgmap v234: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:59.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:50:58 smithi076 ceph-mon[32176]: pgmap v234: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:50:59.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:50:58 smithi183 ceph-mon[36296]: pgmap v234: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:00 smithi192 ceph-mon[36653]: pgmap v235: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:00 smithi076 ceph-mon[32176]: pgmap v235: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:01.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:00 smithi183 ceph-mon[36296]: pgmap v235: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:03.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:02 smithi192 ceph-mon[36653]: pgmap v236: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:51:03.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:02 smithi076 ceph-mon[32176]: pgmap v236: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:51:03.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:02 smithi183 ceph-mon[36296]: pgmap v236: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:51:05.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:04 smithi192 ceph-mon[36653]: pgmap v237: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:51:05.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:04 smithi076 ceph-mon[32176]: pgmap v237: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:51:05.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:04 smithi183 ceph-mon[36296]: pgmap v237: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:51:07.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:07 smithi192 ceph-mon[36653]: pgmap v238: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:51:07.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:07 smithi076 ceph-mon[32176]: pgmap v238: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:51:07.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:07 smithi183 ceph-mon[36296]: pgmap v238: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:51:09.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:09 smithi192 ceph-mon[36653]: pgmap v239: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:51:09.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:09 smithi076 ceph-mon[32176]: pgmap v239: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:51:09.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:09 smithi183 ceph-mon[36296]: pgmap v239: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:51:10.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:10 smithi192 ceph-mon[36653]: osdmap e91: 12 total, 12 up, 12 in 2024-03-23T15:51:10.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:10 smithi076 ceph-mon[32176]: osdmap e91: 12 total, 12 up, 12 in 2024-03-23T15:51:10.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:10 smithi183 ceph-mon[36296]: osdmap e91: 12 total, 12 up, 12 in 2024-03-23T15:51:11.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:11 smithi192 ceph-mon[36653]: pgmap v241: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:51:11.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:11 smithi076 ceph-mon[32176]: pgmap v241: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:51:11.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:11 smithi183 ceph-mon[36296]: pgmap v241: 97 pgs: 97 active+clean; 1.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:51:13.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:13 smithi192 ceph-mon[36653]: pgmap v242: 97 pgs: 19 active+clean+snaptrim, 40 active+clean, 38 active+clean+snaptrim_wait; 804 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:13.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:13 smithi076 ceph-mon[32176]: pgmap v242: 97 pgs: 19 active+clean+snaptrim, 40 active+clean, 38 active+clean+snaptrim_wait; 804 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:13.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:13 smithi183 ceph-mon[36296]: pgmap v242: 97 pgs: 19 active+clean+snaptrim, 40 active+clean, 38 active+clean+snaptrim_wait; 804 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:15.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:15 smithi192 ceph-mon[36653]: pgmap v243: 97 pgs: 19 active+clean+snaptrim, 40 active+clean, 38 active+clean+snaptrim_wait; 804 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:15 smithi076 ceph-mon[32176]: pgmap v243: 97 pgs: 19 active+clean+snaptrim, 40 active+clean, 38 active+clean+snaptrim_wait; 804 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:15.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:15 smithi183 ceph-mon[36296]: pgmap v243: 97 pgs: 19 active+clean+snaptrim, 40 active+clean, 38 active+clean+snaptrim_wait; 804 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:17.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:17 smithi192 ceph-mon[36653]: pgmap v244: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:17.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:17 smithi076 ceph-mon[32176]: pgmap v244: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:17.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:17 smithi183 ceph-mon[36296]: pgmap v244: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:19.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:19 smithi192 ceph-mon[36653]: pgmap v245: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:19.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:19 smithi192 ceph-mon[36653]: osdmap e92: 12 total, 12 up, 12 in 2024-03-23T15:51:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:19 smithi076 ceph-mon[32176]: pgmap v245: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:19 smithi076 ceph-mon[32176]: osdmap e92: 12 total, 12 up, 12 in 2024-03-23T15:51:19.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:19 smithi183 ceph-mon[36296]: pgmap v245: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:19.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:19 smithi183 ceph-mon[36296]: osdmap e92: 12 total, 12 up, 12 in 2024-03-23T15:51:21.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:21 smithi192 ceph-mon[36653]: pgmap v247: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:21.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:21 smithi076 ceph-mon[32176]: pgmap v247: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:21.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:21 smithi183 ceph-mon[36296]: pgmap v247: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:51:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:23 smithi192 ceph-mon[36653]: pgmap v248: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:51:23.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:23 smithi076 ceph-mon[32176]: pgmap v248: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:51:23.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:23 smithi183 ceph-mon[36296]: pgmap v248: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:51:25.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:25 smithi192 ceph-mon[36653]: pgmap v249: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:51:25.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:25 smithi076 ceph-mon[32176]: pgmap v249: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:51:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:25 smithi183 ceph-mon[36296]: pgmap v249: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:51:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:27 smithi192 ceph-mon[36653]: pgmap v250: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:51:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:27 smithi076 ceph-mon[32176]: pgmap v250: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:51:27.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:27 smithi183 ceph-mon[36296]: pgmap v250: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:51:29.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:29 smithi192 ceph-mon[36653]: pgmap v251: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:51:29.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:29 smithi076 ceph-mon[32176]: pgmap v251: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:51:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:29 smithi183 ceph-mon[36296]: pgmap v251: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:51:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:31 smithi192 ceph-mon[36653]: pgmap v252: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-23T15:51:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:31 smithi076 ceph-mon[32176]: pgmap v252: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-23T15:51:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:31 smithi183 ceph-mon[36296]: pgmap v252: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-23T15:51:32.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:32 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:51:32.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:32 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:51:32.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:32 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:51:33.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:33 smithi192 ceph-mon[36653]: pgmap v253: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:33.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:33 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:51:33.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:33 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:51:33.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:33 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:51:33.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:33 smithi076 ceph-mon[32176]: pgmap v253: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:33.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:33 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:51:33.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:33 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:51:33.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:33 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:51:33.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:33 smithi183 ceph-mon[36296]: pgmap v253: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:33.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:33 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:51:33.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:33 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:51:33.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:33 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:51:35.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:35 smithi192 ceph-mon[36653]: pgmap v254: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:35.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:35 smithi076 ceph-mon[32176]: pgmap v254: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:35 smithi183 ceph-mon[36296]: pgmap v254: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:37.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:37 smithi192 ceph-mon[36653]: pgmap v255: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:37.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:37 smithi076 ceph-mon[32176]: pgmap v255: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:37.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:37 smithi183 ceph-mon[36296]: pgmap v255: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:39.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:39 smithi192 ceph-mon[36653]: pgmap v256: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:39.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:39 smithi076 ceph-mon[32176]: pgmap v256: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:39.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:39 smithi183 ceph-mon[36296]: pgmap v256: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:41.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:41 smithi192 ceph-mon[36653]: pgmap v257: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:41.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:41 smithi076 ceph-mon[32176]: pgmap v257: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:41.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:41 smithi183 ceph-mon[36296]: pgmap v257: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:43.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:43 smithi192 ceph-mon[36653]: pgmap v258: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:43.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:43 smithi076 ceph-mon[32176]: pgmap v258: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:43.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:43 smithi183 ceph-mon[36296]: pgmap v258: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:45.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:44 smithi192 ceph-mon[36653]: pgmap v259: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:45.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:44 smithi076 ceph-mon[32176]: pgmap v259: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:45.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:44 smithi183 ceph-mon[36296]: pgmap v259: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:47.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:46 smithi192 ceph-mon[36653]: pgmap v260: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:47.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:46 smithi076 ceph-mon[32176]: pgmap v260: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:47.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:46 smithi183 ceph-mon[36296]: pgmap v260: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:49.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:48 smithi192 ceph-mon[36653]: pgmap v261: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:49.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:48 smithi076 ceph-mon[32176]: pgmap v261: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:49.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:48 smithi183 ceph-mon[36296]: pgmap v261: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:51.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:50 smithi192 ceph-mon[36653]: pgmap v262: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:51.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:50 smithi076 ceph-mon[32176]: pgmap v262: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:51.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:50 smithi183 ceph-mon[36296]: pgmap v262: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:53.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:52 smithi192 ceph-mon[36653]: pgmap v263: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:53.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:52 smithi076 ceph-mon[32176]: pgmap v263: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:52 smithi183 ceph-mon[36296]: pgmap v263: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:55.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:55 smithi192 ceph-mon[36653]: pgmap v264: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:55.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:55 smithi076 ceph-mon[32176]: pgmap v264: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:55.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:55 smithi183 ceph-mon[36296]: pgmap v264: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:57.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:57 smithi192 ceph-mon[36653]: pgmap v265: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:57.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:57 smithi076 ceph-mon[32176]: pgmap v265: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:57.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:57 smithi183 ceph-mon[36296]: pgmap v265: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:51:59.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:51:59 smithi192 ceph-mon[36653]: pgmap v266: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:59.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:51:59 smithi076 ceph-mon[32176]: pgmap v266: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:51:59.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:51:59 smithi183 ceph-mon[36296]: pgmap v266: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:01.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:01 smithi192 ceph-mon[36653]: pgmap v267: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:01.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:01 smithi076 ceph-mon[32176]: pgmap v267: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:01.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:01 smithi183 ceph-mon[36296]: pgmap v267: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:03.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:03 smithi192 ceph-mon[36653]: pgmap v268: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:52:03.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:03 smithi076 ceph-mon[32176]: pgmap v268: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:52:03.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:03 smithi183 ceph-mon[36296]: pgmap v268: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:52:05.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:05 smithi192 ceph-mon[36653]: pgmap v269: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:52:05.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:05 smithi076 ceph-mon[32176]: pgmap v269: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:52:05.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:05 smithi183 ceph-mon[36296]: pgmap v269: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:52:07.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:07 smithi192 ceph-mon[36653]: pgmap v270: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:52:07.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:07 smithi076 ceph-mon[32176]: pgmap v270: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:52:07.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:07 smithi183 ceph-mon[36296]: pgmap v270: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:52:09.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:09 smithi192 ceph-mon[36653]: pgmap v271: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:52:09.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:09 smithi076 ceph-mon[32176]: pgmap v271: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:52:09.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:09 smithi183 ceph-mon[36296]: pgmap v271: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:52:10.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:10 smithi192 ceph-mon[36653]: osdmap e93: 12 total, 12 up, 12 in 2024-03-23T15:52:10.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:10 smithi076 ceph-mon[32176]: osdmap e93: 12 total, 12 up, 12 in 2024-03-23T15:52:10.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:10 smithi183 ceph-mon[36296]: osdmap e93: 12 total, 12 up, 12 in 2024-03-23T15:52:11.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:11 smithi192 ceph-mon[36653]: pgmap v273: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:52:11.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:11 smithi076 ceph-mon[32176]: pgmap v273: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:52:11.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:11 smithi183 ceph-mon[36296]: pgmap v273: 97 pgs: 97 active+clean; 134 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:52:13.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:13 smithi192 ceph-mon[36653]: pgmap v274: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:13.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:13 smithi076 ceph-mon[32176]: pgmap v274: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:13.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:13 smithi183 ceph-mon[36296]: pgmap v274: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:15.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:15 smithi192 ceph-mon[36653]: pgmap v275: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:15.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:15 smithi076 ceph-mon[32176]: pgmap v275: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:15.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:15 smithi183 ceph-mon[36296]: pgmap v275: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:17.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:17 smithi192 ceph-mon[36653]: pgmap v276: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:17.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:17 smithi076 ceph-mon[32176]: pgmap v276: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:17.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:17 smithi183 ceph-mon[36296]: pgmap v276: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:19.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:19 smithi192 ceph-mon[36653]: pgmap v277: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:19.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:19 smithi192 ceph-mon[36653]: osdmap e94: 12 total, 12 up, 12 in 2024-03-23T15:52:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:19 smithi076 ceph-mon[32176]: pgmap v277: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:19 smithi076 ceph-mon[32176]: osdmap e94: 12 total, 12 up, 12 in 2024-03-23T15:52:19.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:19 smithi183 ceph-mon[36296]: pgmap v277: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:19.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:19 smithi183 ceph-mon[36296]: osdmap e94: 12 total, 12 up, 12 in 2024-03-23T15:52:21.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:21 smithi192 ceph-mon[36653]: pgmap v279: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:21.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:21 smithi076 ceph-mon[32176]: pgmap v279: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:21.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:21 smithi183 ceph-mon[36296]: pgmap v279: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:52:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:23 smithi192 ceph-mon[36653]: pgmap v280: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:52:23.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:23 smithi076 ceph-mon[32176]: pgmap v280: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:52:23.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:23 smithi183 ceph-mon[36296]: pgmap v280: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:52:25.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:25 smithi192 ceph-mon[36653]: pgmap v281: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:52:25.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:25 smithi076 ceph-mon[32176]: pgmap v281: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:52:25.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:25 smithi183 ceph-mon[36296]: pgmap v281: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:52:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:27 smithi192 ceph-mon[36653]: pgmap v282: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:52:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:27 smithi076 ceph-mon[32176]: pgmap v282: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:52:27.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:27 smithi183 ceph-mon[36296]: pgmap v282: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:52:29.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:29 smithi192 ceph-mon[36653]: pgmap v283: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:52:29.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:29 smithi076 ceph-mon[32176]: pgmap v283: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:52:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:29 smithi183 ceph-mon[36296]: pgmap v283: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-23T15:52:31.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:31 smithi192 ceph-mon[36653]: pgmap v284: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-23T15:52:31.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:31 smithi076 ceph-mon[32176]: pgmap v284: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-23T15:52:31.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:31 smithi183 ceph-mon[36296]: pgmap v284: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-23T15:52:33.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:33 smithi076 ceph-mon[32176]: pgmap v285: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:33.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:33 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:52:33.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:33 smithi183 ceph-mon[36296]: pgmap v285: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:33.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:33 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:52:33.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:33 smithi192 ceph-mon[36653]: pgmap v285: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:33.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:33 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:52:34.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:34 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:52:34.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:34 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:52:34.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:34 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:52:34.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:34 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:52:34.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:34 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:52:34.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:34 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:52:34.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:34 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:52:34.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:34 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:52:34.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:34 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:52:35.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:35 smithi192 ceph-mon[36653]: pgmap v286: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:35.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:35 smithi076 ceph-mon[32176]: pgmap v286: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:35.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:35 smithi183 ceph-mon[36296]: pgmap v286: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:37 smithi192 ceph-mon[36653]: pgmap v287: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:37.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:37 smithi076 ceph-mon[32176]: pgmap v287: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:37 smithi183 ceph-mon[36296]: pgmap v287: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:39.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:39 smithi192 ceph-mon[36653]: pgmap v288: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:39.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:39 smithi076 ceph-mon[32176]: pgmap v288: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:39.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:39 smithi183 ceph-mon[36296]: pgmap v288: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:41.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:41 smithi192 ceph-mon[36653]: pgmap v289: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:41.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:41 smithi076 ceph-mon[32176]: pgmap v289: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:41.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:41 smithi183 ceph-mon[36296]: pgmap v289: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:43.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:43 smithi192 ceph-mon[36653]: pgmap v290: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:43.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:43 smithi076 ceph-mon[32176]: pgmap v290: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:43.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:43 smithi183 ceph-mon[36296]: pgmap v290: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:45.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:45 smithi192 ceph-mon[36653]: pgmap v291: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:45.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:45 smithi076 ceph-mon[32176]: pgmap v291: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:45.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:45 smithi183 ceph-mon[36296]: pgmap v291: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:47.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:47 smithi192 ceph-mon[36653]: pgmap v292: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:47.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:47 smithi076 ceph-mon[32176]: pgmap v292: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:47.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:47 smithi183 ceph-mon[36296]: pgmap v292: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:49.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:49 smithi192 ceph-mon[36653]: pgmap v293: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:49.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:49 smithi076 ceph-mon[32176]: pgmap v293: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:49.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:49 smithi183 ceph-mon[36296]: pgmap v293: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:51.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:51 smithi192 ceph-mon[36653]: pgmap v294: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:51.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:51 smithi076 ceph-mon[32176]: pgmap v294: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:51.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:51 smithi183 ceph-mon[36296]: pgmap v294: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:53.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:53 smithi192 ceph-mon[36653]: pgmap v295: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:53.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:53 smithi076 ceph-mon[32176]: pgmap v295: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:53.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:53 smithi183 ceph-mon[36296]: pgmap v295: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:55.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:55 smithi192 ceph-mon[36653]: pgmap v296: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:55.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:55 smithi076 ceph-mon[32176]: pgmap v296: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:55.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:55 smithi183 ceph-mon[36296]: pgmap v296: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:57.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:57 smithi192 ceph-mon[36653]: pgmap v297: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:57.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:57 smithi076 ceph-mon[32176]: pgmap v297: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:57.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:57 smithi183 ceph-mon[36296]: pgmap v297: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:52:59.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:52:59 smithi192 ceph-mon[36653]: pgmap v298: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:59.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:52:59 smithi076 ceph-mon[32176]: pgmap v298: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:52:59.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:52:59 smithi183 ceph-mon[36296]: pgmap v298: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:01.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:01 smithi192 ceph-mon[36653]: pgmap v299: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:01.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:01 smithi076 ceph-mon[32176]: pgmap v299: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:01.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:01 smithi183 ceph-mon[36296]: pgmap v299: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:03.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:03 smithi192 ceph-mon[36653]: pgmap v300: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:53:03.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:03 smithi076 ceph-mon[32176]: pgmap v300: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:53:03.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:03 smithi183 ceph-mon[36296]: pgmap v300: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:53:05.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:05 smithi192 ceph-mon[36653]: pgmap v301: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:53:05.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:05 smithi076 ceph-mon[32176]: pgmap v301: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:53:05.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:05 smithi183 ceph-mon[36296]: pgmap v301: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:53:07.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:07 smithi192 ceph-mon[36653]: pgmap v302: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:53:07.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:07 smithi076 ceph-mon[32176]: pgmap v302: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:53:07.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:07 smithi183 ceph-mon[36296]: pgmap v302: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:53:09.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:09 smithi192 ceph-mon[36653]: pgmap v303: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:53:09.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:09 smithi192 ceph-mon[36653]: osdmap e95: 12 total, 12 up, 12 in 2024-03-23T15:53:09.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:09 smithi076 ceph-mon[32176]: pgmap v303: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:53:09.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:09 smithi076 ceph-mon[32176]: osdmap e95: 12 total, 12 up, 12 in 2024-03-23T15:53:09.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:09 smithi183 ceph-mon[36296]: pgmap v303: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:53:09.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:09 smithi183 ceph-mon[36296]: osdmap e95: 12 total, 12 up, 12 in 2024-03-23T15:53:11.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:11 smithi192 ceph-mon[36653]: pgmap v305: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:53:11.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:11 smithi076 ceph-mon[32176]: pgmap v305: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:53:11.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:11 smithi183 ceph-mon[36296]: pgmap v305: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-23T15:53:13.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:13 smithi192 ceph-mon[36653]: pgmap v306: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:53:13.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:13 smithi076 ceph-mon[32176]: pgmap v306: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:53:13.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:13 smithi183 ceph-mon[36296]: pgmap v306: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:53:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:14 smithi192 ceph-mon[36653]: osdmap e96: 12 total, 12 up, 12 in 2024-03-23T15:53:15.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:14 smithi192 ceph-mon[36653]: pgmap v308: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-23T15:53:15.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:14 smithi076 ceph-mon[32176]: osdmap e96: 12 total, 12 up, 12 in 2024-03-23T15:53:15.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:14 smithi076 ceph-mon[32176]: pgmap v308: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-23T15:53:15.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:14 smithi183 ceph-mon[36296]: osdmap e96: 12 total, 12 up, 12 in 2024-03-23T15:53:15.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:14 smithi183 ceph-mon[36296]: pgmap v308: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-03-23T15:53:17.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:17 smithi192 ceph-mon[36653]: pgmap v309: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:53:17.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:17 smithi076 ceph-mon[32176]: pgmap v309: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:53:17.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:17 smithi183 ceph-mon[36296]: pgmap v309: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-23T15:53:19.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:19 smithi192 ceph-mon[36653]: pgmap v310: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-03-23T15:53:19.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:19 smithi076 ceph-mon[32176]: pgmap v310: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-03-23T15:53:19.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:19 smithi183 ceph-mon[36296]: pgmap v310: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-03-23T15:53:21.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:21 smithi192 ceph-mon[36653]: pgmap v311: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:53:21.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:21 smithi076 ceph-mon[32176]: pgmap v311: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:53:21.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:21 smithi183 ceph-mon[36296]: pgmap v311: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-23T15:53:23.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:23 smithi192 ceph-mon[36653]: pgmap v312: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:53:23.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:23 smithi076 ceph-mon[32176]: pgmap v312: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:53:23.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:23 smithi183 ceph-mon[36296]: pgmap v312: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-23T15:53:25.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:24 smithi192 ceph-mon[36653]: pgmap v313: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2024-03-23T15:53:25.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:24 smithi076 ceph-mon[32176]: pgmap v313: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2024-03-23T15:53:25.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:24 smithi183 ceph-mon[36296]: pgmap v313: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 100 B/s wr, 6 op/s 2024-03-23T15:53:27.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:27 smithi192 ceph-mon[36653]: pgmap v314: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:53:27.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:27 smithi076 ceph-mon[32176]: pgmap v314: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:53:27.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:27 smithi183 ceph-mon[36296]: pgmap v314: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-23T15:53:28.325 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 -- 172.21.15.76:0/385941456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28069730 msgr2=0x7f2d2810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/385941456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28069730 0x7f2d2810e560 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009cd0 tx=0x7f2d2002c0f0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 -- 172.21.15.76:0/385941456 shutdown_connections 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/385941456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28069730 0x7f2d2810e560 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/385941456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d28068d90 0x7f2d280691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/385941456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d28107780 0x7f2d28107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 -- 172.21.15.76:0/385941456 >> 172.21.15.76:0/385941456 conn(0x7f2d28077460 msgr2=0x7f2d28077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 -- 172.21.15.76:0/385941456 shutdown_connections 2024-03-23T15:53:28.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 -- 172.21.15.76:0/385941456 wait complete. 2024-03-23T15:53:28.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 Processor -- start 2024-03-23T15:53:28.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.437+0000 7f2d2db3a640 1 -- start start 2024-03-23T15:53:28.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d2db3a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28068d90 0x7f2d28197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d2db3a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d28069730 0x7f2d28198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d2db3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d28107780 0x7f2d2819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d2db3a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2819cc10 con 0x7f2d28068d90 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d2db3a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2819cd80 con 0x7f2d28107780 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d2db3a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2819cf40 con 0x7f2d28069730 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28068d90 0x7f2d28197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28068d90 0x7f2d28197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45978/0 (socket says 172.21.15.76:45978) 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 -- 172.21.15.76:0/829854374 learned_addr learned my addr 172.21.15.76:0/829854374 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d1effd640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d28069730 0x7f2d28198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d27fff640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d28107780 0x7f2d2819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 -- 172.21.15.76:0/829854374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d28069730 msgr2=0x7f2d28198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d28069730 0x7f2d28198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 -- 172.21.15.76:0/829854374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d28107780 msgr2=0x7f2d2819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d28107780 0x7f2d2819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 -- 172.21.15.76:0/829854374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d20009900 con 0x7f2d28068d90 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d277fe640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28068d90 0x7f2d28197d50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e9f0 tx=0x7f2d1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d257fa640 1 -- 172.21.15.76:0/829854374 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18009700 con 0x7f2d28068d90 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2819d220 con 0x7f2d28068d90 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d257fa640 1 -- 172.21.15.76:0/829854374 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1800cd80 con 0x7f2d28068d90 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.438+0000 7f2d257fa640 1 -- 172.21.15.76:0/829854374 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18018920 con 0x7f2d28068d90 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.439+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281a4aa0 con 0x7f2d28068d90 2024-03-23T15:53:28.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.440+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2804f470 con 0x7f2d28068d90 2024-03-23T15:53:28.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.441+0000 7f2d257fa640 1 -- 172.21.15.76:0/829854374 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f2d18018a80 con 0x7f2d28068d90 2024-03-23T15:53:28.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.441+0000 7f2d257fa640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2d140a22d0 0x7f2d140a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.441+0000 7f2d1effd640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2d140a22d0 0x7f2d140a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.442+0000 7f2d257fa640 1 -- 172.21.15.76:0/829854374 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f2d18014070 con 0x7f2d28068d90 2024-03-23T15:53:28.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.442+0000 7f2d1effd640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2d140a22d0 0x7f2d140a4790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d28077a30 tx=0x7f2d1000b410 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:28.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.446+0000 7f2d257fa640 1 -- 172.21.15.76:0/829854374 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2d18053280 con 0x7f2d28068d90 2024-03-23T15:53:28.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.641+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2d28111850 con 0x7f2d140a22d0 2024-03-23T15:53:28.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.648+0000 7f2d257fa640 1 -- 172.21.15.76:0/829854374 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f2d28111850 con 0x7f2d140a22d0 2024-03-23T15:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-23T00:00:00", "created": "2024-03-23T15:43:27", "first": "2024-03-23T15:44:00", "last": "2024-03-23T15:53:00", "last_pruned": "2024-03-23T15:53:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-23T15:53:28.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2d140a22d0 msgr2=0x7f2d140a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2d140a22d0 0x7f2d140a4790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d28077a30 tx=0x7f2d1000b410 comp rx=0 tx=0).stop 2024-03-23T15:53:28.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28068d90 msgr2=0x7f2d28197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28068d90 0x7f2d28197d50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e9f0 tx=0x7f2d1800eeb0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 shutdown_connections 2024-03-23T15:53:28.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f2d140a22d0 0x7f2d140a4790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d28107780 0x7f2d2819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d28069730 0x7f2d28198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 --2- 172.21.15.76:0/829854374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d28068d90 0x7f2d28197d50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 >> 172.21.15.76:0/829854374 conn(0x7f2d28077460 msgr2=0x7f2d280ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:28.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 shutdown_connections 2024-03-23T15:53:28.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.650+0000 7f2d2db3a640 1 -- 172.21.15.76:0/829854374 wait complete. 2024-03-23T15:53:28.664 DEBUG:teuthology.orchestra.run.smithi076:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-23T15:53:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 -- 172.21.15.76:0/677770090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 msgr2=0x7f285010d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 --2- 172.21.15.76:0/677770090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285010d440 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2848009ca0 tx=0x7f284802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 -- 172.21.15.76:0/677770090 shutdown_connections 2024-03-23T15:53:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 --2- 172.21.15.76:0/677770090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285010d440 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 --2- 172.21.15.76:0/677770090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2850100860 0x7f2850100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 --2- 172.21.15.76:0/677770090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28501055b0 0x7f2850105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 -- 172.21.15.76:0/677770090 >> 172.21.15.76:0/677770090 conn(0x7f2850077460 msgr2=0x7f2850077870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 -- 172.21.15.76:0/677770090 shutdown_connections 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.764+0000 7f2855f6b640 1 -- 172.21.15.76:0/677770090 wait complete. 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 Processor -- start 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 -- start start 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2850100860 0x7f2850199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28501055b0 0x7f285019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f285019ed90 con 0x7f2850101220 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f285019ef00 con 0x7f28501055b0 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f2855f6b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f285019f0c0 con 0x7f2850100860 2024-03-23T15:53:28.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:45998/0 (socket says 172.21.15.76:45998) 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 -- 172.21.15.76:0/967613318 learned_addr learned my addr 172.21.15.76:0/967613318 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284f7fe640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2850100860 0x7f2850199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284ffff640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28501055b0 0x7f285019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 -- 172.21.15.76:0/967613318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2850100860 msgr2=0x7f2850199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2850100860 0x7f2850199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 -- 172.21.15.76:0/967613318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28501055b0 msgr2=0x7f285019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28501055b0 0x7f285019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.765+0000 7f284effd640 1 -- 172.21.15.76:0/967613318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2848009900 con 0x7f2850101220 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.766+0000 7f284effd640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285019a460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f284400e9f0 tx=0x7f284400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.766+0000 7f284cff9640 1 -- 172.21.15.76:0/967613318 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844009700 con 0x7f2850101220 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.766+0000 7f284cff9640 1 -- 172.21.15.76:0/967613318 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f284400cd80 con 0x7f2850101220 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.766+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285019f3a0 con 0x7f2850101220 2024-03-23T15:53:28.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.766+0000 7f284cff9640 1 -- 172.21.15.76:0/967613318 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844018920 con 0x7f2850101220 2024-03-23T15:53:28.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.766+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28501a6c20 con 0x7f2850101220 2024-03-23T15:53:28.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.767+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285004f470 con 0x7f2850101220 2024-03-23T15:53:28.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.768+0000 7f284cff9640 1 -- 172.21.15.76:0/967613318 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f2844018a80 con 0x7f2850101220 2024-03-23T15:53:28.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.769+0000 7f284cff9640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f28380a22d0 0x7f28380a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:28.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.769+0000 7f284f7fe640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f28380a22d0 0x7f28380a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:28.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.769+0000 7f284cff9640 1 -- 172.21.15.76:0/967613318 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f2844014070 con 0x7f2850101220 2024-03-23T15:53:28.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.769+0000 7f284f7fe640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f28380a22d0 0x7f28380a4790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bd0 tx=0x7f2840005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:28.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.773+0000 7f284cff9640 1 -- 172.21.15.76:0/967613318 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f2844053280 con 0x7f2850101220 2024-03-23T15:53:28.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.968+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 --> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2850110850 con 0x7f28380a22d0 2024-03-23T15:53:28.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.974+0000 7f284cff9640 1 -- 172.21.15.76:0/967613318 <== mgr.15306 v2:172.21.15.76:6800/1113739493 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2850110850 con 0x7f28380a22d0 2024-03-23T15:53:28.978 INFO:teuthology.orchestra.run.smithi076.stdout:/ 1m 6m3h 2024-03-23T15:53:28.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f28380a22d0 msgr2=0x7f28380a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f28380a22d0 0x7f28380a4790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2840009bd0 tx=0x7f2840005f90 comp rx=0 tx=0).stop 2024-03-23T15:53:28.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 msgr2=0x7f285019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:28.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285019a460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f284400e9f0 tx=0x7f284400eeb0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 shutdown_connections 2024-03-23T15:53:28.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f28380a22d0 0x7f28380a4790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28501055b0 0x7f285019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2850101220 0x7f285019a460 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 --2- 172.21.15.76:0/967613318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2850100860 0x7f2850199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:28.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 >> 172.21.15.76:0/967613318 conn(0x7f2850077460 msgr2=0x7f2850068e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:28.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 shutdown_connections 2024-03-23T15:53:28.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:28.976+0000 7f2855f6b640 1 -- 172.21.15.76:0/967613318 wait complete. 2024-03-23T15:53:28.991 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-23T15:53:29.013 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-23T15:53:29.013 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-23T15:53:29.014 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-23T15:53:29.030 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 -- 172.21.15.76:0/650387621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 msgr2=0x7fe6a010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/650387621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a010d4a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe698009cd0 tx=0x7fe69802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 -- 172.21.15.76:0/650387621 shutdown_connections 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/650387621 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a010d4a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/650387621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a01005d0 0x7fe6a0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/650387621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a0106670 0x7fe6a0106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 -- 172.21.15.76:0/650387621 >> 172.21.15.76:0/650387621 conn(0x7fe6a00776b0 msgr2=0x7fe6a0077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 -- 172.21.15.76:0/650387621 shutdown_connections 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.128+0000 7fe6a7b01640 1 -- 172.21.15.76:0/650387621 wait complete. 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 Processor -- start 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 -- start start 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a01005d0 0x7fe6a019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a0106670 0x7fe6a0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0198300 con 0x7fe6a0100f90 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0198470 con 0x7fe6a0106670 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a7b01640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0198630 con 0x7fe6a01005d0 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46012/0 (socket says 172.21.15.76:46012) 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 -- 172.21.15.76:0/3497180596 learned_addr learned my addr 172.21.15.76:0/3497180596 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a589d640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a01005d0 0x7fe6a019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a609e640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a0106670 0x7fe6a0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 -- 172.21.15.76:0/3497180596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a01005d0 msgr2=0x7fe6a019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a01005d0 0x7fe6a019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 -- 172.21.15.76:0/3497180596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a0106670 msgr2=0x7fe6a0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a0106670 0x7fe6a0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.129+0000 7fe6a509c640 1 -- 172.21.15.76:0/3497180596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe698009900 con 0x7fe6a0100f90 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.130+0000 7fe6a509c640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a019dfa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe69000e9f0 tx=0x7fe69000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.130+0000 7fe696ffd640 1 -- 172.21.15.76:0/3497180596 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690009720 con 0x7fe6a0100f90 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.130+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a0198910 con 0x7fe6a0100f90 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.130+0000 7fe696ffd640 1 -- 172.21.15.76:0/3497180596 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69000cd80 con 0x7fe6a0100f90 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.130+0000 7fe696ffd640 1 -- 172.21.15.76:0/3497180596 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690010690 con 0x7fe6a0100f90 2024-03-23T15:53:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.130+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01a49e0 con 0x7fe6a0100f90 2024-03-23T15:53:29.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.131+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a0101d10 con 0x7fe6a0100f90 2024-03-23T15:53:29.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.132+0000 7fe696ffd640 1 -- 172.21.15.76:0/3497180596 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fe6900046c0 con 0x7fe6a0100f90 2024-03-23T15:53:29.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.132+0000 7fe696ffd640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe68c0a2230 0x7fe68c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.133+0000 7fe6a589d640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe68c0a2230 0x7fe68c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.133+0000 7fe696ffd640 1 -- 172.21.15.76:0/3497180596 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe690014070 con 0x7fe6a0100f90 2024-03-23T15:53:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.133+0000 7fe6a589d640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe68c0a2230 0x7fe68c0a46f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0101710 tx=0x7fe69c005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:29.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.137+0000 7fe696ffd640 1 -- 172.21.15.76:0/3497180596 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fe6900970a0 con 0x7fe6a0100f90 2024-03-23T15:53:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.310+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6a0068950 con 0x7fe6a0100f90 2024-03-23T15:53:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.311+0000 7fe696ffd640 1 -- 172.21.15.76:0/3497180596 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23197 (secure 0 0 0) 0x7fe69009a760 con 0x7fe6a0100f90 2024-03-23T15:53:29.316 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:53:29.316 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":96,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:53:13.770262+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/4023925236":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6801/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6800/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/1911361196":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/4249446235":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/3432469273":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:53:29.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe68c0a2230 msgr2=0x7fe68c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe68c0a2230 0x7fe68c0a46f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0101710 tx=0x7fe69c005f90 comp rx=0 tx=0).stop 2024-03-23T15:53:29.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 msgr2=0x7fe6a019dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a019dfa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe69000e9f0 tx=0x7fe69000edf0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 shutdown_connections 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fe68c0a2230 0x7fe68c0a46f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a0106670 0x7fe6a0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a0100f90 0x7fe6a019dfa0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 --2- 172.21.15.76:0/3497180596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6a01005d0 0x7fe6a019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 >> 172.21.15.76:0/3497180596 conn(0x7fe6a00776b0 msgr2=0x7fe6a00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 shutdown_connections 2024-03-23T15:53:29.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.313+0000 7fe6a7b01640 1 -- 172.21.15.76:0/3497180596 wait complete. 2024-03-23T15:53:29.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:29 smithi192 ceph-mon[36653]: pgmap v315: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:29.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:29 smithi192 ceph-mon[36653]: from='client.16173 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:53:29.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:29 smithi076 ceph-mon[32176]: pgmap v315: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:29.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:29 smithi076 ceph-mon[32176]: from='client.16173 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:53:29.328 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:53:29.328 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:53:29.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:29 smithi183 ceph-mon[36296]: pgmap v315: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:29.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:29 smithi183 ceph-mon[36296]: from='client.16173 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:53:29.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.429+0000 7fa3c393e640 1 -- 172.21.15.76:0/3644094105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 msgr2=0x7fa3bc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.429+0000 7fa3c393e640 1 --2- 172.21.15.76:0/3644094105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc100a30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009900 tx=0x7fa3ac02bc30 comp rx=0 tx=0).stop 2024-03-23T15:53:29.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.429+0000 7fa3c393e640 1 -- 172.21.15.76:0/3644094105 shutdown_connections 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.429+0000 7fa3c393e640 1 --2- 172.21.15.76:0/3644094105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3bc100f70 0x7fa3bc10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.429+0000 7fa3c393e640 1 --2- 172.21.15.76:0/3644094105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc100a30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.429+0000 7fa3c393e640 1 --2- 172.21.15.76:0/3644094105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3bc106670 0x7fa3bc106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 -- 172.21.15.76:0/3644094105 >> 172.21.15.76:0/3644094105 conn(0x7fa3bc0777c0 msgr2=0x7fa3bc077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 -- 172.21.15.76:0/3644094105 shutdown_connections 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 -- 172.21.15.76:0/3644094105 wait complete. 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 Processor -- start 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 -- start start 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3bc100f70 0x7fa3bc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3bc106670 0x7fa3bc1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc197000 con 0x7fa3bc1005b0 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc197170 con 0x7fa3bc100f70 2024-03-23T15:53:29.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c393e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3bc197330 con 0x7fa3bc106670 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c16da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c16da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46036/0 (socket says 172.21.15.76:46036) 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.430+0000 7fa3c16da640 1 -- 172.21.15.76:0/633776015 learned_addr learned my addr 172.21.15.76:0/633776015 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c0ed9640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3bc100f70 0x7fa3bc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c1edb640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3bc106670 0x7fa3bc1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c16da640 1 -- 172.21.15.76:0/633776015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3bc106670 msgr2=0x7fa3bc1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c16da640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3bc106670 0x7fa3bc1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c16da640 1 -- 172.21.15.76:0/633776015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3bc100f70 msgr2=0x7fa3bc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c16da640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3bc100f70 0x7fa3bc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c16da640 1 -- 172.21.15.76:0/633776015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac009560 con 0x7fa3bc1005b0 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c16da640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc19c7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800eb90 tx=0x7fa3b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:29.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3b27fc640 1 -- 172.21.15.76:0/633776015 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80099b0 con 0x7fa3bc1005b0 2024-03-23T15:53:29.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3b27fc640 1 -- 172.21.15.76:0/633776015 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b800ee90 con 0x7fa3bc1005b0 2024-03-23T15:53:29.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3bc197610 con 0x7fa3bc1005b0 2024-03-23T15:53:29.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3b27fc640 1 -- 172.21.15.76:0/633776015 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80107a0 con 0x7fa3bc1005b0 2024-03-23T15:53:29.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.431+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3bc1a3850 con 0x7fa3bc1005b0 2024-03-23T15:53:29.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.432+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3bc068500 con 0x7fa3bc1005b0 2024-03-23T15:53:29.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.433+0000 7fa3b27fc640 1 -- 172.21.15.76:0/633776015 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fa3b8005020 con 0x7fa3bc1005b0 2024-03-23T15:53:29.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.433+0000 7fa3b27fc640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3a80a2230 0x7fa3a80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.434+0000 7fa3c0ed9640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3a80a2230 0x7fa3a80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.434+0000 7fa3b27fc640 1 -- 172.21.15.76:0/633776015 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fa3b8014070 con 0x7fa3bc1005b0 2024-03-23T15:53:29.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.434+0000 7fa3c0ed9640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3a80a2230 0x7fa3a80a46f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac006010 tx=0x7fa3ac005e40 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:29.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.438+0000 7fa3b27fc640 1 -- 172.21.15.76:0/633776015 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fa3b8097500 con 0x7fa3bc1005b0 2024-03-23T15:53:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.611+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3bc108a80 con 0x7fa3bc1005b0 2024-03-23T15:53:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.612+0000 7fa3b27fc640 1 -- 172.21.15.76:0/633776015 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fa3b809abc0 con 0x7fa3bc1005b0 2024-03-23T15:53:29.616 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:53:29.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3a80a2230 msgr2=0x7fa3a80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3a80a2230 0x7fa3a80a46f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac006010 tx=0x7fa3ac005e40 comp rx=0 tx=0).stop 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 msgr2=0x7fa3bc19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc19c7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800eb90 tx=0x7fa3b800c6a0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 shutdown_connections 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fa3a80a2230 0x7fa3a80a46f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3bc106670 0x7fa3bc1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3bc100f70 0x7fa3bc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 --2- 172.21.15.76:0/633776015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3bc1005b0 0x7fa3bc19c7b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 >> 172.21.15.76:0/633776015 conn(0x7fa3bc0777c0 msgr2=0x7fa3bc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 shutdown_connections 2024-03-23T15:53:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.614+0000 7fa3c393e640 1 -- 172.21.15.76:0/633776015 wait complete. 2024-03-23T15:53:29.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 -- 172.21.15.76:0/4080765368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1005b0 msgr2=0x7f9f7c10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4080765368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1005b0 0x7f9f7c10e590 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009cd0 tx=0x7f9f7402c0f0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 -- 172.21.15.76:0/4080765368 shutdown_connections 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4080765368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1005b0 0x7f9f7c10e590 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4080765368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 0x7f9f7c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4080765368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f7c1076d0 0x7f9f7c107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 -- 172.21.15.76:0/4080765368 >> 172.21.15.76:0/4080765368 conn(0x7f9f7c076bd0 msgr2=0x7f9f7c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 -- 172.21.15.76:0/4080765368 shutdown_connections 2024-03-23T15:53:29.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 -- 172.21.15.76:0/4080765368 wait complete. 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.729+0000 7f9f839c7640 1 Processor -- start 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f839c7640 1 -- start start 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f839c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 0x7f9f7c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f839c7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f7c1005b0 0x7f9f7c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f839c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1076d0 0x7f9f7c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f839c7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c198140 con 0x7f9f7c1076d0 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f839c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c1982b0 con 0x7f9f7c0ffc10 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f839c7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c198470 con 0x7f9f7c1005b0 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1076d0 0x7f9f7c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81763640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 0x7f9f7c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81763640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 0x7f9f7c19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.76:41636/0 (socket says 172.21.15.76:41636) 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81763640 1 -- 172.21.15.76:0/4095774104 learned_addr learned my addr 172.21.15.76:0/4095774104 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1076d0 0x7f9f7c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46050/0 (socket says 172.21.15.76:46050) 2024-03-23T15:53:29.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f80f62640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f7c1005b0 0x7f9f7c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 -- 172.21.15.76:0/4095774104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f7c1005b0 msgr2=0x7f9f7c19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f7c1005b0 0x7f9f7c19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 -- 172.21.15.76:0/4095774104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 msgr2=0x7f9f7c19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 0x7f9f7c19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 -- 172.21.15.76:0/4095774104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c009490 con 0x7f9f7c1076d0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81763640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 0x7f9f7c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.730+0000 7f9f81f64640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1076d0 0x7f9f7c197af0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f7402c630 tx=0x7f9f74034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.731+0000 7f9f727fc640 1 -- 172.21.15.76:0/4095774104 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740328b0 con 0x7f9f7c1076d0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.731+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f74009900 con 0x7f9f7c1076d0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.731+0000 7f9f727fc640 1 -- 172.21.15.76:0/4095774104 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f74003e50 con 0x7f9f7c1076d0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.731+0000 7f9f727fc640 1 -- 172.21.15.76:0/4095774104 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74004860 con 0x7f9f7c1076d0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.731+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c1a4800 con 0x7f9f7c1076d0 2024-03-23T15:53:29.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.731+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c068950 con 0x7f9f7c1076d0 2024-03-23T15:53:29.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.733+0000 7f9f727fc640 1 -- 172.21.15.76:0/4095774104 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f9f7400a040 con 0x7f9f7c1076d0 2024-03-23T15:53:29.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.733+0000 7f9f727fc640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9f680a2230 0x7f9f680a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:29.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.733+0000 7f9f81763640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9f680a2230 0x7f9f680a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:29.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.733+0000 7f9f727fc640 1 -- 172.21.15.76:0/4095774104 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f9f740f0c70 con 0x7f9f7c1076d0 2024-03-23T15:53:29.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.734+0000 7f9f81763640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9f680a2230 0x7f9f680a46f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f78011490 tx=0x7f9f78005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:29.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.737+0000 7f9f727fc640 1 -- 172.21.15.76:0/4095774104 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f9f740b99e0 con 0x7f9f7c1076d0 2024-03-23T15:53:29.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.910+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f7c109c30 con 0x7f9f7c1076d0 2024-03-23T15:53:29.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.911+0000 7f9f727fc640 1 -- 172.21.15.76:0/4095774104 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f9f740bd0a0 con 0x7f9f7c1076d0 2024-03-23T15:53:29.914 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:53:29.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.912+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9f680a2230 msgr2=0x7f9f680a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.912+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9f680a2230 0x7f9f680a46f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f78011490 tx=0x7f9f78005f90 comp rx=0 tx=0).stop 2024-03-23T15:53:29.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.912+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1076d0 msgr2=0x7f9f7c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.912+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1076d0 0x7f9f7c197af0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f7402c630 tx=0x7f9f74034a90 comp rx=0 tx=0).stop 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 shutdown_connections 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f9f680a2230 0x7f9f680a46f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f7c1076d0 0x7f9f7c197af0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f7c1005b0 0x7f9f7c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 --2- 172.21.15.76:0/4095774104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c0ffc10 0x7f9f7c19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 >> 172.21.15.76:0/4095774104 conn(0x7f9f7c076bd0 msgr2=0x7f9f7c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 shutdown_connections 2024-03-23T15:53:29.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:29.913+0000 7f9f839c7640 1 -- 172.21.15.76:0/4095774104 wait complete. 2024-03-23T15:53:29.927 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:53:30.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 -- 172.21.15.76:0/914510570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c106670 msgr2=0x7f6b7c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 --2- 172.21.15.76:0/914510570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c106670 0x7f6b7c106a50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009900 tx=0x7f6b7002bc30 comp rx=0 tx=0).stop 2024-03-23T15:53:30.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 -- 172.21.15.76:0/914510570 shutdown_connections 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 --2- 172.21.15.76:0/914510570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c100f70 0x7f6b7c10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 --2- 172.21.15.76:0/914510570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b7c1005b0 0x7f6b7c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 --2- 172.21.15.76:0/914510570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c106670 0x7f6b7c106a50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 -- 172.21.15.76:0/914510570 >> 172.21.15.76:0/914510570 conn(0x7f6b7c0777c0 msgr2=0x7f6b7c077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.028+0000 7f6b82902640 1 -- 172.21.15.76:0/914510570 shutdown_connections 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 -- 172.21.15.76:0/914510570 wait complete. 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 Processor -- start 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 -- start start 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b7c1005b0 0x7f6b7c196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c100f70 0x7f6b7c196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106670 0x7f6b7c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1a6080 con 0x7f6b7c100f70 2024-03-23T15:53:30.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1a61f0 con 0x7f6b7c106670 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.029+0000 7f6b82902640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1a63b0 con 0x7f6b7c1005b0 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c100f70 0x7f6b7c196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c100f70 0x7f6b7c196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46058/0 (socket says 172.21.15.76:46058) 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 -- 172.21.15.76:0/3512114649 learned_addr learned my addr 172.21.15.76:0/3512114649 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7bfff640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b7c1005b0 0x7f6b7c196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b80e9f640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106670 0x7f6b7c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 -- 172.21.15.76:0/3512114649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b7c1005b0 msgr2=0x7f6b7c196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b7c1005b0 0x7f6b7c196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 -- 172.21.15.76:0/3512114649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106670 msgr2=0x7f6b7c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106670 0x7f6b7c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 -- 172.21.15.76:0/3512114649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b70009560 con 0x7f6b7c100f70 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b7b7fe640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c100f70 0x7f6b7c196e80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e9f0 tx=0x7f6b6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b797fa640 1 -- 172.21.15.76:0/3512114649 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c009720 con 0x7f6b7c100f70 2024-03-23T15:53:30.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c1a6690 con 0x7f6b7c100f70 2024-03-23T15:53:30.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b797fa640 1 -- 172.21.15.76:0/3512114649 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b6c00cd80 con 0x7f6b7c100f70 2024-03-23T15:53:30.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b797fa640 1 -- 172.21.15.76:0/3512114649 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c010690 con 0x7f6b7c100f70 2024-03-23T15:53:30.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.030+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c1a6ba0 con 0x7f6b7c100f70 2024-03-23T15:53:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.031+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c068500 con 0x7f6b7c100f70 2024-03-23T15:53:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.032+0000 7f6b797fa640 1 -- 172.21.15.76:0/3512114649 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f6b6c0046c0 con 0x7f6b7c100f70 2024-03-23T15:53:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.033+0000 7f6b797fa640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6b640a2230 0x7f6b640a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.033+0000 7f6b7bfff640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6b640a2230 0x7f6b640a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.033+0000 7f6b797fa640 1 -- 172.21.15.76:0/3512114649 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f6b6c014070 con 0x7f6b7c100f70 2024-03-23T15:53:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.033+0000 7f6b7bfff640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6b640a2230 0x7f6b640a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b70006010 tx=0x7f6b70005e40 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:30.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.037+0000 7f6b797fa640 1 -- 172.21.15.76:0/3512114649 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6b6c0970b0 con 0x7f6b7c100f70 2024-03-23T15:53:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.212+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6b7c108ac0 con 0x7f6b7c100f70 2024-03-23T15:53:30.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.212+0000 7f6b797fa640 1 -- 172.21.15.76:0/3512114649 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f6b6c09a770 con 0x7f6b7c100f70 2024-03-23T15:53:30.216 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6b640a2230 msgr2=0x7f6b640a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6b640a2230 0x7f6b640a46f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b70006010 tx=0x7f6b70005e40 comp rx=0 tx=0).stop 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c100f70 msgr2=0x7f6b7c196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c100f70 0x7f6b7c196e80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e9f0 tx=0x7f6b6c00edf0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 shutdown_connections 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6b640a2230 0x7f6b640a46f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b7c106670 0x7f6b7c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b7c100f70 0x7f6b7c196e80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 --2- 172.21.15.76:0/3512114649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b7c1005b0 0x7f6b7c196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 >> 172.21.15.76:0/3512114649 conn(0x7f6b7c0777c0 msgr2=0x7f6b7c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:30.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 shutdown_connections 2024-03-23T15:53:30.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.214+0000 7f6b82902640 1 -- 172.21.15.76:0/3512114649 wait complete. 2024-03-23T15:53:30.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:30 smithi076 ceph-mon[32176]: from='client.16179 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:53:30.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:30 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3497180596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:53:30.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:30 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/633776015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:53:30.227 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:30 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/4095774104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:53:30.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-23T15:53:30.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:30 smithi192 ceph-mon[36653]: from='client.16179 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:53:30.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:30 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3497180596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:53:30.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:30 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/633776015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:53:30.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:30 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/4095774104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:53:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.326+0000 7f0fea64e640 1 -- 172.21.15.76:0/3434500255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4100d40 msgr2=0x7f0fe410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.326+0000 7f0fea64e640 1 --2- 172.21.15.76:0/3434500255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4100d40 0x7f0fe410d4e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009be0 tx=0x7f0fdc02bf80 comp rx=0 tx=0).stop 2024-03-23T15:53:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.326+0000 7f0fea64e640 1 -- 172.21.15.76:0/3434500255 shutdown_connections 2024-03-23T15:53:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.326+0000 7f0fea64e640 1 --2- 172.21.15.76:0/3434500255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4100d40 0x7f0fe410d4e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.326+0000 7f0fea64e640 1 --2- 172.21.15.76:0/3434500255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe4106fd0 0x7f0fe4100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.326+0000 7f0fea64e640 1 --2- 172.21.15.76:0/3434500255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe41066b0 0x7f0fe4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 -- 172.21.15.76:0/3434500255 >> 172.21.15.76:0/3434500255 conn(0x7f0fe4077710 msgr2=0x7f0fe4077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 -- 172.21.15.76:0/3434500255 shutdown_connections 2024-03-23T15:53:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 -- 172.21.15.76:0/3434500255 wait complete. 2024-03-23T15:53:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 Processor -- start 2024-03-23T15:53:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 -- start start 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe4100d40 0x7f0fe419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe41066b0 0x7f0fe419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4106fd0 0x7f0fe4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe41970a0 con 0x7f0fe4106fd0 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4197210 con 0x7f0fe41066b0 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.327+0000 7f0fea64e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe41973d0 con 0x7f0fe4100d40 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4106fd0 0x7f0fe4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4106fd0 0x7f0fe4196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46076/0 (socket says 172.21.15.76:46076) 2024-03-23T15:53:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe41066b0 0x7f0fe419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 -- 172.21.15.76:0/320917853 learned_addr learned my addr 172.21.15.76:0/320917853 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe3fff640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe4100d40 0x7f0fe419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 -- 172.21.15.76:0/320917853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe4100d40 msgr2=0x7f0fe419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe4100d40 0x7f0fe419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 -- 172.21.15.76:0/320917853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe41066b0 msgr2=0x7f0fe419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe41066b0 0x7f0fe419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 -- 172.21.15.76:0/320917853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd4009560 con 0x7f0fe4106fd0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe8beb640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4106fd0 0x7f0fe4196a50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc036040 tx=0x7f0fdc033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe17fa640 1 -- 172.21.15.76:0/320917853 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc0338a0 con 0x7f0fe4106fd0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe17fa640 1 -- 172.21.15.76:0/320917853 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fdc003b80 con 0x7f0fe4106fd0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fe17fa640 1 -- 172.21.15.76:0/320917853 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc034650 con 0x7f0fe4106fd0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fdc009840 con 0x7f0fe4106fd0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.328+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe41a3760 con 0x7f0fe4106fd0 2024-03-23T15:53:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.329+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe4068500 con 0x7f0fe4106fd0 2024-03-23T15:53:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.330+0000 7f0fe17fa640 1 -- 172.21.15.76:0/320917853 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f0fdc0347b0 con 0x7f0fe4106fd0 2024-03-23T15:53:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.331+0000 7f0fe17fa640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f0fcc0a22b0 0x7f0fcc0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:53:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.331+0000 7f0fe3fff640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f0fcc0a22b0 0x7f0fcc0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:53:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.331+0000 7f0fe17fa640 1 -- 172.21.15.76:0/320917853 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f0fdc0f0a50 con 0x7f0fe4106fd0 2024-03-23T15:53:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.331+0000 7f0fe3fff640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f0fcc0a22b0 0x7f0fcc0a4770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4197b90 tx=0x7f0fd4009260 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:53:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.335+0000 7f0fe17fa640 1 -- 172.21.15.76:0/320917853 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f0fdc0b97c0 con 0x7f0fe4106fd0 2024-03-23T15:53:30.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:30 smithi183 ceph-mon[36296]: from='client.16179 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T15:53:30.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:30 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3497180596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:53:30.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:30 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/633776015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:53:30.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:30 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/4095774104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:53:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.555+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f0fe4108ba0 con 0x7f0fe4106fd0 2024-03-23T15:53:30.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.563+0000 7f0fe17fa640 1 -- 172.21.15.76:0/320917853 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+736928 (secure 0 0 0) 0x7f0fdc0bce80 con 0x7f0fe4106fd0 2024-03-23T15:53:30.577 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:53:30.577 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:53:30.577 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 25, 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "active_gid": 15306, 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "active_name": "x", 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "active_addrs": { 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1113739493 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-03-23T15:53:30.578 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1113739493 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: "active_addr": "172.21.15.76:6801/1113739493", 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: "active_change": "2024-03-23T15:43:19.788519+0000", 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: "active_mgr_features": 4540701547738038271, 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: "available": true, 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: "standbys": [ 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.579 INFO:teuthology.orchestra.run.smithi076.stdout: "gid": 25091, 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "y", 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_features": 4540701547738038271, 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "available_modules": [ 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "alerts", 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.580 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.581 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.582 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.583 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.584 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.585 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_destination": { 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_destination", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.586 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Email address to send alerts to", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_from_name": { 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_from_name", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "Ceph", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.587 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Email From: name", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_host": { 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_host", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.588 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP server", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_password": { 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_password", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.589 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Password to authenticate with", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_port": { 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_port", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "465", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.590 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP port", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_sender": { 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_sender", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP envelope sender", 2024-03-23T15:53:30.591 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_ssl": { 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_ssl", 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_user": { 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_user", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.593 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "User to authenticate as", 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "balancer", 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.594 INFO:teuthology.orchestra.run.smithi076.stdout: "active": { 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "active", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.595 INFO:teuthology.orchestra.run.smithi076.stdout: "begin_time": { 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "begin_time", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0000", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.596 INFO:teuthology.orchestra.run.smithi076.stdout: "begin_weekday": { 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "begin_weekday", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "6", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.597 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_max_iterations": { 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_max_iterations", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "25", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "250", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.598 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_metrics": { 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_metrics", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_step": { 2024-03-23T15:53:30.599 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_step", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0.5", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0.001", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "0.999", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "aggressiveness of optimization", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "end_time": { 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "end_time", 2024-03-23T15:53:30.600 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "2359", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "end_weekday": { 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "end_weekday", 2024-03-23T15:53:30.601 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "6", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.602 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.603 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.604 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.605 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.606 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "min_score": { 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "min_score", 2024-03-23T15:53:30.607 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": { 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mode", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.608 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "upmap", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-compat", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "none", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "read", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap-read" 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Balancer mode", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.609 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_ids": { 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pool_ids", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.610 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.611 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap_max_deviation": { 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "upmap_max_deviation", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap_max_optimizations": { 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "upmap_max_optimizations", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.613 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cephadm", 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_down_multiplier": { 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_down_multiplier", 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3.0", 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.614 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_refresh_rate": { 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_refresh_rate", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "20", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.615 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_starting_port": { 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_starting_port", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "4721", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.616 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_ptrace": { 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_ptrace", 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.617 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "autotune_interval": { 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "autotune_interval", 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:30.618 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "autotune_memory_target_ratio": { 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0.7", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.619 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm_log_destination": { 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cephadm_log_destination", 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.620 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "file", 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "file,syslog", 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "syslog" 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.621 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "cgroups_split": { 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cgroups_split", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.622 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "config_checks_enabled": { 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_checks_enabled", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.623 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "config_dashboard": { 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_dashboard", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_alertmanager": { 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_alertmanager", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_base": { 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_base", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Container image name, without the tag", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_elasticsearch": { 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_elasticsearch", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "elasticsearch container image", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.627 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_grafana": { 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_grafana", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_haproxy": { 2024-03-23T15:53:30.628 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_haproxy", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "HAproxy container image", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.629 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_agent": { 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_agent", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger agent container image", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.630 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_collector": { 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_collector", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger collector container image", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_query": { 2024-03-23T15:53:30.631 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_query", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger query container image", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_keepalived": { 2024-03-23T15:53:30.632 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_keepalived", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Keepalived container image", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_loki": { 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_loki", 2024-03-23T15:53:30.633 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Loki container image", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_node_exporter": { 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_node_exporter", 2024-03-23T15:53:30.634 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_nvmeof": { 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_nvmeof", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.635 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Nvme-of container image", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_prometheus": { 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_prometheus", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.636 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_promtail": { 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_promtail", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Promtail container image", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_snmp_gateway": { 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_snmp_gateway", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.638 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SNMP Gateway container image", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "container_init": { 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_init", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.639 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "daemon_cache_timeout": { 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "daemon_cache_timeout", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:30.640 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "default_cephadm_command_timeout": { 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "900", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "default_registry": { 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "default_registry", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.642 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "device_cache_timeout": { 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_cache_timeout", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.643 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1800", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache device inventory", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "device_enhanced_scan": { 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_enhanced_scan", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.644 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "facts_cache_timeout": { 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "facts_cache_timeout", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:30.645 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache host facts data", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "host_check_interval": { 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "host_check_interval", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.646 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to perform a host check", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "hw_monitoring": { 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "hw_monitoring", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.647 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "inventory_list_all": { 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "inventory_list_all", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.648 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.649 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "log_refresh_metadata": { 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_refresh_metadata", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.650 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.651 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.652 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.653 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.654 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.655 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "*", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.656 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "max_count_per_host": { 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_count_per_host", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.657 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "max number of daemons per service per host", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd_draining_count": { 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_osd_draining_count", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.658 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "migration_current": { 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "migration_current", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.659 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "internal - do not modify", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": { 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mode", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "root", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm-package", 2024-03-23T15:53:30.660 INFO:teuthology.orchestra.run.smithi076.stdout: "root" 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "oob_default_addr": { 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "oob_default_addr", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "169.254.1.1", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.661 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "prometheus_alerts_path": { 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prometheus_alerts_path", 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.662 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_insecure": { 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_insecure", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.663 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_password": { 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_password", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.664 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_url": { 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_url", 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.665 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_username": { 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_username", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.666 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "secure_monitoring_stack": { 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "secure_monitoring_stack", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.667 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "service_discovery_port": { 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "service_discovery_port", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8765", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.668 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "cephadm service discovery port", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "ssh_config_file": { 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssh_config_file", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T15:53:30.669 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "use_agent": { 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "use_agent", 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.670 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "use_repo_digest": { 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "use_repo_digest", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T15:53:30.671 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_failed_host_check": { 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_failed_host_check", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.672 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_stray_daemons": { 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_stray_daemons", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.673 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_stray_hosts": { 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_stray_hosts", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.674 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crash", 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.675 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.676 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.677 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.678 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.679 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "retain_interval": { 2024-03-23T15:53:30.680 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "retain_interval", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "31536000", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_recent_interval": { 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_recent_interval", 2024-03-23T15:53:30.681 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1209600", 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "dashboard", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.683 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.684 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.685 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "AUDIT_API_ENABLED": { 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.686 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.687 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.688 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.691 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.692 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.696 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.697 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_URL": { 2024-03-23T15:53:30.698 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_URL", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T15:53:30.699 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T15:53:30.700 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.701 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T15:53:30.702 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.703 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T15:53:30.704 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.705 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.706 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.707 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.708 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.709 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.711 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.712 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.713 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.714 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.715 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8", 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.716 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "45", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.717 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.718 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.719 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.720 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.721 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.722 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.723 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.724 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "cross_origin_url": { 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cross_origin_url", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.725 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "crt_file": { 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crt_file", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.726 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "debug": { 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "debug", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable/disable debug options", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.727 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "jwt_token_ttl": { 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "jwt_token_ttl", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "28800", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "key_file": { 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "key_file", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.729 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.730 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.731 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.732 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.733 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.734 INFO:teuthology.orchestra.run.smithi076.stdout: "motd": { 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "motd", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "The message of the day", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "redirect_resolve_ip_addr": { 2024-03-23T15:53:30.735 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:30.736 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "::", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:30.737 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8080", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl": { 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl", 2024-03-23T15:53:30.738 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl_server_port": { 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl_server_port", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.739 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8443", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_behaviour": { 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_behaviour", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.740 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "redirect", 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "redirect" 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_error_status_code": { 2024-03-23T15:53:30.741 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_error_status_code", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "500", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "400", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "599", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.742 INFO:teuthology.orchestra.run.smithi076.stdout: "url_prefix": { 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "url_prefix", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "devicehealth", 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "enable_monitoring": { 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enable_monitoring", 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.744 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "monitor device health metrics", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.745 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.746 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.747 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.748 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.749 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "mark_out_threshold": { 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mark_out_threshold", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "2419200", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.750 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": { 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pool_name", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "device_health_metrics", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.751 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "retention_period": { 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "retention_period", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15552000", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.752 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long to retain device health metrics", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "scrape_frequency": { 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "scrape_frequency", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "86400", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.753 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "self_heal": { 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "self_heal", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.754 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.755 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_threshold": { 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_threshold", 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "7257600", 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.756 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "diskprediction_local", 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.757 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.758 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.759 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.760 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.761 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "predict_interval": { 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "predict_interval", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.762 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "86400", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "predictor_model": { 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "predictor_model", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.763 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "prophetstor", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.764 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "influx", 2024-03-23T15:53:30.765 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": false, 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "influxdb python module not found", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "batch_size": { 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "batch_size", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5000", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.766 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "database": { 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "database", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "ceph", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.767 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": { 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "hostname", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB server hostname", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "30", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "5", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.769 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.770 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.771 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.772 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "password": { 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "password", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "password of InfluxDB server user", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.775 INFO:teuthology.orchestra.run.smithi076.stdout: "port": { 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "port", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8086", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB server port", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.776 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl": { 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "false", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "threads": { 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "threads", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "32", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "username": { 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "username", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "username of InfluxDB server user", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.779 INFO:teuthology.orchestra.run.smithi076.stdout: "verify_ssl": { 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "verify_ssl", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "true", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "insights", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.781 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.782 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.783 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.784 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.785 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "iostat", 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.786 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.787 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.788 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.789 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.790 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "k8sevents", 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.791 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_event_retention_days": { 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ceph_event_retention_days", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "7", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "config_check_secs": { 2024-03-23T15:53:30.792 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_check_secs", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "10", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.793 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.794 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.795 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.796 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.797 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "localpool", 2024-03-23T15:53:30.798 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "failure_domain": { 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "failure_domain", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "host", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "failure domain for any created local pool", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.799 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.800 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.801 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.802 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.803 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": { 2024-03-23T15:53:30.804 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "min_size", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default min_size for any created local pool", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "num_rep": { 2024-03-23T15:53:30.805 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "num_rep", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default replica count for any created local pool", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": { 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pg_num", 2024-03-23T15:53:30.806 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "128", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "prefix": { 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prefix", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.807 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "name prefix for any created local pool", 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "subtree": { 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "subtree", 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.808 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "rack", 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mds_autoscaler", 2024-03-23T15:53:30.809 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.810 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.811 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.812 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.813 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.814 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mirroring", 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.815 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.816 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.817 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.818 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.819 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "nfs", 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.820 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.821 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.822 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.823 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.824 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "orchestrator", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "fail_fs": { 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "fail_fs", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.826 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.827 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.828 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.829 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.830 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.831 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator": { 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "orchestrator", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "rook", 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: "test_orchestrator" 2024-03-23T15:53:30.832 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Orchestrator backend", 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "osd_perf_query", 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.833 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.834 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.835 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.836 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.837 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "osd_support", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.839 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.840 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.841 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.842 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.843 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pg_autoscaler", 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.844 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.845 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.846 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.847 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.848 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.849 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "threshold": { 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "threshold", 2024-03-23T15:53:30.850 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3.0", 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1.0", 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "scaling threshold", 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "progress", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_pg_recovery_event": { 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_pg_recovery_event", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.852 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "enabled": { 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enabled", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.853 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.854 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.855 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.856 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.857 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.858 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "max_completed_events": { 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_completed_events", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "50", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "number of past completed events to remember", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.859 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long the module is going to sleep", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.860 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prometheus", 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "cache": { 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cache", 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.861 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "exclude_perf_counters": { 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "exclude_perf_counters", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.862 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.863 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.864 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.865 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.866 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.867 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_stats_pools": { 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_stats_pools", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.868 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "300", 2024-03-23T15:53:30.869 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "scrape_interval": { 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "scrape_interval", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15.0", 2024-03-23T15:53:30.870 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "::", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.871 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "9283", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.872 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "stale_cache_strategy": { 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "stale_cache_strategy", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "log", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.873 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_behaviour": { 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_behaviour", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "default", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.874 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "default", 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "error" 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_error_status_code": { 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_error_status_code", 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "500", 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "400", 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "599", 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_support", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.877 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.878 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.880 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.881 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "max_concurrent_snap_create": { 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_concurrent_snap_create", 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.882 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "mirror_snapshot_schedule": { 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.883 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "trash_purge_schedule": { 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "trash_purge_schedule", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "restful", 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: "enable_auth": { 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enable_auth", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "key_file": { 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "key_file", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.888 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.889 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.890 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.891 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.894 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rgw", 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.895 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.896 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rook", 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "storage_class": { 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "storage_class", 2024-03-23T15:53:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "local", 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "selftest", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "roption1": { 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "roption1", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "roption2": { 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "roption2", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "xyz", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption1": { 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption1", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption2": { 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption2", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption3": { 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption3", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption4": { 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption4", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption5": { 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption5", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption6": { 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption6", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption7": { 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption7", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "42", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "testkey": { 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testkey", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "testlkey": { 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testlkey", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "testnewline": { 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testnewline", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snap_schedule", 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_m_granularity": { 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_m_granularity", 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "dump_on_update": { 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "dump_on_update", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "dump database to debug log on update", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "stats", 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "status", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "telegraf", 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "address": { 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "address", 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "telemetry", 2024-03-23T15:53:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_basic": { 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_basic", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T15:53:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_crash": { 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_crash", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_device": { 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_device", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_ident": { 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_ident", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_perf": { 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_perf", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "contact": { 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "contact", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "description": { 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "description", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "device_url": { 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_url", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "enabled": { 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enabled", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "24", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "8", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "last_opt_revision": { 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "last_opt_revision", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "leaderboard": { 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "leaderboard", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "leaderboard_description": { 2024-03-23T15:53:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "leaderboard_description", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "organization": { 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "organization", 2024-03-23T15:53:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "proxy": { 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "proxy", 2024-03-23T15:53:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "url": { 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "url", 2024-03-23T15:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "test_orchestrator", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "volumes", 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "max_concurrent_clones": { 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_concurrent_clones", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "4", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "periodic_async_work": { 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "periodic_async_work", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Periodically check for async work", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "snapshot_clone_delay": { 2024-03-23T15:53:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snapshot_clone_delay", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "snapshot_clone_no_wait": { 2024-03-23T15:53:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix", 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "discovery_interval": { 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "discovery_interval", 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "100", 2024-03-23T15:53:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "identifier": { 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "identifier", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_host": { 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_host", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_port": { 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_port", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10051", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_sender": { 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_sender", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "gid": 25105, 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "z", 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_features": 4540701547738038271, 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "available_modules": [ 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "alerts", 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:30.997 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:30.998 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:30.999 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.000 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.001 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.002 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.003 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_destination": { 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_destination", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Email address to send alerts to", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.004 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_from_name": { 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_from_name", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "Ceph", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Email From: name", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_host": { 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_host", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP server", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.006 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_password": { 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_password", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Password to authenticate with", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_port": { 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_port", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "465", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP port", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.008 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_sender": { 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_sender", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP envelope sender", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_ssl": { 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_ssl", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_user": { 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_user", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "User to authenticate as", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "balancer", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "active": { 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "active", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.012 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "begin_time": { 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "begin_time", 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0000", 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.013 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "begin_weekday": { 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "begin_weekday", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0", 2024-03-23T15:53:31.014 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "6", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_max_iterations": { 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_max_iterations", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "25", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.015 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "250", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_metrics": { 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_metrics", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.016 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_step": { 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_step", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0.5", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0.001", 2024-03-23T15:53:31.017 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "0.999", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "aggressiveness of optimization", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "end_time": { 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "end_time", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "2359", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.018 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "end_weekday": { 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "end_weekday", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "6", 2024-03-23T15:53:31.019 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.020 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.021 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.022 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.023 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.024 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "min_score": { 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "min_score", 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T15:53:31.025 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": { 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mode", 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "upmap", 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.026 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-compat", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "none", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "read", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap-read" 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Balancer mode", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_ids": { 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pool_ids", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.027 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.028 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap_max_deviation": { 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "upmap_max_deviation", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.029 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap_max_optimizations": { 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "upmap_max_optimizations", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.030 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cephadm", 2024-03-23T15:53:31.031 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_down_multiplier": { 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_down_multiplier", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3.0", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.032 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_refresh_rate": { 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_refresh_rate", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "20", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.033 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_starting_port": { 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_starting_port", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "4721", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_ptrace": { 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_ptrace", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "autotune_interval": { 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "autotune_interval", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "autotune_memory_target_ratio": { 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0.7", 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T15:53:31.037 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm_log_destination": { 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cephadm_log_destination", 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.038 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "file", 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "file,syslog", 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "syslog" 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "cgroups_split": { 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cgroups_split", 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.039 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "config_checks_enabled": { 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_checks_enabled", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.040 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "config_dashboard": { 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_dashboard", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.041 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_alertmanager": { 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_alertmanager", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.042 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_base": { 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_base", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.043 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Container image name, without the tag", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_elasticsearch": { 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_elasticsearch", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T15:53:31.044 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "elasticsearch container image", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_grafana": { 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_grafana", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.045 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_haproxy": { 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_haproxy", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.046 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "HAproxy container image", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_agent": { 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_agent", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.047 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger agent container image", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_collector": { 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_collector", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.048 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger collector container image", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_query": { 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_query", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.049 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger query container image", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_keepalived": { 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_keepalived", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.050 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Keepalived container image", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_loki": { 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_loki", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Loki container image", 2024-03-23T15:53:31.051 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_node_exporter": { 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_node_exporter", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.052 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_nvmeof": { 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_nvmeof", 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.053 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Nvme-of container image", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_prometheus": { 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_prometheus", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.054 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_promtail": { 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_promtail", 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.055 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Promtail container image", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_snmp_gateway": { 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_snmp_gateway", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.056 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SNMP Gateway container image", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "container_init": { 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_init", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.057 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "daemon_cache_timeout": { 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "daemon_cache_timeout", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.058 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "default_cephadm_command_timeout": { 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "900", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.059 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "default_registry": { 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "default_registry", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.060 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "device_cache_timeout": { 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_cache_timeout", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1800", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.061 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache device inventory", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "device_enhanced_scan": { 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_enhanced_scan", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.062 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "facts_cache_timeout": { 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "facts_cache_timeout", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.063 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache host facts data", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "host_check_interval": { 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "host_check_interval", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.064 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to perform a host check", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "hw_monitoring": { 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "hw_monitoring", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.065 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "inventory_list_all": { 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "inventory_list_all", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.066 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.067 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "log_refresh_metadata": { 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_refresh_metadata", 2024-03-23T15:53:31.068 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.069 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.070 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.071 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.072 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T15:53:31.073 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "*", 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.074 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "max_count_per_host": { 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_count_per_host", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "max number of daemons per service per host", 2024-03-23T15:53:31.075 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd_draining_count": { 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_osd_draining_count", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T15:53:31.076 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "migration_current": { 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "migration_current", 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.077 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "internal - do not modify", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": { 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mode", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "root", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.078 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm-package", 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "root" 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "oob_default_addr": { 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "oob_default_addr", 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "169.254.1.1", 2024-03-23T15:53:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "prometheus_alerts_path": { 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prometheus_alerts_path", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T15:53:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_insecure": { 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_insecure", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.081 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_password": { 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_password", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.082 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_url": { 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_url", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_username": { 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_username", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.084 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "secure_monitoring_stack": { 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "secure_monitoring_stack", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.085 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "service_discovery_port": { 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "service_discovery_port", 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.086 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8765", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "cephadm service discovery port", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "ssh_config_file": { 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssh_config_file", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.087 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "use_agent": { 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "use_agent", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.088 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "use_repo_digest": { 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "use_repo_digest", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.089 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_failed_host_check": { 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_failed_host_check", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.090 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_stray_daemons": { 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_stray_daemons", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.091 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_stray_hosts": { 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_stray_hosts", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.092 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crash", 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.093 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.094 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.095 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.096 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.097 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.098 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "retain_interval": { 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "retain_interval", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "31536000", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.099 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_recent_interval": { 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_recent_interval", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1209600", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.100 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "dashboard", 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.101 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.102 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.103 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "AUDIT_API_ENABLED": { 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.104 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.105 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.106 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.107 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.108 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.109 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.110 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.111 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.112 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.113 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.114 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.115 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.116 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_URL": { 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_URL", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.117 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.118 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.119 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.120 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.121 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.122 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.123 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.124 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.125 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.126 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.127 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.128 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.129 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.130 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.131 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.132 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.133 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.134 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.135 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "45", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.136 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.137 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.138 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.139 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.140 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.141 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.142 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.143 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "cross_origin_url": { 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cross_origin_url", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.144 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "crt_file": { 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crt_file", 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.145 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "debug": { 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "debug", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable/disable debug options", 2024-03-23T15:53:31.146 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "jwt_token_ttl": { 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "jwt_token_ttl", 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "28800", 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.147 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "key_file": { 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "key_file", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.148 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.149 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.150 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.151 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.152 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.153 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "motd": { 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "motd", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.154 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "The message of the day", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "redirect_resolve_ip_addr": { 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.155 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "::", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.156 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8080", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.157 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl": { 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.158 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl_server_port": { 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl_server_port", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8443", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.159 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_behaviour": { 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_behaviour", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "redirect", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.160 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "redirect" 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_error_status_code": { 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_error_status_code", 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "500", 2024-03-23T15:53:31.161 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "400", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "599", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "url_prefix": { 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "url_prefix", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.162 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "devicehealth", 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.163 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "enable_monitoring": { 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enable_monitoring", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "monitor device health metrics", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.164 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.165 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.166 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.167 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "mark_out_threshold": { 2024-03-23T15:53:31.169 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mark_out_threshold", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "2419200", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": { 2024-03-23T15:53:31.170 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pool_name", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "device_health_metrics", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "retention_period": { 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "retention_period", 2024-03-23T15:53:31.171 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15552000", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long to retain device health metrics", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "scrape_frequency": { 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "scrape_frequency", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.172 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "86400", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "self_heal": { 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "self_heal", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.173 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.174 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_threshold": { 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_threshold", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.175 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "7257600", 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "diskprediction_local", 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.176 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.177 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.178 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.179 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.180 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.181 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "predict_interval": { 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "predict_interval", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "86400", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.182 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "predictor_model": { 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "predictor_model", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "prophetstor", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.183 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.184 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "influx", 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": false, 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "influxdb python module not found", 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "batch_size": { 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "batch_size", 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.185 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5000", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "database": { 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "database", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.186 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "ceph", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": { 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "hostname", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.187 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB server hostname", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.188 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "30", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "5", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.189 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.191 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.192 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.193 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "password": { 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "password", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.194 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "password of InfluxDB server user", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "port": { 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "port", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8086", 2024-03-23T15:53:31.195 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB server port", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl": { 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "false", 2024-03-23T15:53:31.196 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "threads": { 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "threads", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.197 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "32", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "username": { 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "username", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.198 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "username of InfluxDB server user", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "verify_ssl": { 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "verify_ssl", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "true", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.199 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "insights", 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.200 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.201 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.202 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.203 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.204 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "iostat", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.206 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.207 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.208 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.209 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.210 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "k8sevents", 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_event_retention_days": { 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ceph_event_retention_days", 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "7", 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.211 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "config_check_secs": { 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_check_secs", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "10", 2024-03-23T15:53:31.212 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.213 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.214 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.215 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.216 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.217 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "localpool", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "failure_domain": { 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "failure_domain", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "host", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.219 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "failure domain for any created local pool", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.220 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.221 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.222 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.223 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.224 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": { 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "min_size", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default min_size for any created local pool", 2024-03-23T15:53:31.225 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "num_rep": { 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "num_rep", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default replica count for any created local pool", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.226 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": { 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pg_num", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "128", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.227 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "prefix": { 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prefix", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "name prefix for any created local pool", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.228 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "subtree": { 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "subtree", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "rack", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mds_autoscaler", 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.230 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.231 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.232 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.233 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.234 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.235 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mirroring", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.236 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.237 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.238 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.239 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.240 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "nfs", 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.241 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.242 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.243 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.245 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.246 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "orchestrator", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "fail_fs": { 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "fail_fs", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T15:53:31.247 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.249 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.250 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.251 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.252 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator": { 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "orchestrator", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.253 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm", 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "rook", 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "test_orchestrator" 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Orchestrator backend", 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "osd_perf_query", 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.254 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.255 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.256 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.257 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.259 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "osd_support", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.260 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.261 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.262 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.263 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.264 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pg_autoscaler", 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.265 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.266 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.267 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.268 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.269 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.270 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "threshold": { 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "threshold", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3.0", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1.0", 2024-03-23T15:53:31.271 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "scaling threshold", 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "progress", 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_pg_recovery_event": { 2024-03-23T15:53:31.272 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_pg_recovery_event", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "enabled": { 2024-03-23T15:53:31.273 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enabled", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.276 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.277 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.278 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "max_completed_events": { 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_completed_events", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "50", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "number of past completed events to remember", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.280 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long the module is going to sleep", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prometheus", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "cache": { 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cache", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.282 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "exclude_perf_counters": { 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "exclude_perf_counters", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.283 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.284 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.285 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.286 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.287 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.288 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_stats_pools": { 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_stats_pools", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.289 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "300", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.290 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "scrape_interval": { 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "scrape_interval", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15.0", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.291 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "::", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.292 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "9283", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T15:53:31.293 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "stale_cache_strategy": { 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "stale_cache_strategy", 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "log", 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.294 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_behaviour": { 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_behaviour", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "default", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.295 INFO:teuthology.orchestra.run.smithi076.stdout: "default", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "error" 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_error_status_code": { 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_error_status_code", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "500", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "400", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "599", 2024-03-23T15:53:31.296 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_support", 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.297 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.298 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.299 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.300 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.301 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "max_concurrent_snap_create": { 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_concurrent_snap_create", 2024-03-23T15:53:31.302 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "mirror_snapshot_schedule": { 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T15:53:31.303 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "trash_purge_schedule": { 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "trash_purge_schedule", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.304 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.305 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "restful", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "enable_auth": { 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enable_auth", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.306 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "key_file": { 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "key_file", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.307 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.308 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.309 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.310 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.314 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rgw", 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.315 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.316 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.317 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.318 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.319 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rook", 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.320 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.321 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.323 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.324 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "storage_class": { 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "storage_class", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.325 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "local", 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "selftest", 2024-03-23T15:53:31.326 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.327 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.328 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.329 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "roption1": { 2024-03-23T15:53:31.331 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "roption1", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "roption2": { 2024-03-23T15:53:31.332 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "roption2", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "xyz", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption1": { 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption1", 2024-03-23T15:53:31.333 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption2": { 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption2", 2024-03-23T15:53:31.334 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption3": { 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption3", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.335 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption4": { 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption4", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.336 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption5": { 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption5", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.337 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption6": { 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption6", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.338 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption7": { 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption7", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.339 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "42", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "testkey": { 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testkey", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.340 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "testlkey": { 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testlkey", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.341 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "testnewline": { 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testnewline", 2024-03-23T15:53:31.342 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snap_schedule", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_m_granularity": { 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_m_granularity", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T15:53:31.347 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "dump_on_update": { 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "dump_on_update", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "dump database to debug log on update", 2024-03-23T15:53:31.348 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.349 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.350 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.351 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.352 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.353 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "stats", 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.354 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.357 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.358 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "status", 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.359 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.360 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.361 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.362 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.363 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.364 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "telegraf", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "address": { 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "address", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.365 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.366 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.367 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.368 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.369 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.370 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "telemetry", 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_basic": { 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_basic", 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_crash": { 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_crash", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_device": { 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_device", 2024-03-23T15:53:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_ident": { 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_ident", 2024-03-23T15:53:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_perf": { 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_perf", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.376 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "contact": { 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "contact", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.377 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "description": { 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "description", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.378 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "device_url": { 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_url", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.379 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "enabled": { 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enabled", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.380 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.381 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "24", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "8", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "last_opt_revision": { 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "last_opt_revision", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.382 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "leaderboard": { 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "leaderboard", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.383 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "leaderboard_description": { 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "leaderboard_description", 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.384 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.385 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.387 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.388 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.389 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "organization": { 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "organization", 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.390 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "proxy": { 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "proxy", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.391 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "url": { 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "url", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.392 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "test_orchestrator", 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.393 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.394 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.395 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.396 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.397 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "volumes", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.399 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.400 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.401 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.402 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.403 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "max_concurrent_clones": { 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_concurrent_clones", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "4", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T15:53:31.404 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "periodic_async_work": { 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "periodic_async_work", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Periodically check for async work", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.405 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "snapshot_clone_delay": { 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snapshot_clone_delay", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.406 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "snapshot_clone_no_wait": { 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.407 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix", 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "discovery_interval": { 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "discovery_interval", 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "100", 2024-03-23T15:53:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "identifier": { 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "identifier", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.411 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.412 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.413 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.414 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.415 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_host": { 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_host", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.416 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_port": { 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_port", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10051", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.417 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_sender": { 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_sender", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.418 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "modules": [ 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm", 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "dashboard", 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "iostat", 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "nfs", 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "restful", 2024-03-23T15:53:31.419 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_schedule" 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "available_modules": [ 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "alerts", 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.420 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How frequently to reexamine health status", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.421 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.422 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.423 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.424 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.425 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_destination": { 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_destination", 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.426 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Email address to send alerts to", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_from_name": { 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_from_name", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "Ceph", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.427 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Email From: name", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_host": { 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_host", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP server", 2024-03-23T15:53:31.428 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_password": { 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_password", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Password to authenticate with", 2024-03-23T15:53:31.429 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_port": { 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_port", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "465", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP port", 2024-03-23T15:53:31.430 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_sender": { 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_sender", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SMTP envelope sender", 2024-03-23T15:53:31.431 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_ssl": { 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_ssl", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.432 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "smtp_user": { 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "smtp_user", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "User to authenticate as", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.433 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "balancer", 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "active": { 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "active", 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.434 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "automatically balance PGs across cluster", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "begin_time": { 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "begin_time", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0000", 2024-03-23T15:53:31.435 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "beginning time of day to automatically balance", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "begin_weekday": { 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "begin_weekday", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.436 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "6", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_max_iterations": { 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_max_iterations", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "25", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.437 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "250", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_metrics": { 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_metrics", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "pgs,objects,bytes", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.438 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_compat_step": { 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crush_compat_step", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0.5", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0.001", 2024-03-23T15:53:31.439 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "0.999", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "aggressiveness of optimization", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "end_time": { 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "end_time", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "2359", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.440 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "ending time of day to automatically balance", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "end_weekday": { 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "end_weekday", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "0", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "6", 2024-03-23T15:53:31.441 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.442 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.443 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.444 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.445 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.446 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "min_score": { 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "min_score", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.447 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": { 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mode", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "upmap", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-compat", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "none", 2024-03-23T15:53:31.448 INFO:teuthology.orchestra.run.smithi076.stdout: "read", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap-read" 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Balancer mode", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_ids": { 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pool_ids", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.449 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.450 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap_max_deviation": { 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "upmap_max_deviation", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.451 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "upmap_max_optimizations": { 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "upmap_max_optimizations", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.452 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cephadm", 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_down_multiplier": { 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_down_multiplier", 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.453 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3.0", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_refresh_rate": { 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_refresh_rate", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.454 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "20", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "agent_starting_port": { 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "agent_starting_port", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.455 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "4721", 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_ptrace": { 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_ptrace", 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.456 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "autotune_interval": { 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "autotune_interval", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.457 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "autotune_memory_target_ratio": { 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "autotune_memory_target_ratio", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.458 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0.7", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm_log_destination": { 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cephadm_log_destination", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.459 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "file", 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "file,syslog", 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "syslog" 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.460 INFO:teuthology.orchestra.run.smithi076.stdout: "cgroups_split": { 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cgroups_split", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "config_checks_enabled": { 2024-03-23T15:53:31.461 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_checks_enabled", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "config_dashboard": { 2024-03-23T15:53:31.462 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_dashboard", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_alertmanager": { 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_alertmanager", 2024-03-23T15:53:31.463 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_base": { 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_base", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.464 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Container image name, without the tag", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_elasticsearch": { 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_elasticsearch", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.465 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "elasticsearch container image", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_grafana": { 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_grafana", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.466 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_haproxy": { 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_haproxy", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.467 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "HAproxy container image", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_agent": { 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_agent", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-23T15:53:31.468 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger agent container image", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_collector": { 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_collector", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-23T15:53:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger collector container image", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_jaeger_query": { 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_jaeger_query", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-23T15:53:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Jaeger query container image", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_keepalived": { 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_keepalived", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.471 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Keepalived container image", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_loki": { 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_loki", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.472 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Loki container image", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_node_exporter": { 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_node_exporter", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.473 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_nvmeof": { 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_nvmeof", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.474 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Nvme-of container image", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_prometheus": { 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_prometheus", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.475 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Prometheus container image", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_promtail": { 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_promtail", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Promtail container image", 2024-03-23T15:53:31.476 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image_snmp_gateway": { 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_image_snmp_gateway", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "SNMP Gateway container image", 2024-03-23T15:53:31.477 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "container_init": { 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "container_init", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Run podman/docker with `--init`", 2024-03-23T15:53:31.478 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "daemon_cache_timeout": { 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "daemon_cache_timeout", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.479 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "default_cephadm_command_timeout": { 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "default_cephadm_command_timeout", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "900", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "default_registry": { 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "default_registry", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "docker.io", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.481 INFO:teuthology.orchestra.run.smithi076.stdout: "device_cache_timeout": { 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_cache_timeout", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1800", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache device inventory", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "device_enhanced_scan": { 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_enhanced_scan", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "facts_cache_timeout": { 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "facts_cache_timeout", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "seconds to cache host facts data", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.484 INFO:teuthology.orchestra.run.smithi076.stdout: "host_check_interval": { 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "host_check_interval", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to perform a host check", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "hw_monitoring": { 2024-03-23T15:53:31.485 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "hw_monitoring", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: "inventory_list_all": { 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "inventory_list_all", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.487 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.488 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "log_refresh_metadata": { 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_refresh_metadata", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.489 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.490 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.491 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.492 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.493 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.494 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "*", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "max_count_per_host": { 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_count_per_host", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.495 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "max number of daemons per service per host", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd_draining_count": { 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_osd_draining_count", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.496 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "migration_current": { 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "migration_current", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.497 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "internal - do not modify", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": { 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mode", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.498 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "root", 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm-package", 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "root" 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "mode for remote execution of cephadm", 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "oob_default_addr": { 2024-03-23T15:53:31.499 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "oob_default_addr", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "169.254.1.1", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "prometheus_alerts_path": { 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prometheus_alerts_path", 2024-03-23T15:53:31.500 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_insecure": { 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_insecure", 2024-03-23T15:53:31.501 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_password": { 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_password", 2024-03-23T15:53:31.502 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_url": { 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_url", 2024-03-23T15:53:31.503 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "registry_username": { 2024-03-23T15:53:31.504 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "registry_username", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "secure_monitoring_stack": { 2024-03-23T15:53:31.505 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "secure_monitoring_stack", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.506 INFO:teuthology.orchestra.run.smithi076.stdout: "service_discovery_port": { 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "service_discovery_port", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8765", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "cephadm service discovery port", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "ssh_config_file": { 2024-03-23T15:53:31.507 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssh_config_file", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "use_agent": { 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "use_agent", 2024-03-23T15:53:31.508 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "use_repo_digest": { 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "use_repo_digest", 2024-03-23T15:53:31.509 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_failed_host_check": { 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_failed_host_check", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.510 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if the host check fails", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_stray_daemons": { 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_stray_daemons", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.511 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_on_stray_hosts": { 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_on_stray_hosts", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.512 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crash", 2024-03-23T15:53:31.513 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.514 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.515 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.516 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.517 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.518 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "retain_interval": { 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "retain_interval", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "31536000", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.519 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_recent_interval": { 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_recent_interval", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1209600", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.520 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "dashboard", 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.521 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "ALERTMANAGER_API_HOST": { 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.522 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.523 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "AUDIT_API_ENABLED": { 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "AUDIT_API_ENABLED", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.524 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.525 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "ENABLE_BROWSABLE_API": { 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.526 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.527 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.528 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.529 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.530 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.531 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.532 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.533 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.534 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_PASSWORD": { 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.535 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.536 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_URL": { 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_URL", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.537 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_API_USERNAME": { 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.538 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.540 INFO:teuthology.orchestra.run.smithi076.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-23T15:53:31.541 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "PROMETHEUS_API_HOST": { 2024-03-23T15:53:31.542 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-23T15:53:31.543 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-23T15:53:31.544 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-23T15:53:31.545 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.546 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.550 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.551 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.552 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.553 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.554 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_ENABLED": { 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.555 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.556 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.557 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.558 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "45", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.559 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_ACCESS_KEY": { 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "admin", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_SECRET_KEY": { 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "RGW_API_SSL_VERIFY": { 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.563 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.564 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "cross_origin_url": { 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cross_origin_url", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.567 INFO:teuthology.orchestra.run.smithi076.stdout: "crt_file": { 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "crt_file", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "debug": { 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "debug", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Enable/disable debug options", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.569 INFO:teuthology.orchestra.run.smithi076.stdout: "jwt_token_ttl": { 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "jwt_token_ttl", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "28800", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.570 INFO:teuthology.orchestra.run.smithi076.stdout: "key_file": { 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "key_file", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.571 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.572 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.573 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.574 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "motd": { 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "motd", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "The message of the day", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "redirect_resolve_ip_addr": { 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "redirect_resolve_ip_addr", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "::", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8080", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl": { 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl_server_port": { 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl_server_port", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8443", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_behaviour": { 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_behaviour", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "redirect", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "redirect" 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_error_status_code": { 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_error_status_code", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "500", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "400", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "599", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "url_prefix": { 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "url_prefix", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "devicehealth", 2024-03-23T15:53:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "enable_monitoring": { 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enable_monitoring", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "monitor device health metrics", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.586 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.587 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.588 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.589 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.590 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.591 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "mark_out_threshold": { 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mark_out_threshold", 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "2419200", 2024-03-23T15:53:31.592 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": { 2024-03-23T15:53:31.593 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pool_name", 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "device_health_metrics", 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.594 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "retention_period": { 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "retention_period", 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.595 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15552000", 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long to retain device health metrics", 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.596 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "scrape_frequency": { 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "scrape_frequency", 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "86400", 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.597 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "self_heal": { 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "self_heal", 2024-03-23T15:53:31.598 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-23T15:53:31.599 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.600 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how frequently to wake up and check device health", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "warn_threshold": { 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "warn_threshold", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "7257600", 2024-03-23T15:53:31.601 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "diskprediction_local", 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.602 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.603 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.604 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.605 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.606 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "predict_interval": { 2024-03-23T15:53:31.607 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "predict_interval", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "86400", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "predictor_model": { 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "predictor_model", 2024-03-23T15:53:31.608 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "prophetstor", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.609 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "600", 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "influx", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": false, 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "influxdb python module not found", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "batch_size": { 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "batch_size", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5000", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-23T15:53:31.611 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "database": { 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "database", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "ceph", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.612 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": { 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "hostname", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB server hostname", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.613 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "30", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "5", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.614 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.615 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.616 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.617 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.618 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.619 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "password": { 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "password", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "password of InfluxDB server user", 2024-03-23T15:53:31.620 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "port": { 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "port", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "8086", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "InfluxDB server port", 2024-03-23T15:53:31.621 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "ssl": { 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ssl", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "false", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T15:53:31.622 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "threads": { 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "threads", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "32", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.623 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "username": { 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "username", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "username of InfluxDB server user", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.624 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "verify_ssl": { 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "verify_ssl", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "true", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.625 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "insights", 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.626 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.627 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.628 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.629 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.630 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "iostat", 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.631 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.633 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.634 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.635 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "k8sevents", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_event_retention_days": { 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "ceph_event_retention_days", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "7", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.637 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "config_check_secs": { 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "config_check_secs", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "10", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.638 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.639 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.640 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.641 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.642 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.643 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "localpool", 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "failure_domain": { 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "failure_domain", 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "host", 2024-03-23T15:53:31.644 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "failure domain for any created local pool", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.645 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.646 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.647 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.648 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.649 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": { 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "min_size", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.650 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default min_size for any created local pool", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "num_rep": { 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "num_rep", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.651 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default replica count for any created local pool", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": { 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pg_num", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "128", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "default pg_num for any created local pool", 2024-03-23T15:53:31.652 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "prefix": { 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prefix", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "name prefix for any created local pool", 2024-03-23T15:53:31.653 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "subtree": { 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "subtree", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "rack", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-23T15:53:31.654 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mds_autoscaler", 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.655 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.656 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.658 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.659 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mirroring", 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.660 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.661 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.662 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.663 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.664 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "nfs", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.666 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.667 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.668 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.669 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.670 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "orchestrator", 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "fail_fs": { 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "fail_fs", 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.671 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.672 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.673 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.674 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.675 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.676 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator": { 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "orchestrator", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.677 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm", 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "rook", 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "test_orchestrator" 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Orchestrator backend", 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "osd_perf_query", 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.678 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.679 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.680 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.681 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.683 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "osd_support", 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.684 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.687 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.688 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "pg_autoscaler", 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.689 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.690 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.691 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.692 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.693 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.694 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "threshold": { 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "threshold", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.695 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "3.0", 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1.0", 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "scaling threshold", 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.696 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "progress", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_pg_recovery_event": { 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_pg_recovery_event", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.697 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "enabled": { 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enabled", 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.698 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.699 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.700 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.701 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.702 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.703 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "max_completed_events": { 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_completed_events", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "50", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "number of past completed events to remember", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.704 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "sleep_interval": { 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "sleep_interval", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "5", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "how long the module is going to sleep", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.705 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "prometheus", 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "cache": { 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "cache", 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.706 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "exclude_perf_counters": { 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "exclude_perf_counters", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.707 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.708 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.709 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.710 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.711 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_stats_pools": { 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_stats_pools", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.713 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "300", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.714 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "scrape_interval": { 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "scrape_interval", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15.0", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.715 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "::", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.716 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "9283", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.717 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "stale_cache_strategy": { 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "stale_cache_strategy", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "log", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.718 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_behaviour": { 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_behaviour", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "default", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.719 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "default", 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "error" 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "standby_error_status_code": { 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "standby_error_status_code", 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.720 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "500", 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "400", 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "599", 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_support", 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.721 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.722 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.723 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.724 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.725 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.726 INFO:teuthology.orchestra.run.smithi076.stdout: "max_concurrent_snap_create": { 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_concurrent_snap_create", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "mirror_snapshot_schedule": { 2024-03-23T15:53:31.727 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mirror_snapshot_schedule", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "trash_purge_schedule": { 2024-03-23T15:53:31.728 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "trash_purge_schedule", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "restful", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "enable_auth": { 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enable_auth", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.730 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "key_file": { 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "key_file", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.731 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.732 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.733 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.734 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.735 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.736 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "server_addr": { 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_addr", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.737 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "server_port": { 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "server_port", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.738 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rgw", 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.739 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.740 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.741 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.742 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.743 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rook", 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.744 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.745 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.746 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "storage_class": { 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "storage_class", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "local", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "selftest", 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.751 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.752 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.753 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.754 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.755 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "roption1": { 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "roption1", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.756 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "roption2": { 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "roption2", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "xyz", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.757 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption1": { 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption1", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.758 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption2": { 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption2", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.759 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption3": { 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption3", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "float", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.760 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption4": { 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption4", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption5": { 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption5", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.762 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption6": { 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption6", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "rwoption7": { 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rwoption7", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "1", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "42", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "testkey": { 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testkey", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "testlkey": { 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testlkey", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.766 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "testnewline": { 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "testnewline", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.767 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snap_schedule", 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_m_granularity": { 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "allow_m_granularity", 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.768 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "allow minute scheduled snapshots", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "dump_on_update": { 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "dump_on_update", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.769 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "dump database to debug log on update", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.770 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.771 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.772 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.773 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.774 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "stats", 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.775 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.776 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.777 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.778 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.779 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "status", 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.780 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.781 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.782 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.783 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.785 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "telegraf", 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "address": { 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "address", 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-23T15:53:31.786 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "15", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.787 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.788 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.789 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.790 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.791 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.792 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "telemetry", 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_basic": { 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_basic", 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_crash": { 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_crash", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.794 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_device": { 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_device", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.795 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_ident": { 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_ident", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.796 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "channel_perf": { 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "channel_perf", 2024-03-23T15:53:31.797 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.798 INFO:teuthology.orchestra.run.smithi076.stdout: "contact": { 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "contact", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.799 INFO:teuthology.orchestra.run.smithi076.stdout: "description": { 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "description", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "device_url": { 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "device_url", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.801 INFO:teuthology.orchestra.run.smithi076.stdout: "enabled": { 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "enabled", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.802 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "24", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "8", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.803 INFO:teuthology.orchestra.run.smithi076.stdout: "last_opt_revision": { 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "last_opt_revision", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "1", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.804 INFO:teuthology.orchestra.run.smithi076.stdout: "leaderboard": { 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "leaderboard", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.805 INFO:teuthology.orchestra.run.smithi076.stdout: "leaderboard_description": { 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "leaderboard_description", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.806 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.807 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.808 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.809 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.810 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "organization": { 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "organization", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.811 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "proxy": { 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "proxy", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.812 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "url": { 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "url", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.813 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "test_orchestrator", 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.814 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.815 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.816 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.817 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.819 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "volumes", 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.820 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.821 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.822 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.823 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.824 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "max_concurrent_clones": { 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "max_concurrent_clones", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "4", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-23T15:53:31.825 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "periodic_async_work": { 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "periodic_async_work", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Periodically check for async work", 2024-03-23T15:53:31.826 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "snapshot_clone_delay": { 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snapshot_clone_delay", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "0", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.827 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "snapshot_clone_no_wait": { 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snapshot_clone_no_wait", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "True", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.828 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix", 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "can_run": true, 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "error_string": "", 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "module_options": { 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "discovery_interval": { 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "discovery_interval", 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "uint", 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.829 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "100", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "identifier": { 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "identifier", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.830 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "interval": { 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "interval", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "secs", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "60", 2024-03-23T15:53:31.831 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "log_level": { 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_level", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.832 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.833 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster": { 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.834 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_cluster_level": { 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_cluster_level", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "info", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [ 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "critical", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "debug", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "error", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "info", 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: "warning" 2024-03-23T15:53:31.835 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": { 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "log_to_file", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "bool", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "False", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.836 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_host": { 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_host", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.837 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_port": { 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_port", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "int", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "10051", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "zabbix_sender": { 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "zabbix_sender", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "str", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "level": "advanced", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 0, 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "min": "", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "max": "", 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "enum_allowed": [], 2024-03-23T15:53:31.839 INFO:teuthology.orchestra.run.smithi076.stdout: "desc": "", 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "long_desc": "", 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "tags": [], 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "see_also": [] 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "services": { 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "dashboard": "https://172.21.15.76:8443/" 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "always_on_modules": { 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus": [ 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "balancer", 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "crash", 2024-03-23T15:53:31.840 INFO:teuthology.orchestra.run.smithi076.stdout: "devicehealth", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscaler", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "progress", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_support", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "status", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "telemetry", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "volumes" 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific": [ 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "balancer", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "crash", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "devicehealth", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscaler", 2024-03-23T15:53:31.841 INFO:teuthology.orchestra.run.smithi076.stdout: "progress", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_support", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "status", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "telemetry", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "volumes" 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy": [ 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "balancer", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "crash", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "devicehealth", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscaler", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "progress", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_support", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "status", 2024-03-23T15:53:31.842 INFO:teuthology.orchestra.run.smithi076.stdout: "telemetry", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "volumes" 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "reef": [ 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "balancer", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "crash", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "devicehealth", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscaler", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "progress", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_support", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "status", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "telemetry", 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "volumes" 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-03-23T15:53:31.843 INFO:teuthology.orchestra.run.smithi076.stdout: "squid": [ 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "balancer", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "crash", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "devicehealth", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "orchestrator", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscaler", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "progress", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_support", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "status", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "telemetry", 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "volumes" 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "last_failure_osd_epoch": 88, 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: "active_clients": [ 2024-03-23T15:53:31.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "devicehealth", 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:0", 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 308080676 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "libcephsqlite", 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-03-23T15:53:31.845 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:0", 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 478417831 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "rbd_support", 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:0", 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1678581997 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snap_schedule", 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:0", 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1269642672 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "snap_schedule", 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:0", 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3575908852 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "volumes", 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:0", 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2632760467 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.848 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:31.849 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-03-23T15:53:31.849 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:53:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:31 smithi183 ceph-mon[36296]: pgmap v316: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:31 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3512114649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:53:31.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:31 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/320917853' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T15:53:31.850 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:31 smithi192 ceph-mon[36653]: pgmap v316: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:31.850 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:31 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3512114649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:53:31.851 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:31 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/320917853' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f0fcc0a22b0 msgr2=0x7f0fcc0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f0fcc0a22b0 0x7f0fcc0a4770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4197b90 tx=0x7f0fd4009260 comp rx=0 tx=0).stop 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4106fd0 msgr2=0x7f0fe4196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4106fd0 0x7f0fe4196a50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc036040 tx=0x7f0fdc033680 comp rx=0 tx=0).stop 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 shutdown_connections 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f0fcc0a22b0 0x7f0fcc0a4770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe4106fd0 0x7f0fe4196a50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:31.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe41066b0 0x7f0fe419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:31.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 --2- 172.21.15.76:0/320917853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe4100d40 0x7f0fe419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:53:31.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 >> 172.21.15.76:0/320917853 conn(0x7f0fe4077710 msgr2=0x7f0fe40ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:53:31.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 shutdown_connections 2024-03-23T15:53:31.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:53:30.568+0000 7f0fea64e640 1 -- 172.21.15.76:0/320917853 wait complete. 2024-03-23T15:53:31.858 DEBUG:tasks.check_counter:Getting stats from a 2024-03-23T15:53:31.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-23T15:53:31.858 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:31 smithi076 ceph-mon[32176]: pgmap v316: 97 pgs: 97 active+clean; 43 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:31.859 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:31 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3512114649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:53:31.859 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:31 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/320917853' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-23T15:53:32.112 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:53:33.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:33 smithi192 ceph-mon[36653]: pgmap v317: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:33.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:33 smithi076 ceph-mon[32176]: pgmap v317: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:33.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:33 smithi183 ceph-mon[36296]: pgmap v317: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:33.965 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:53:33.965 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:33.965 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 0, 2024-03-23T15:53:33.965 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 2, 2024-03-23T15:53:33.965 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 0, 2024-03-23T15:53:33.965 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 0, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 4, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 0, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.001269403, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.000275573, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.966 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 211, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 201, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1429472, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 123424, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 2, 2024-03-23T15:53:33.967 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 1, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.036900433, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.012641733, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.027055249, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.004037106, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 200, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.009429019, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000047145 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:33.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1429472, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 123424 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 6, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 155, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 3344, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 182752, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012243296, 2024-03-23T15:53:33.969 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.008186466, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.002087091, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 153, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.012202317, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000079753 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:33.970 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 182752 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 0, 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:33.971 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:33.972 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:33.973 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 148977, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 30, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-03-23T15:53:33.974 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 928, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 2, 2024-03-23T15:53:33.975 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 213, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1399736, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:33.976 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 213, 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 213, 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1399736, 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:33.977 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:33.978 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:33.978 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:53:34.290 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:34 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:53:34.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:34 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:53:34.342 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:34 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T15:53:34.651 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-23T15:53:34.651 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-23T15:53:34.651 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-23T15:53:34.651 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-23T15:53:34.651 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-23T15:53:34.651 DEBUG:tasks.check_counter:Getting stats from d 2024-03-23T15:53:34.652 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-23T15:53:35.320 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:35 smithi192 ceph-mon[36653]: pgmap v318: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:35.326 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:35 smithi076 ceph-mon[32176]: pgmap v318: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:35.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:35 smithi183 ceph-mon[36296]: pgmap v318: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:35.668 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:53:36.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.327 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.328 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.329 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:36 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.384 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.385 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.386 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:36 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:36.571 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:36 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:37.384 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:53:37.384 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:37.384 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 220, 2024-03-23T15:53:37.384 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 208, 2024-03-23T15:53:37.384 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1433856, 2024-03-23T15:53:37.384 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 127072, 2024-03-23T15:53:37.385 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 4, 2024-03-23T15:53:37.385 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 0, 2024-03-23T15:53:37.385 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.044656013, 2024-03-23T15:53:37.385 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.013313580, 2024-03-23T15:53:37.385 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.034485344, 2024-03-23T15:53:37.385 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.004130923, 2024-03-23T15:53:37.385 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 208, 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.010494791, 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000050455 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:37.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1433856, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 127072 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 2, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 158, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 320, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 183184, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-03-23T15:53:37.387 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-03-23T15:53:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012259208, 2024-03-23T15:53:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.008303078, 2024-03-23T15:53:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.001811133, 2024-03-23T15:53:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 155, 2024-03-23T15:53:37.388 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.011982502, 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000077306 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 183184 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.389 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:37.390 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 0, 2024-03-23T15:53:37.390 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 2, 2024-03-23T15:53:37.390 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 0, 2024-03-23T15:53:37.390 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 0, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 2, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 1, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.000195876, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.000120983, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:37.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.392 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:37.392 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:37.392 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:37.392 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:37.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.392 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:37.392 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 0, 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:37.394 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:37.395 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:37.396 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:37.397 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:37.397 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:37.397 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:37.397 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:37.397 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 148955, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 30, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:37.398 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:37.399 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 928, 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 2, 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:37.400 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 218, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1400386, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 218, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-23T15:53:37.401 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 218, 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1400386, 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:37.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:37.403 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:53:37.403 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:37 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-23T15:53:37.403 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:37 smithi076 ceph-mon[32176]: Unable to set osd_memory_target on smithi192 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-03-23T15:53:37.403 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:37 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi183 to 174.1M 2024-03-23T15:53:37.403 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:37 smithi076 ceph-mon[32176]: Unable to set osd_memory_target on smithi183 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-23T15:53:37.403 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:37 smithi076 ceph-mon[32176]: pgmap v319: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:37 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-23T15:53:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:37 smithi192 ceph-mon[36653]: Unable to set osd_memory_target on smithi192 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-03-23T15:53:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:37 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi183 to 174.1M 2024-03-23T15:53:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:37 smithi192 ceph-mon[36653]: Unable to set osd_memory_target on smithi183 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-23T15:53:37.570 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:37 smithi192 ceph-mon[36653]: pgmap v319: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:37.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:37 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi192 to 174.1M 2024-03-23T15:53:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:37 smithi183 ceph-mon[36296]: Unable to set osd_memory_target on smithi192 to 182616268: error parsing value: Value '182616268' is below minimum 939524096 2024-03-23T15:53:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:37 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi183 to 174.1M 2024-03-23T15:53:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:37 smithi183 ceph-mon[36296]: Unable to set osd_memory_target on smithi183 to 182616985: error parsing value: Value '182616985' is below minimum 939524096 2024-03-23T15:53:37.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:37 smithi183 ceph-mon[36296]: pgmap v319: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:38.066 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-23T15:53:38.066 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-23T15:53:38.067 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-23T15:53:38.067 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-23T15:53:38.067 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-23T15:53:38.067 DEBUG:tasks.check_counter:Getting stats from g 2024-03-23T15:53:38.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-23T15:53:38.331 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:53:38.711 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:38.711 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:38.712 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:38.712 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:38.712 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:38.712 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:38.712 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:53:38.712 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:53:38.713 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:38 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:39.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:39.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:39.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:53:39.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:53:39.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:38 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T15:53:39.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:38 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' 2024-03-23T15:53:39.792 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:39 smithi076 ceph-mon[32176]: pgmap v320: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:39.792 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:39 smithi076 ceph-mon[32176]: Adjusting osd_memory_target on smithi076 to 176.2M 2024-03-23T15:53:39.792 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:39 smithi076 ceph-mon[32176]: Unable to set osd_memory_target on smithi076 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-03-23T15:53:40.055 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 217, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 205, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1433040, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 125152, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 4, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 0, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.039387577, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.012871643, 2024-03-23T15:53:40.056 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.029299201, 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.004008924, 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 205, 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.010522581, 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000051329 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:40.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1433040, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 125152 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 2, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 155, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 320, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 182656, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012982062, 2024-03-23T15:53:40.058 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.009037337, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.001650942, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 152, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.011013366, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000072456 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.059 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 182656 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 0, 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 2, 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 0, 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 0, 2024-03-23T15:53:40.060 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 2, 2024-03-23T15:53:40.061 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 1, 2024-03-23T15:53:40.061 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.000352458, 2024-03-23T15:53:40.061 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T15:53:40.061 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.000292202, 2024-03-23T15:53:40.061 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:40.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 0, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:40.063 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:40.064 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:40.065 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 148994, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 31, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-03-23T15:53:40.066 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 928, 2024-03-23T15:53:40.067 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 2, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 215, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1399996, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 215, 2024-03-23T15:53:40.068 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 215, 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1399996, 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:40.069 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:53:40.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:39 smithi192 ceph-mon[36653]: pgmap v320: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:40.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:39 smithi192 ceph-mon[36653]: Adjusting osd_memory_target on smithi076 to 176.2M 2024-03-23T15:53:40.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:39 smithi192 ceph-mon[36653]: Unable to set osd_memory_target on smithi076 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-03-23T15:53:40.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:39 smithi183 ceph-mon[36296]: pgmap v320: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:40.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:39 smithi183 ceph-mon[36296]: Adjusting osd_memory_target on smithi076 to 176.2M 2024-03-23T15:53:40.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:39 smithi183 ceph-mon[36296]: Unable to set osd_memory_target on smithi076 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-03-23T15:53:40.746 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-23T15:53:40.746 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-23T15:53:40.746 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-23T15:53:40.746 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-23T15:53:40.746 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-23T15:53:40.746 DEBUG:tasks.check_counter:Getting stats from j 2024-03-23T15:53:40.747 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-23T15:53:40.991 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.a/config 2024-03-23T15:53:42.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:41 smithi192 ceph-mon[36653]: pgmap v321: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:42.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:41 smithi076 ceph-mon[32176]: pgmap v321: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:42.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:41 smithi183 ceph-mon[36296]: pgmap v321: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T15:53:42.771 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-03-23T15:53:42.771 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 219, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 209, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1431648, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 128544, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 4, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 0, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.043529602, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.012623729, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.033536457, 2024-03-23T15:53:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.005262095, 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 208, 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.009592309, 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000046116 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1431648, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 128544 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 6, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 162, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 3344, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 183984, 2024-03-23T15:53:42.774 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.012686748, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.008124711, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.002578800, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 160, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.012195843, 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000076224 2024-03-23T15:53:42.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 183984 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 0, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 2, 2024-03-23T15:53:42.776 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 0, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 0, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 2, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 1, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.000406540, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.000346430, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:42.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-03-23T15:53:42.778 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 0, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:42.779 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:42.780 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:42.781 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 148977, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 30, 2024-03-23T15:53:42.782 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 0, 2024-03-23T15:53:42.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 928, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 2, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-03-23T15:53:42.784 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 221, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1400776, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 221, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 221, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1400776, 2024-03-23T15:53:42.785 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-03-23T15:53:42.786 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-03-23T15:53:42.786 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-03-23T15:53:42.786 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-03-23T15:53:42.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:42.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-03-23T15:53:42.786 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-03-23T15:53:43.545 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-23T15:53:43.545 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-23T15:53:43.545 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-23T15:53:43.545 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-23T15:53:43.545 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-23T15:53:43.545 DEBUG:tasks.check_counter:Getting stats from b 2024-03-23T15:53:43.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-23T15:53:43.810 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:53:43.826 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:43 smithi076 ceph-mon[32176]: pgmap v322: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:43.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:43 smithi183 ceph-mon[36296]: pgmap v322: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:44.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:43 smithi192 ceph-mon[36653]: pgmap v322: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-23T15:53:45.703 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-03-23T15:53:45.703 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:45.703 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 28430, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 30852, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 10379389, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 110501162, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 24, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 2.857006255, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 1.317044104, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.757308094, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.764114579, 2024-03-23T15:53:45.704 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 30854, 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 17.216848150, 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000558010 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 23930, 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004010674, 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000167 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.705 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 314160 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 18434, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 18170, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 10134148, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 74585593, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 43, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 1.736514750, 2024-03-23T15:53:45.706 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.543924323, 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.343871420, 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.754240924, 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 18168, 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 17.411292323, 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000958349 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 429, 2024-03-23T15:53:45.707 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000053392, 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000124 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1664240, 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 175008 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 80615, 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 131376, 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 27431353, 2024-03-23T15:53:45.708 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 102421123, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 27, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 8, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 5.488468335, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 2.554739034, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 55.288871401, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.570861983, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 131371, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 15.307658972, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000116522 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.709 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 170135, 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.026432200, 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000155 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 306480, 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 214960 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.710 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 12109, 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 119.769961682, 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.009890987 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-03-23T15:53:45.711 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 15853, 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 6.979253535, 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000440248 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 35755, 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 35572, 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-03-23T15:53:45.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 35572, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 193.619409551, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005443028 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 183, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 8, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 2, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 5192, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 14, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-03-23T15:53:45.713 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 28469, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 15798, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 12597, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 74, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 5640, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 1852, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 3, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 3, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 49, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 44957, 2024-03-23T15:53:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 43954, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 80, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 172, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 1, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 3, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 9, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 142, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 1345, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 122, 2024-03-23T15:53:45.715 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 2157, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 8533, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 6, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 8533, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T15:53:45.716 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 880, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 3, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 877, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 7653, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 268, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 3882, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 27653, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 156, 2024-03-23T15:53:45.717 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 22360, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 12941, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 209, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 17988 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-03-23T15:53:45.718 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 179, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 346, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 1607, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 12423, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 10797, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 63, 2024-03-23T15:53:45.719 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 8, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 3, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 143, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-03-23T15:53:45.720 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 14, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 69170, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 64052, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 64052, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 5118, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-03-23T15:53:45.721 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 4985, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 99, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 92, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 92, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 7, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 1, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 7, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 164903490, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 178782502, 2024-03-23T15:53:45.722 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5591, 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 396.980116133, 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.071003419 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-03-23T15:53:45.723 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 13194, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 25368, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 12174, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 1635, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 1847, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 212, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 28469, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 41275, 2024-03-23T15:53:45.724 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 12806, 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 3, 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 13761, 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 13758, 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 207432, 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 231804 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 44066, 2024-03-23T15:53:45.725 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 2343, 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 35755, 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 425, 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 4330, 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11277, 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 27.748753540, 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002460650 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-03-23T15:53:45.726 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 299, 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.954245265, 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003191455 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-03-23T15:53:45.727 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 172, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.466215873, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.020152417 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6738, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 16.581532391, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002460898 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.728 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000176272, 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000176272 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.729 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-03-23T15:53:45.730 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.731 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.731 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.731 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-03-23T15:53:45.731 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10, 2024-03-23T15:53:45.731 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002554137, 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000255413 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1616, 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 10.288766179, 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006366810 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 324, 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.734474507, 2024-03-23T15:53:45.732 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.011526155 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10, 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5.404519519, 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.540451951 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 45, 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.027020438, 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000600454 2024-03-23T15:53:45.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1578, 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.658085900, 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001684465 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 337, 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 7.883853306, 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.023394223 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.734 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1706, 2024-03-23T15:53:45.735 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 84.634374030, 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.049609832 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4, 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.017753019, 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004438254 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 158, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.406903452, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.021562680 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.737 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.019188681, 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002398585 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-03-23T15:53:45.738 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 329, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.376209845, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.007222522 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10960, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 24.414783197, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002227626 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.739 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 5, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 3, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 375, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 187, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 603, 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-03-23T15:53:45.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 480, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 480, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:45.741 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:45.742 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 13686432, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 16521, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 528, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 6, 2024-03-23T15:53:45.743 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8224, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 171, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 56279490, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1414405, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:45.744 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 46929, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 214004012, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 34, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 46929, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 46929, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4225.537232130, 2024-03-23T15:53:45.745 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.090041066 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 46929, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 86340 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 46929, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 8894, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 38035, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 3533, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 14446, 2024-03-23T15:53:45.746 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 333, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 5921, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 223, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 45, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 14, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 11770, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-03-23T15:53:45.747 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 8546, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 29204, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 184, 2024-03-23T15:53:45.748 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-03-23T15:53:45.749 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 96, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 12, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-03-23T15:53:45.750 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 10494, 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 35, 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 1592 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 1 2024-03-23T15:53:45.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 14, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 10, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 11224, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 10797, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:45.752 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 127475, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 36407701, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 127475, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 127475, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 36407701, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.753 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:45.754 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 46929, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 215290413, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 46929, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 215290413, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 46929, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 46929, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 46929, 2024-03-23T15:53:45.756 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 46929, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 10797, 2024-03-23T15:53:45.757 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1287001, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 10797, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 330, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1287001, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.758 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x559b6dc6e500": { 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 69170, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 174588198, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 69170, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:45.759 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 5591, 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 174588198, 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:45.760 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-03-23T15:53:45.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:45 smithi183 ceph-mon[36296]: pgmap v323: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:46.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:45 smithi192 ceph-mon[36653]: pgmap v323: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:46.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:45 smithi076 ceph-mon[32176]: pgmap v323: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:46.470 INFO:tasks.check_counter:Daemon mds.b mds.exported=142 2024-03-23T15:53:46.470 INFO:tasks.check_counter:Daemon mds.b mds.imported=122 2024-03-23T15:53:46.470 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-23T15:53:46.470 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-23T15:53:46.470 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-23T15:53:46.470 DEBUG:tasks.check_counter:Getting stats from e 2024-03-23T15:53:46.471 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-23T15:53:46.737 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:53:47.999 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:47 smithi183 ceph-mon[36296]: pgmap v324: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:48.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:47 smithi192 ceph-mon[36653]: pgmap v324: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:48.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:47 smithi076 ceph-mon[32176]: pgmap v324: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 248, 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 226, 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1504930, 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 131629, 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.035156733, 2024-03-23T15:53:48.488 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.008805695, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.032092907, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.003647715, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 225, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.007139904, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000031732 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.489 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1504096, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 131312 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 673, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 828, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4362712, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 647362, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.054690982, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.016887904, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.014963931, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.017558657, 2024-03-23T15:53:48.490 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 826, 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.025823135, 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000031262 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-23T15:53:48.491 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 439920 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1587, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1589, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 8279721, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 493615, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.107491879, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.034616450, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.030375377, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.034291041, 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:48.492 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1587, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.035948145, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000022651 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-03-23T15:53:48.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3845, 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.079281999, 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000020619 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.494 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-03-23T15:53:48.495 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 2159, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 2159, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 2158, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 3, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-03-23T15:53:48.496 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-03-23T15:53:48.497 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-03-23T15:53:48.498 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-03-23T15:53:48.499 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T15:53:48.500 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4583, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-03-23T15:53:48.501 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 7, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 16199748, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 16197187, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.502 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 4583 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 528, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 2434, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1906, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 90, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 388, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 298, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 2159, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 3799, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 1640, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-03-23T15:53:48.503 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 61532, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215420 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.504 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.505 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-03-23T15:53:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.508 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.509 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-03-23T15:53:48.510 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-03-23T15:53:48.511 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.514 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.516 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.517 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-03-23T15:53:48.518 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 3, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2716, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:48.519 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:48.520 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:48.521 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 221563, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 686, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8224, 2024-03-23T15:53:48.522 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 171, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 3097509, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 152396, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:48.523 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 2255, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 2255, 2024-03-23T15:53:48.524 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2255, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.970947284, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000874034 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2255, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2256 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 2255, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 2255, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-03-23T15:53:48.525 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1370, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 883, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-03-23T15:53:48.526 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T15:53:48.527 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-03-23T15:53:48.528 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 96, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 11, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 6, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-03-23T15:53:48.529 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-03-23T15:53:48.530 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:48.531 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2504, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 13906349, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2504, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2504, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 13906349, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.532 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2255, 2024-03-23T15:53:48.533 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 12005449, 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2255, 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12005449, 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:48.534 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2255, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2255, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2255, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2255, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.535 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:48.536 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5561c6f466e0": { 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:48.537 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:48.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:48.539 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-03-23T15:53:49.280 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-23T15:53:49.281 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-23T15:53:49.281 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-23T15:53:49.281 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-23T15:53:49.281 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T15:53:49.281 DEBUG:tasks.check_counter:Getting stats from h 2024-03-23T15:53:49.281 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-23T15:53:49.528 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:53:49.905 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:49 smithi183 ceph-mon[36296]: pgmap v325: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:50.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:49 smithi192 ceph-mon[36653]: pgmap v325: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:50.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:49 smithi076 ceph-mon[32176]: pgmap v325: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:51.242 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 14250, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 11821, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5459299, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 8372668, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 10, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.834613192, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.375456004, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.542941212, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.027309463, 2024-03-23T15:53:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11821, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.307941780, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000026050 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 23966, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003675342, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000153 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1298, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1341, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 347099, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 4892023, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 17, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.097258367, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.030255719, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.025618079, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.033231416, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1339, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.090439916, 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000067542 2024-03-23T15:53:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 44, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000007001, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000159 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 5680, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 134288 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5571, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 9521, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3338180, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 9437729, 2024-03-23T15:53:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 12, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.410885044, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.179173675, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.224984690, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.030876755, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9518, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.265757689, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000027921 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10513, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001907453, 2024-03-23T15:53:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000181 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1515744, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 471680 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2588, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.778750018, 2024-03-23T15:53:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001073705 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2066, 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.233710445, 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000113122 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 2727, 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 2693, 2024-03-23T15:53:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2693, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 19.523331803, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.007249659 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 34, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 1, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 1248, 2024-03-23T15:53:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 7, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 3, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 1238, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 1238, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 1814, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 4, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 5534, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 4242, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 30, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 167, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-03-23T15:53:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 2, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 9, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 72, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 1352, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 82, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 862, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 383, 2024-03-23T15:53:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 4, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 4, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 383, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 9, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 9, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 374, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 54, 2024-03-23T15:53:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 511, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 1685, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 37, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 1262, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 691, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 105, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 2088 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-03-23T15:53:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 186, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 114, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 110, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 1046, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 936, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 16, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 3, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-03-23T15:53:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 72, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 4583, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-03-23T15:53:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4583, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 7, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 7, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 16199748, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-03-23T15:53:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1082, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.697863766, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003417619 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 541, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 2409, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1868, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 135, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 411, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 276, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 1248, 2024-03-23T15:53:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 3992, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 2744, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 1678, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 1678, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 90952, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223612 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 4648, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 729, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 2727, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 103, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 1847, 2024-03-23T15:53:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 149, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.804391488, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005398600 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 18, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.030555949, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001697552 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 29, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.430611683, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.014848678 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 895, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4.710847801, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005263517 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-03-23T15:53:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 57, 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.120165179, 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002108161 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 67, 2024-03-23T15:53:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.144931468, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002163156 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001484712, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001484712 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-03-23T15:53:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 179, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.160216266, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000895062 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 89, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.186603055, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.035804528 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 117, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 6.875829273, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.058767771 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 39, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.056733661, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001454709 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 78, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.109307725, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001401381 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 975, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.891653543, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002965798 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-03-23T15:53:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 1039, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 519, 2024-03-23T15:53:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 600, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 1198415, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 834, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 264, 2024-03-23T15:53:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 3, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8224, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 171, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 2594425, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 154247, 2024-03-23T15:53:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3159, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 12252779, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3159, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3159, 2024-03-23T15:53:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 8.561767262, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002710277 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3159, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3350 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3159, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 535, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 2624, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-03-23T15:53:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 151, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 1233, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 100, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 14, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 1092, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 384, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-03-23T15:53:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 41, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-03-23T15:53:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 96, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 11, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-03-23T15:53:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 219, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 84 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-03-23T15:53:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 14, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 10, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 1092, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 936, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 21115, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 7226176, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 21115, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 21115, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 7226176, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-03-23T15:53:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3159, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 12359899, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3159, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12359899, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3159, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3159, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3159, 2024-03-23T15:53:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3159, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 936, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 107136, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 936, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 151, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 107136, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x559984fea500": { 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-03-23T15:53:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4583, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 12005444, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4583, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1082, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12005444, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:51.288 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-03-23T15:53:51.996 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:51 smithi183 ceph-mon[36296]: pgmap v326: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:52.023 INFO:tasks.check_counter:Daemon mds.h mds.exported=72 2024-03-23T15:53:52.023 INFO:tasks.check_counter:Daemon mds.h mds.imported=82 2024-03-23T15:53:52.023 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=4 2024-03-23T15:53:52.023 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-23T15:53:52.023 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T15:53:52.023 DEBUG:tasks.check_counter:Getting stats from k 2024-03-23T15:53:52.024 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-23T15:53:52.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:51 smithi192 ceph-mon[36653]: pgmap v326: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:52.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:51 smithi076 ceph-mon[32176]: pgmap v326: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:52.288 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.b/config 2024-03-23T15:53:53.954 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:53 smithi183 ceph-mon[36296]: pgmap v327: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 0, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 0, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 0, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.001034960, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T15:53:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000250013, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 162, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 320, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 183888, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.007967949, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.004693364, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.001758814, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:53:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 159, 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.009863080, 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000062031 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 183888 2024-03-23T15:53:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 227, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 215, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1435824, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 131072, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.035812509, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.008813396, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.033341242, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.003889109, 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 215, 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.007225143, 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000033605 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1435824, 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 131072 2024-03-23T15:53:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 149007, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 30, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-03-23T15:53:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 225, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1401360, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 225, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 225, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1401360, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-03-23T15:53:53.966 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-03-23T15:53:53.967 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-03-23T15:53:53.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:53.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-03-23T15:53:53.967 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-03-23T15:53:54.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:53 smithi192 ceph-mon[36653]: pgmap v327: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:54.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:53 smithi076 ceph-mon[32176]: pgmap v327: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:54.663 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-23T15:53:54.663 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-23T15:53:54.663 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-23T15:53:54.663 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-23T15:53:54.663 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-23T15:53:54.663 DEBUG:tasks.check_counter:Getting stats from c 2024-03-23T15:53:54.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-23T15:53:54.910 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:53:56.068 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:55 smithi192 ceph-mon[36653]: pgmap v328: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:56.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:55 smithi076 ceph-mon[32176]: pgmap v328: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:56.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:55 smithi183 ceph-mon[36296]: pgmap v328: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1606, 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1608, 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 67579457, 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 499476, 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 8, 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.134569103, 2024-03-23T15:53:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.030112730, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.058048356, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.036892258, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1606, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.037632306, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000023432 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 505, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 483, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 68685858, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 211314, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 6, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.077004542, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.013268949, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.064129024, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.008439678, 2024-03-23T15:53:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 482, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.015267643, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000031675 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1504592, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 131632 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 775, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 931, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 40574439, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 684104, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 8, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.073619863, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.021318113, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.031745120, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.015096355, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 929, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.029274385, 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000031511 2024-03-23T15:53:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 444944 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-03-23T15:53:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 4322, 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.374817807, 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000086723 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-03-23T15:53:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 39407, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 22560, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 16847, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 7179, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 2, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-03-23T15:53:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 4, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 234, 2024-03-23T15:53:56.802 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 6, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T15:53:56.803 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-03-23T15:53:56.804 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-03-23T15:53:56.805 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 69170, 2024-03-23T15:53:56.806 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 7, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 19, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 164903490, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 178782502, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 178782390, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.807 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 69170 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 13187, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 25264, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 12077, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 1628, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 1788, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 160, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 39407, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 40818, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 1411, 2024-03-23T15:53:56.808 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 134076, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215420 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.809 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.810 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.811 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.812 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.813 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.813 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-03-23T15:53:56.813 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.813 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.813 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.814 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-03-23T15:53:56.815 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.816 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.817 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.818 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.819 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.820 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 5, 2024-03-23T15:53:56.821 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 3, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 4, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 3414, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:56.822 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:56.823 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:56.824 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 823008, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 15851, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 8224, 2024-03-23T15:53:56.825 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 171, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 63107812, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 1354789, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-03-23T15:53:56.826 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 2631, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 26, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 4, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 2631, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2631, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2.899939644, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001102219 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2631, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2632 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.827 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 2631, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 2631, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 1595, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 1033, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-03-23T15:53:56.828 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 2, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-03-23T15:53:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-03-23T15:53:56.830 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 96, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 11, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 11, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 11, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-03-23T15:53:56.831 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T15:53:56.832 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 2882, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 176564622, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 2882, 2024-03-23T15:53:56.833 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2882, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 176564622, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:56.834 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2631, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 174600140, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2631, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 174600140, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.835 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2631, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 2631, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2631, 2024-03-23T15:53:56.836 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 2631, 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:56.837 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.838 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55b254bde6e0": { 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-03-23T15:53:56.839 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:56.840 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:56.840 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:56.840 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:56.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:56.840 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-23T15:53:57.663 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-23T15:53:57.663 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-23T15:53:57.663 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-23T15:53:57.663 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-23T15:53:57.663 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T15:53:57.663 DEBUG:tasks.check_counter:Getting stats from f 2024-03-23T15:53:57.664 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-23T15:53:57.914 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:53:57.999 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:57 smithi192 ceph-mon[36653]: pgmap v329: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:58.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:57 smithi076 ceph-mon[32176]: pgmap v329: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:58.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:57 smithi183 ceph-mon[36296]: pgmap v329: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-23T15:53:59.662 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-23T15:53:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:53:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1757, 2024-03-23T15:53:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1759, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 13088443, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 546434, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 6, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.114335121, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.033482640, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.032766582, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.039043968, 2024-03-23T15:53:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1757, 2024-03-23T15:53:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.040921996, 2024-03-23T15:53:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000023290 2024-03-23T15:53:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 569, 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 728, 2024-03-23T15:53:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 3113512, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 621071, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 7, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.043947471, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.014911484, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.013771741, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.010541197, 2024-03-23T15:53:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 725, 2024-03-23T15:53:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.025387161, 2024-03-23T15:53:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000035016 2024-03-23T15:53:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 444672 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:53:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 255, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 231, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1508268, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 133511, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 3, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.033371866, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.008501758, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.031426706, 2024-03-23T15:53:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.003524471, 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 231, 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.009239201, 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000039996 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1508000, 2024-03-23T15:53:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 133200 2024-03-23T15:53:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-23T15:53:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-03-23T15:53:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-23T15:53:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3967, 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.073171077, 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000018444 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-03-23T15:53:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-03-23T15:53:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-03-23T15:53:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 2968, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 2968, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 2968, 2024-03-23T15:53:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 1, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-03-23T15:53:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 0, 2024-03-23T15:53:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-03-23T15:53:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-03-23T15:53:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-03-23T15:53:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-03-23T15:53:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 5852, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 9, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 19743022, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 19739988, 2024-03-23T15:53:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 5852 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 797, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 2621, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 1824, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 137, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 442, 2024-03-23T15:53:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 305, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 2968, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 4293, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 1325, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 60200, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215420 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-03-23T15:53:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-03-23T15:53:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-03-23T15:53:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-03-23T15:53:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-03-23T15:53:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-03-23T15:53:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 5, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 3, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 4, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 3416, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:53:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-23T15:53:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 225492, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 1008, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:53:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 8224, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 171, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-23T15:53:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 4542110, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 304715, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 2325, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 21, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 3, 2024-03-23T15:53:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 2325, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2325, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.828962146, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000786650 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2325, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2326 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 2325, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 2325, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-03-23T15:53:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 1427, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 896, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-03-23T15:53:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 1, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-03-23T15:53:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-03-23T15:53:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 96, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 11, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 5, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 5, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-03-23T15:53:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 2577, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 17462483, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 2577, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:53:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2577, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 17462483, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2325, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 15548723, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2325, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 15548723, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-03-23T15:53:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2325, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 2325, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2325, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 2325, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-03-23T15:53:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x558dc3b9e6e0": { 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:53:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:53:59.716 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-23T15:53:59.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:53:59 smithi192 ceph-mon[36653]: pgmap v330: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:54:00.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:53:59 smithi076 ceph-mon[32176]: pgmap v330: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:54:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:53:59 smithi183 ceph-mon[36296]: pgmap v330: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:54:00.575 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-23T15:54:00.575 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-23T15:54:00.575 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-23T15:54:00.575 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-23T15:54:00.575 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T15:54:00.575 DEBUG:tasks.check_counter:Getting stats from i 2024-03-23T15:54:00.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-23T15:54:00.842 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:54:02.019 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:01 smithi192 ceph-mon[36653]: pgmap v331: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:54:02.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:01 smithi076 ceph-mon[32176]: pgmap v331: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:54:02.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:01 smithi183 ceph-mon[36296]: pgmap v331: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-23T15:54:02.616 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 7934, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 12690, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 4021510, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 19911294, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 20, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.586943393, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.254486283, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.316001282, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.058412783, 2024-03-23T15:54:02.617 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 12689, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.317559551, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000025026 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 13815, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.002287437, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000165 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1527088, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 168400 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:54:02.618 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 13873, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 11636, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 5046727, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 4224198, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 12, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.622641785, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.253348579, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.453435571, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.014003804, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 11634, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.238257545, 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000020479 2024-03-23T15:54:02.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 23851, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.002974777, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000124 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 4736, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 140848 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 3306, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 3435, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1135358, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 4239714, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 9, 2024-03-23T15:54:02.620 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 7, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.193753055, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.076843413, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.091673803, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.031873551, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3433, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.109827123, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000031991 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3780, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000468407, 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000123 2024-03-23T15:54:02.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 318704 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3424, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 3.595201222, 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001050000 2024-03-23T15:54:02.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2529, 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.156714117, 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000061966 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 3550, 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 3509, 2024-03-23T15:54:02.623 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3509, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 15.438939223, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.004399811 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 41, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 11, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 2001, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 7, 2024-03-23T15:54:02.624 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 4, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 1990, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 1990, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 1556, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 1, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 2, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 4, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 6670, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 5373, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 29, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 217, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 8, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 2, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-03-23T15:54:02.625 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 70, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 1035, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 80, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 713, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 362, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 11, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 4, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 234, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 4, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 362, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-03-23T15:54:02.626 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 11, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 11, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 351, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 92, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 840, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 2260, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 68, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 1832, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 882, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 135, 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 2345 2024-03-23T15:54:02.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 238, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 143, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 179, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 1185, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 997, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 27, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 1, 2024-03-23T15:54:02.628 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 71, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-03-23T15:54:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 5852, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 5852, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 9, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 9, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-03-23T15:54:02.630 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 19743022, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1550, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 5.583577026, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.003602307 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 811, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 2622, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 1811, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 189, 2024-03-23T15:54:02.631 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 484, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 295, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 2001, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 4554, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 2553, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 2, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 1717, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 1715, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 99872, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 223612 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 5957, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 946, 2024-03-23T15:54:02.632 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 3550, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 182, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 2116, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 270, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.511761458, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001895412 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 71, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.191634274, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002699074 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.633 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 71, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.485928100, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.006844057 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 962, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.478503750, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000497405 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-03-23T15:54:02.634 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000098954, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000098954 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-03-23T15:54:02.635 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.636 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 127, 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.229529660, 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001807320 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-03-23T15:54:02.637 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 143, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.241558959, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001689223 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.010628379, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005314189 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 224, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.211943207, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000946175 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 147, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2.310351242, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.015716675 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.639 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 208, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 6.157879238, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.029605188 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.640 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 69, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.167844521, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002432529 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.641 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-03-23T15:54:02.642 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 122, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.246620558, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002021479 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1092, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 4.194656923, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.003841260 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.643 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 5, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 3, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 2, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 1048, 2024-03-23T15:54:02.644 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 524, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 614, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:54:02.645 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:54:02.646 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 1756687, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 1270, 2024-03-23T15:54:02.647 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 176, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 2, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 8224, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 171, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 4047382, 2024-03-23T15:54:02.648 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 310772, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 3846, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 15809846, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 33, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 3846, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3846, 2024-03-23T15:54:02.649 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 12.824732424, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.003334563 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3846, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 4034 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 3846, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 538, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 3308, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 0, 2024-03-23T15:54:02.650 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 10, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 176, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 1726, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 117, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 18, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 1236, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 362, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 20, 2024-03-23T15:54:02.651 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 76, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-03-23T15:54:02.652 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 96, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 11, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 12, 2024-03-23T15:54:02.653 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 12, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 211, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 82 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-03-23T15:54:02.654 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 14, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 10, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 1239, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 997, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:54:02.655 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 25109, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 7924589, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 25109, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 25109, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 7924589, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 3846, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 15922903, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 3846, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 15922903, 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:54:02.657 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-03-23T15:54:02.658 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 3846, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 3846, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 3846, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 3846, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 997, 2024-03-23T15:54:02.659 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 113057, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 997, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 176, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 113057, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x562ff7b9a500": { 2024-03-23T15:54:02.660 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 5852, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 15548718, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 5852, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1550, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 15548718, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:02.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:02.662 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-23T15:54:03.408 INFO:tasks.check_counter:Daemon mds.i mds.exported=70 2024-03-23T15:54:03.408 INFO:tasks.check_counter:Daemon mds.i mds.imported=80 2024-03-23T15:54:03.408 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=4 2024-03-23T15:54:03.408 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-23T15:54:03.408 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-23T15:54:03.408 DEBUG:tasks.check_counter:Getting stats from l 2024-03-23T15:54:03.408 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de shell --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-23T15:54:03.657 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/mon.c/config 2024-03-23T15:54:04.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:03 smithi192 ceph-mon[36653]: pgmap v332: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:54:04.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:03 smithi076 ceph-mon[32176]: pgmap v332: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:54:04.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:03 smithi183 ceph-mon[36296]: pgmap v332: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:54:05.447 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 231, 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 219, 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1436912, 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 133632, 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.033539665, 2024-03-23T15:54:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.007959329, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.030900795, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.003717756, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 219, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.008392941, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000038323 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1436912, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 133632 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 165, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 320, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 184416, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-03-23T15:54:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.007066702, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.004460603, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001135280, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 162, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.014368364, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000088693 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 184416 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 0, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 0, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 0, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.000285235, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000000000, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000221746, 2024-03-23T15:54:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-03-23T15:54:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-03-23T15:54:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-03-23T15:54:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-03-23T15:54:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 149017, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 31, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2024-03-23T15:54:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-03-23T15:54:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-23T15:54:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-03-23T15:54:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-03-23T15:54:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-03-23T15:54:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 229, 2024-03-23T15:54:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1401880, 2024-03-23T15:54:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 229, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 229, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1401880, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-03-23T15:54:05.461 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-03-23T15:54:06.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:05 smithi192 ceph-mon[36653]: pgmap v333: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:54:06.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:05 smithi076 ceph-mon[32176]: pgmap v333: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:54:06.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:05 smithi183 ceph-mon[36296]: pgmap v333: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:54:06.222 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-23T15:54:06.222 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-23T15:54:06.222 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-23T15:54:06.222 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-23T15:54:06.222 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-23T15:54:06.223 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-23T15:54:06.234 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-23T15:54:06.235 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:54:06.235 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T15:54:06.256 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:54:06.256 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T15:54:06.316 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:54:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 -- 172.21.15.76:0/2376637311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 msgr2=0x7f549010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 --2- 172.21.15.76:0/2376637311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549010d4a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5488009930 tx=0x7f548802bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 -- 172.21.15.76:0/2376637311 shutdown_connections 2024-03-23T15:54:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 --2- 172.21.15.76:0/2376637311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549010d4a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 --2- 172.21.15.76:0/2376637311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54901005d0 0x7f5490100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 --2- 172.21.15.76:0/2376637311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5490106670 0x7f5490106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 -- 172.21.15.76:0/2376637311 >> 172.21.15.76:0/2376637311 conn(0x7f54900776b0 msgr2=0x7f5490077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:06.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 -- 172.21.15.76:0/2376637311 shutdown_connections 2024-03-23T15:54:06.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.474+0000 7f5495264640 1 -- 172.21.15.76:0/2376637311 wait complete. 2024-03-23T15:54:06.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 Processor -- start 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 -- start start 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54901005d0 0x7f549019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5490106670 0x7f5490196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5490196fb0 con 0x7f5490100f90 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5490197120 con 0x7f54901005d0 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f5495264640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f54901972e0 con 0x7f5490106670 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f548e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43200/0 (socket says 172.21.15.76:43200) 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.475+0000 7f548e7fc640 1 -- 172.21.15.76:0/2825128623 learned_addr learned my addr 172.21.15.76:0/2825128623 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548effd640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54901005d0 0x7f549019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548f7fe640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5490106670 0x7f5490196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548e7fc640 1 -- 172.21.15.76:0/2825128623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5490106670 msgr2=0x7f5490196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548e7fc640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5490106670 0x7f5490196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548e7fc640 1 -- 172.21.15.76:0/2825128623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54901005d0 msgr2=0x7f549019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548e7fc640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54901005d0 0x7f549019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548e7fc640 1 -- 172.21.15.76:0/2825128623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5480009840 con 0x7f5490100f90 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548effd640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54901005d0 0x7f549019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548f7fe640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5490106670 0x7f5490196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f548e7fc640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549019ccf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f548400e9f0 tx=0x7f548400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:06.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f5477fff640 1 -- 172.21.15.76:0/2825128623 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484009720 con 0x7f5490100f90 2024-03-23T15:54:06.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f5477fff640 1 -- 172.21.15.76:0/2825128623 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f548400cd80 con 0x7f5490100f90 2024-03-23T15:54:06.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5488009560 con 0x7f5490100f90 2024-03-23T15:54:06.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f5477fff640 1 -- 172.21.15.76:0/2825128623 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484010690 con 0x7f5490100f90 2024-03-23T15:54:06.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.476+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54901a3670 con 0x7f5490100f90 2024-03-23T15:54:06.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.477+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5490068950 con 0x7f5490100f90 2024-03-23T15:54:06.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.478+0000 7f5477fff640 1 -- 172.21.15.76:0/2825128623 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f54840046c0 con 0x7f5490100f90 2024-03-23T15:54:06.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.479+0000 7f5477fff640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54780a2230 0x7f54780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.479+0000 7f548effd640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54780a2230 0x7f54780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.479+0000 7f5477fff640 1 -- 172.21.15.76:0/2825128623 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f5484014070 con 0x7f5490100f90 2024-03-23T15:54:06.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.480+0000 7f548effd640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54780a2230 0x7f54780a46f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5480009fd0 tx=0x7f5480009500 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:06.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.484+0000 7f5477fff640 1 -- 172.21.15.76:0/2825128623 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f54840970a0 con 0x7f5490100f90 2024-03-23T15:54:06.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.660+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5490197e90 con 0x7f5490100f90 2024-03-23T15:54:06.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.661+0000 7f5477fff640 1 -- 172.21.15.76:0/2825128623 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23197 (secure 0 0 0) 0x7f548409a760 con 0x7f5490100f90 2024-03-23T15:54:06.665 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:54:06.666 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":96,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:53:13.770262+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/4023925236":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6801/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6800/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/1911361196":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/4249446235":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/3432469273":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:54:06.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.663+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54780a2230 msgr2=0x7f54780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54780a2230 0x7f54780a46f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5480009fd0 tx=0x7f5480009500 comp rx=0 tx=0).stop 2024-03-23T15:54:06.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 msgr2=0x7f549019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549019ccf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f548400e9f0 tx=0x7f548400edf0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 shutdown_connections 2024-03-23T15:54:06.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f54780a2230 0x7f54780a46f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5490106670 0x7f5490196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5490100f90 0x7f549019ccf0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 --2- 172.21.15.76:0/2825128623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54901005d0 0x7f549019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 >> 172.21.15.76:0/2825128623 conn(0x7f54900776b0 msgr2=0x7f54900fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:06.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 shutdown_connections 2024-03-23T15:54:06.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.664+0000 7f5495264640 1 -- 172.21.15.76:0/2825128623 wait complete. 2024-03-23T15:54:06.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T15:36:39.745272+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T15:41:16.242982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T15:41:17.182844+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T15:54:06.681 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T15:54:06.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 -- 172.21.15.76:0/2104570276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 msgr2=0x7f4a4c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2104570276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c100020 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009930 tx=0x7f4a3c02bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:06.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 -- 172.21.15.76:0/2104570276 shutdown_connections 2024-03-23T15:54:06.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2104570276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a4c100560 0x7f4a4c10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2104570276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c100020 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2104570276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c1076c0 0x7f4a4c107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 -- 172.21.15.76:0/2104570276 >> 172.21.15.76:0/2104570276 conn(0x7f4a4c076b20 msgr2=0x7f4a4c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 -- 172.21.15.76:0/2104570276 shutdown_connections 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.784+0000 7f4a5388d640 1 -- 172.21.15.76:0/2104570276 wait complete. 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 Processor -- start 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 -- start start 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c100560 0x7f4a4c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a4c1076c0 0x7f4a4c19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c19cbf0 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c19cd60 con 0x7f4a4c1076c0 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a5388d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c19cf20 con 0x7f4a4c100560 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43206/0 (socket says 172.21.15.76:43206) 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 -- 172.21.15.76:0/2506997036 learned_addr learned my addr 172.21.15.76:0/2506997036 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51e2a640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a4c1076c0 0x7f4a4c19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a50e28640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c100560 0x7f4a4c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 -- 172.21.15.76:0/2506997036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c100560 msgr2=0x7f4a4c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c100560 0x7f4a4c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 -- 172.21.15.76:0/2506997036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a4c1076c0 msgr2=0x7f4a4c19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a4c1076c0 0x7f4a4c19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a51629640 1 -- 172.21.15.76:0/2506997036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c009560 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.785+0000 7f4a50e28640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c100560 0x7f4a4c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.786+0000 7f4a51629640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c197d30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9f0 tx=0x7f4a4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.786+0000 7f4a427fc640 1 -- 172.21.15.76:0/2506997036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48009700 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.786+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c19d200 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.786+0000 7f4a427fc640 1 -- 172.21.15.76:0/2506997036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4800cd80 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.786+0000 7f4a427fc640 1 -- 172.21.15.76:0/2506997036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48018920 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.786+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1a4a80 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.787+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c068950 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.788+0000 7f4a427fc640 1 -- 172.21.15.76:0/2506997036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f4a48018a80 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.788+0000 7f4a427fc640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4a380a2230 0x7f4a380a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.788+0000 7f4a50e28640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4a380a2230 0x7f4a380a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:06.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.788+0000 7f4a427fc640 1 -- 172.21.15.76:0/2506997036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f4a48014070 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.789+0000 7f4a50e28640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4a380a2230 0x7f4a380a46f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c006010 tx=0x7f4a3c0047a0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:06.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.792+0000 7f4a427fc640 1 -- 172.21.15.76:0/2506997036 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f4a48097500 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.966+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4a4c109c80 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.967+0000 7f4a427fc640 1 -- 172.21.15.76:0/2506997036 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f4a4809abc0 con 0x7f4a4c0ffba0 2024-03-23T15:54:06.971 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4a380a2230 msgr2=0x7f4a380a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4a380a2230 0x7f4a380a46f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c006010 tx=0x7f4a3c0047a0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 msgr2=0x7f4a4c197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c197d30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9f0 tx=0x7f4a4800eeb0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 shutdown_connections 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f4a380a2230 0x7f4a380a46f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a4c1076c0 0x7f4a4c19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c100560 0x7f4a4c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 --2- 172.21.15.76:0/2506997036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a4c0ffba0 0x7f4a4c197d30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:06.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 >> 172.21.15.76:0/2506997036 conn(0x7f4a4c076b20 msgr2=0x7f4a4c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:06.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 shutdown_connections 2024-03-23T15:54:06.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:06.969+0000 7f4a5388d640 1 -- 172.21.15.76:0/2506997036 wait complete. 2024-03-23T15:54:06.982 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:06 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2825128623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:54:06.983 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T15:54:07.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:06 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2825128623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:54:07.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 -- 172.21.15.76:0/3708280956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 msgr2=0x7f902010d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 --2- 172.21.15.76:0/3708280956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902010d490 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9018009cd0 tx=0x7f901802c0f0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 -- 172.21.15.76:0/3708280956 shutdown_connections 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 --2- 172.21.15.76:0/3708280956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902010d490 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 --2- 172.21.15.76:0/3708280956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90201005c0 0x7f9020100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 --2- 172.21.15.76:0/3708280956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9020106660 0x7f9020106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 -- 172.21.15.76:0/3708280956 >> 172.21.15.76:0/3708280956 conn(0x7f9020076990 msgr2=0x7f9020076da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 -- 172.21.15.76:0/3708280956 shutdown_connections 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.084+0000 7f902720e640 1 -- 172.21.15.76:0/3708280956 wait complete. 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 Processor -- start 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 -- start start 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90201005c0 0x7f902019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020106660 0x7f9020197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9020198110 con 0x7f9020100f80 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9020198280 con 0x7f90201005c0 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f902720e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9020198440 con 0x7f9020106660 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902019de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43230/0 (socket says 172.21.15.76:43230) 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f90257ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020106660 0x7f9020197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f9024faa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90201005c0 0x7f902019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 -- 172.21.15.76:0/3131891613 learned_addr learned my addr 172.21.15.76:0/3131891613 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 -- 172.21.15.76:0/3131891613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020106660 msgr2=0x7f9020197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020106660 0x7f9020197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 -- 172.21.15.76:0/3131891613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90201005c0 msgr2=0x7f902019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90201005c0 0x7f902019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 -- 172.21.15.76:0/3131891613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9014009490 con 0x7f9020100f80 2024-03-23T15:54:07.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f9024faa640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90201005c0 0x7f902019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:54:07.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901ffff640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902019de50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f900c00e9f0 tx=0x7f900c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:07.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.085+0000 7f901dffb640 1 -- 172.21.15.76:0/3131891613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c009720 con 0x7f9020100f80 2024-03-23T15:54:07.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.086+0000 7f901dffb640 1 -- 172.21.15.76:0/3131891613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f900c00cd80 con 0x7f9020100f80 2024-03-23T15:54:07.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.086+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9018009900 con 0x7f9020100f80 2024-03-23T15:54:07.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.086+0000 7f901dffb640 1 -- 172.21.15.76:0/3131891613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c010690 con 0x7f9020100f80 2024-03-23T15:54:07.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.086+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902010dd90 con 0x7f9020100f80 2024-03-23T15:54:07.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.087+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9020101d00 con 0x7f9020100f80 2024-03-23T15:54:07.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.088+0000 7f901dffb640 1 -- 172.21.15.76:0/3131891613 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f900c0046c0 con 0x7f9020100f80 2024-03-23T15:54:07.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.088+0000 7f901dffb640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f90100a6640 0x7f90100a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.088+0000 7f9024faa640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f90100a6640 0x7f90100a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.088+0000 7f901dffb640 1 -- 172.21.15.76:0/3131891613 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f900c014070 con 0x7f9020100f80 2024-03-23T15:54:07.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.089+0000 7f9024faa640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f90100a6640 0x7f90100a8b00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9020198c00 tx=0x7f9014005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:07.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.092+0000 7f901dffb640 1 -- 172.21.15.76:0/3131891613 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f900c0972d0 con 0x7f9020100f80 2024-03-23T15:54:07.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:06 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2825128623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:54:07.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.264+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9020108bb0 con 0x7f9020100f80 2024-03-23T15:54:07.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.264+0000 7f901dffb640 1 -- 172.21.15.76:0/3131891613 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f900c09a990 con 0x7f9020100f80 2024-03-23T15:54:07.268 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f90100a6640 msgr2=0x7f90100a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f90100a6640 0x7f90100a8b00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9020198c00 tx=0x7f9014005f90 comp rx=0 tx=0).stop 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 msgr2=0x7f902019de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902019de50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f900c00e9f0 tx=0x7f900c00edf0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 shutdown_connections 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f90100a6640 0x7f90100a8b00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9020106660 0x7f9020197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9020100f80 0x7f902019de50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 --2- 172.21.15.76:0/3131891613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90201005c0 0x7f902019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 >> 172.21.15.76:0/3131891613 conn(0x7f9020076990 msgr2=0x7f90200fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:07.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 shutdown_connections 2024-03-23T15:54:07.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.266+0000 7f902720e640 1 -- 172.21.15.76:0/3131891613 wait complete. 2024-03-23T15:54:07.280 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T15:54:07.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 -- 172.21.15.76:0/2067772007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848107860 msgr2=0x7fd848107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/2067772007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848107860 0x7fd848107c40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd838009930 tx=0x7fd83802bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 -- 172.21.15.76:0/2067772007 shutdown_connections 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/2067772007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd84806a720 0x7fd84810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/2067772007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd848069d80 0x7fd84806a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/2067772007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848107860 0x7fd848107c40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 -- 172.21.15.76:0/2067772007 >> 172.21.15.76:0/2067772007 conn(0x7fd8480773b0 msgr2=0x7fd8480777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 -- 172.21.15.76:0/2067772007 shutdown_connections 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 -- 172.21.15.76:0/2067772007 wait complete. 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.382+0000 7fd84cc9d640 1 Processor -- start 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84cc9d640 1 -- start start 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84cc9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848069d80 0x7fd84819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84cc9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd84806a720 0x7fd84819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84cc9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd848107860 0x7fd848197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84cc9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd848198100 con 0x7fd848069d80 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84cc9d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd848198270 con 0x7fd848107860 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84cc9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd848198430 con 0x7fd84806a720 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848069d80 0x7fd84819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848069d80 0x7fd84819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43262/0 (socket says 172.21.15.76:43262) 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 -- 172.21.15.76:0/40914465 learned_addr learned my addr 172.21.15.76:0/40914465 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd83ffff640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd84806a720 0x7fd84819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd846d9d640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd848107860 0x7fd848197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 -- 172.21.15.76:0/40914465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd84806a720 msgr2=0x7fd84819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd84806a720 0x7fd84819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 -- 172.21.15.76:0/40914465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd848107860 msgr2=0x7fd848197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd848107860 0x7fd848197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 -- 172.21.15.76:0/40914465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd838009560 con 0x7fd848069d80 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd84659c640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848069d80 0x7fd84819d900 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd838009230 tx=0x7fd83802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd83f7fe640 1 -- 172.21.15.76:0/40914465 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838004350 con 0x7fd848069d80 2024-03-23T15:54:07.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.383+0000 7fd83f7fe640 1 -- 172.21.15.76:0/40914465 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8380044b0 con 0x7fd848069d80 2024-03-23T15:54:07.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.384+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8481986b0 con 0x7fd848069d80 2024-03-23T15:54:07.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.384+0000 7fd83f7fe640 1 -- 172.21.15.76:0/40914465 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838003d00 con 0x7fd848069d80 2024-03-23T15:54:07.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.384+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8481a49a0 con 0x7fd848069d80 2024-03-23T15:54:07.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.385+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd848102dc0 con 0x7fd848069d80 2024-03-23T15:54:07.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.386+0000 7fd83f7fe640 1 -- 172.21.15.76:0/40914465 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fd838004df0 con 0x7fd848069d80 2024-03-23T15:54:07.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.386+0000 7fd83f7fe640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8340a2230 0x7fd8340a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.386+0000 7fd83ffff640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8340a2230 0x7fd8340a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.386+0000 7fd83f7fe640 1 -- 172.21.15.76:0/40914465 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd8380eff80 con 0x7fd848069d80 2024-03-23T15:54:07.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.387+0000 7fd83ffff640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8340a2230 0x7fd8340a46f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd848106d40 tx=0x7fd830005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:07.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.391+0000 7fd83f7fe640 1 -- 172.21.15.76:0/40914465 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fd8380b8cf0 con 0x7fd848069d80 2024-03-23T15:54:07.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.563+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd848109bf0 con 0x7fd848069d80 2024-03-23T15:54:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.563+0000 7fd83f7fe640 1 -- 172.21.15.76:0/40914465 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7fd838003e60 con 0x7fd848069d80 2024-03-23T15:54:07.570 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8340a2230 msgr2=0x7fd8340a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8340a2230 0x7fd8340a46f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd848106d40 tx=0x7fd830005f90 comp rx=0 tx=0).stop 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848069d80 msgr2=0x7fd84819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848069d80 0x7fd84819d900 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd838009230 tx=0x7fd83802c650 comp rx=0 tx=0).stop 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 shutdown_connections 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fd8340a2230 0x7fd8340a46f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd848107860 0x7fd848197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd84806a720 0x7fd84819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 --2- 172.21.15.76:0/40914465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd848069d80 0x7fd84819d900 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.565+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 >> 172.21.15.76:0/40914465 conn(0x7fd8480773b0 msgr2=0x7fd8480ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.566+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 shutdown_connections 2024-03-23T15:54:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.566+0000 7fd84cc9d640 1 -- 172.21.15.76:0/40914465 wait complete. 2024-03-23T15:54:07.581 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T15:54:07.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 -- 172.21.15.76:0/2299739927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 msgr2=0x7fc6d4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 --2- 172.21.15.76:0/2299739927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d4106a40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009930 tx=0x7fc6c402bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 -- 172.21.15.76:0/2299739927 shutdown_connections 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 --2- 172.21.15.76:0/2299739927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4100f80 0x7fc6d410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 --2- 172.21.15.76:0/2299739927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d41005c0 0x7fc6d4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 --2- 172.21.15.76:0/2299739927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d4106a40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 -- 172.21.15.76:0/2299739927 >> 172.21.15.76:0/2299739927 conn(0x7fc6d4076990 msgr2=0x7fc6d4076da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 -- 172.21.15.76:0/2299739927 shutdown_connections 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 -- 172.21.15.76:0/2299739927 wait complete. 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 Processor -- start 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.681+0000 7fc6da86b640 1 -- start start 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6da86b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d41005c0 0x7fc6d4197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6da86b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4100f80 0x7fc6d4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6da86b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6da86b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d419cbe0 con 0x7fc6d4106660 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6da86b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d419cd50 con 0x7fc6d4100f80 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6da86b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d419cf10 con 0x7fc6d41005c0 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d419c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43282/0 (socket says 172.21.15.76:43282) 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 -- 172.21.15.76:0/416523041 learned_addr learned my addr 172.21.15.76:0/416523041 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d37fe640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4100f80 0x7fc6d4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d3fff640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d41005c0 0x7fc6d4197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 -- 172.21.15.76:0/416523041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d41005c0 msgr2=0x7fc6d4197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d41005c0 0x7fc6d4197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 -- 172.21.15.76:0/416523041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4100f80 msgr2=0x7fc6d4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4100f80 0x7fc6d4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 -- 172.21.15.76:0/416523041 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c4009560 con 0x7fc6d4106660 2024-03-23T15:54:07.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d8e08640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d419c5e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00eb10 tx=0x7fc6cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:07.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d17fa640 1 -- 172.21.15.76:0/416523041 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc004b20 con 0x7fc6d4106660 2024-03-23T15:54:07.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.682+0000 7fc6d17fa640 1 -- 172.21.15.76:0/416523041 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6cc004c80 con 0x7fc6d4106660 2024-03-23T15:54:07.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.683+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d419d1f0 con 0x7fc6d4106660 2024-03-23T15:54:07.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.683+0000 7fc6d17fa640 1 -- 172.21.15.76:0/416523041 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc010650 con 0x7fc6d4106660 2024-03-23T15:54:07.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.683+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d41a4a70 con 0x7fc6d4106660 2024-03-23T15:54:07.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.683+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d4101d00 con 0x7fc6d4106660 2024-03-23T15:54:07.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.684+0000 7fc6d17fa640 1 -- 172.21.15.76:0/416523041 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fc6cc005020 con 0x7fc6d4106660 2024-03-23T15:54:07.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.685+0000 7fc6d17fa640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fc6bc0a2230 0x7fc6bc0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:07.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.685+0000 7fc6d3fff640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fc6bc0a2230 0x7fc6bc0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:07.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.685+0000 7fc6d17fa640 1 -- 172.21.15.76:0/416523041 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fc6cc014070 con 0x7fc6d4106660 2024-03-23T15:54:07.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.685+0000 7fc6d3fff640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fc6bc0a2230 0x7fc6bc0a46f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4000c00 tx=0x7fc6c4005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:07.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.689+0000 7fc6d17fa640 1 -- 172.21.15.76:0/416523041 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fc6cc0972e0 con 0x7fc6d4106660 2024-03-23T15:54:07.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.877+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc6d4108bc0 con 0x7fc6d4106660 2024-03-23T15:54:07.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.877+0000 7fc6d17fa640 1 -- 172.21.15.76:0/416523041 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) ==== 81+0+940 (secure 0 0 0) 0x7fc6d4108bc0 con 0x7fc6d4106660 2024-03-23T15:54:07.881 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/4023925236 2024-03-24T15:43:19.788306+0000 2024-03-23T15:54:07.881 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6801/2479514030 2024-03-24T15:43:19.788306+0000 2024-03-23T15:54:07.881 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6800/2479514030 2024-03-24T15:43:19.788306+0000 2024-03-23T15:54:07.881 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/1911361196 2024-03-24T15:43:19.788306+0000 2024-03-23T15:54:07.881 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/4249446235 2024-03-24T15:43:19.788306+0000 2024-03-23T15:54:07.881 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/3432469273 2024-03-24T15:43:19.788306+0000 2024-03-23T15:54:07.881 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/871302578 2024-03-24T15:34:15.395471+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/3336791879 2024-03-24T15:34:15.395471+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6801/36977200 2024-03-24T15:34:15.395471+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/2354876004 2024-03-24T15:33:50.341648+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/2171503933 2024-03-24T15:33:50.341648+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6800/36977200 2024-03-24T15:34:15.395471+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/4014456486 2024-03-24T15:33:50.341648+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:0/542720275 2024-03-24T15:34:15.395471+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6800/3268079315 2024-03-24T15:33:50.341648+0000 2024-03-23T15:54:07.882 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.15.76:6801/3268079315 2024-03-24T15:33:50.341648+0000 2024-03-23T15:54:07.883 INFO:teuthology.orchestra.run.smithi076.stderr:listed 16 entries 2024-03-23T15:54:07.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fc6bc0a2230 msgr2=0x7fc6bc0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fc6bc0a2230 0x7fc6bc0a46f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4000c00 tx=0x7fc6c4005fd0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 msgr2=0x7fc6d419c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d419c5e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00eb10 tx=0x7fc6cc00c2d0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 shutdown_connections 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fc6bc0a2230 0x7fc6bc0a46f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6d4106660 0x7fc6d419c5e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4100f80 0x7fc6d4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 --2- 172.21.15.76:0/416523041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6d41005c0 0x7fc6d4197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 >> 172.21.15.76:0/416523041 conn(0x7fc6d4076990 msgr2=0x7fc6d40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 shutdown_connections 2024-03-23T15:54:07.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:07.879+0000 7fc6da86b640 1 -- 172.21.15.76:0/416523041 wait complete. 2024-03-23T15:54:07.892 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:07 smithi076 ceph-mon[32176]: pgmap v334: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:54:07.892 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:07 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2506997036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:54:07.893 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:07 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3131891613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:54:07.893 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:07 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/40914465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:54:07.894 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-23T15:54:07.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:54:07.894 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-23T15:54:07.946 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:54:07.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:54:07.947 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T15:54:07.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:54:07.990 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-23T15:54:08.063 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T15:54:08.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:54:08.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T15:54:08.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:07 smithi192 ceph-mon[36653]: pgmap v334: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:54:08.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:07 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2506997036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:54:08.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:07 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3131891613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:54:08.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:07 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/40914465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:54:08.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:07 smithi183 ceph-mon[36296]: pgmap v334: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-23T15:54:08.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:07 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2506997036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T15:54:08.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:07 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3131891613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-23T15:54:08.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:07 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/40914465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-23T15:54:08.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:54:08.152 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-03-23T15:54:08.211 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-23T15:54:08.211 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-03-23T15:54:08.211 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T15:54:08.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=707 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/e8a451dd4bde82a8d5284b3c0098747c4fff0b0386541631450a23124aca90ec/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/e8a451dd4bde82a8d5284b3c0098747c4fff0b0386541631450a23124aca90ec/diff,workdir=/var/lib/containers/storage/overlay/e8a451dd4bde82a8d5284b3c0098747c4fff0b0386541631450a23124aca90ec/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/033e7338fdf0892776395b0f46e2981f18399db3c1c33c043301b9e67551c5be/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/033e7338fdf0892776395b0f46e2981f18399db3c1c33c043301b9e67551c5be/diff,workdir=/var/lib/containers/storage/overlay/033e7338fdf0892776395b0f46e2981f18399db3c1c33c043301b9e67551c5be/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/ff5add3da27a75c58dd0f88cecce48e80e77a07fea82fa422224f0973422f112/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/ff5add3da27a75c58dd0f88cecce48e80e77a07fea82fa422224f0973422f112/diff,workdir=/var/lib/containers/storage/overlay/ff5add3da27a75c58dd0f88cecce48e80e77a07fea82fa422224f0973422f112/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e505aef935e021fcc6cb6a0981220305bce8c08b01c76502fe0ca74dfcb6638d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/e505aef935e021fcc6cb6a0981220305bce8c08b01c76502fe0ca74dfcb6638d/diff,workdir=/var/lib/containers/storage/overlay/e505aef935e021fcc6cb6a0981220305bce8c08b01c76502fe0ca74dfcb6638d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/04e05aaced26703517ec70281f15691af22f279561ce8a5976da080189f08474/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/04e05aaced26703517ec70281f15691af22f279561ce8a5976da080189f08474/diff,workdir=/var/lib/containers/storage/overlay/04e05aaced26703517ec70281f15691af22f279561ce8a5976da080189f08474/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b97158e8e2a8cdd0436738b76e6d6267d3d96a5438dff806c6cb5680801590ca/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/b97158e8e2a8cdd0436738b76e6d6267d3d96a5438dff806c6cb5680801590ca/diff,workdir=/var/lib/containers/storage/overlay/b97158e8e2a8cdd0436738b76e6d6267d3d96a5438dff806c6cb5680801590ca/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/99ad22f6339bbe58d8bc7862933f8a207ecb462a02b0951f26d9eb84c0de982d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/99ad22f6339bbe58d8bc7862933f8a207ecb462a02b0951f26d9eb84c0de982d/diff,workdir=/var/lib/containers/storage/overlay/99ad22f6339bbe58d8bc7862933f8a207ecb462a02b0951f26d9eb84c0de982d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/59b2412b97d01f2ad1a77c1c976c410c68a5923979d4dd67ca79e5e371f1d113/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/59b2412b97d01f2ad1a77c1c976c410c68a5923979d4dd67ca79e5e371f1d113/diff,workdir=/var/lib/containers/storage/overlay/59b2412b97d01f2ad1a77c1c976c410c68a5923979d4dd67ca79e5e371f1d113/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4e1f3439f29c6a8955358fb1a68681c2bb942105ba86f8a404a31aa4fa8d2d1b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/4e1f3439f29c6a8955358fb1a68681c2bb942105ba86f8a404a31aa4fa8d2d1b/diff,workdir=/var/lib/containers/storage/overlay/4e1f3439f29c6a8955358fb1a68681c2bb942105ba86f8a404a31aa4fa8d2d1b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4088f36312385329602356b0db01475280a545d7abb59313fc16903e8a1dfbda/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KM666LIRTDKSA2CBMUVZ2A447F:/var/lib/containers/storage/overlay/l/ATN2CKOAFE4AKVHSSNBGW65YUH,upperdir=/var/lib/containers/storage/overlay/4088f36312385329602356b0db01475280a545d7abb59313fc16903e8a1dfbda/diff,workdir=/var/lib/containers/storage/overlay/4088f36312385329602356b0db01475280a545d7abb59313fc16903e8a1dfbda/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 2024-03-23T15:54:08.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:54:08.228 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-03-23T15:54:08.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T15:54:08.285 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-03-23T15:54:08.355 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2024-03-23T15:54:08.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:54:08.357 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-23T15:54:08.369 INFO:tasks.cephadm:Teardown begin 2024-03-23T15:54:08.369 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T15:54:08.369 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:54:08.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.496+0000 7f5c702c0640 1 -- 172.21.15.76:0/3437815769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 msgr2=0x7f5c68106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.496+0000 7f5c702c0640 1 --2- 172.21.15.76:0/3437815769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68106a70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009900 tx=0x7f5c6402bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:08.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- 172.21.15.76:0/3437815769 shutdown_connections 2024-03-23T15:54:08.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 --2- 172.21.15.76:0/3437815769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c68100f90 0x7f5c6810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 --2- 172.21.15.76:0/3437815769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c681005d0 0x7f5c68100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 --2- 172.21.15.76:0/3437815769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68106a70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- 172.21.15.76:0/3437815769 >> 172.21.15.76:0/3437815769 conn(0x7f5c68076990 msgr2=0x7f5c68076da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- 172.21.15.76:0/3437815769 shutdown_connections 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- 172.21.15.76:0/3437815769 wait complete. 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 Processor -- start 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- start start 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c681005d0 0x7f5c6819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c68100f90 0x7f5c6819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68198130 con 0x7f5c68106690 2024-03-23T15:54:08.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681982a0 con 0x7f5c681005d0 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.497+0000 7f5c702c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68198460 con 0x7f5c68100f90 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e05c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c681005d0 0x7f5c6819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43304/0 (socket says 172.21.15.76:43304) 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 -- 172.21.15.76:0/2850955574 learned_addr learned my addr 172.21.15.76:0/2850955574 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6d85b640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c68100f90 0x7f5c6819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 -- 172.21.15.76:0/2850955574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c68100f90 msgr2=0x7f5c6819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c68100f90 0x7f5c6819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 -- 172.21.15.76:0/2850955574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c681005d0 msgr2=0x7f5c6819d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c681005d0 0x7f5c6819d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 -- 172.21.15.76:0/2850955574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c64009560 con 0x7f5c68106690 2024-03-23T15:54:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c6e85d640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68197ae0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000eb10 tx=0x7f5c6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:08.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c5f7fe640 1 -- 172.21.15.76:0/2850955574 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60004b20 con 0x7f5c68106690 2024-03-23T15:54:08.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c68198740 con 0x7f5c68106690 2024-03-23T15:54:08.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c5f7fe640 1 -- 172.21.15.76:0/2850955574 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c60004c80 con 0x7f5c68106690 2024-03-23T15:54:08.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c5f7fe640 1 -- 172.21.15.76:0/2850955574 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60010650 con 0x7f5c68106690 2024-03-23T15:54:08.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.498+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c681a4980 con 0x7f5c68106690 2024-03-23T15:54:08.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.499+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c68101cf0 con 0x7f5c68106690 2024-03-23T15:54:08.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.500+0000 7f5c5f7fe640 1 -- 172.21.15.76:0/2850955574 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f5c60005020 con 0x7f5c68106690 2024-03-23T15:54:08.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.501+0000 7f5c5f7fe640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5c540a2230 0x7f5c540a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.501+0000 7f5c6e05c640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5c540a2230 0x7f5c540a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.501+0000 7f5c5f7fe640 1 -- 172.21.15.76:0/2850955574 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f5c60014070 con 0x7f5c68106690 2024-03-23T15:54:08.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.501+0000 7f5c6e05c640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5c540a2230 0x7f5c540a46f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009230 tx=0x7f5c64005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.505+0000 7f5c5f7fe640 1 -- 172.21.15.76:0/2850955574 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f5c60097270 con 0x7f5c68106690 2024-03-23T15:54:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.716+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c68068950 con 0x7f5c68106690 2024-03-23T15:54:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.717+0000 7f5c5f7fe640 1 -- 172.21.15.76:0/2850955574 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11874 (secure 0 0 0) 0x7f5c6009a930 con 0x7f5c68106690 2024-03-23T15:54:08.721 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:54:08.721 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:50:25.940983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":60,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":57,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:54:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 33 2024-03-23T15:54:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5c540a2230 msgr2=0x7f5c540a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5c540a2230 0x7f5c540a46f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009230 tx=0x7f5c64005fd0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 msgr2=0x7f5c68197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68197ae0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000eb10 tx=0x7f5c6000c2d0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 shutdown_connections 2024-03-23T15:54:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f5c540a2230 0x7f5c540a46f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c68106690 0x7f5c68197ae0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c68100f90 0x7f5c6819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 --2- 172.21.15.76:0/2850955574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c681005d0 0x7f5c6819d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.719+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 >> 172.21.15.76:0/2850955574 conn(0x7f5c68076990 msgr2=0x7f5c680feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.720+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 shutdown_connections 2024-03-23T15:54:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.720+0000 7f5c702c0640 1 -- 172.21.15.76:0/2850955574 wait complete. 2024-03-23T15:54:08.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T15:54:08.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 -- 172.21.15.76:0/2537546617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 msgr2=0x7f3f4810d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 --2- 172.21.15.76:0/2537546617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4810d450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009900 tx=0x7f3f3802bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 -- 172.21.15.76:0/2537546617 shutdown_connections 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 --2- 172.21.15.76:0/2537546617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4810d450 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 --2- 172.21.15.76:0/2537546617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f481005d0 0x7f3f48100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 --2- 172.21.15.76:0/2537546617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f48106690 0x7f3f48106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 -- 172.21.15.76:0/2537546617 >> 172.21.15.76:0/2537546617 conn(0x7f3f48076990 msgr2=0x7f3f48076da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 -- 172.21.15.76:0/2537546617 shutdown_connections 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 -- 172.21.15.76:0/2537546617 wait complete. 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 Processor -- start 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.848+0000 7f3f4d302640 1 -- start start 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f4d302640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f481005d0 0x7f3f4819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f4d302640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f4d302640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f48106690 0x7f3f48196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f4d302640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3f48196fc0 con 0x7f3f48100f90 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f4d302640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f48197130 con 0x7f3f48106690 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f4d302640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f481972f0 con 0x7f3f481005d0 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43314/0 (socket says 172.21.15.76:43314) 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f46ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f481005d0 0x7f3f4819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 -- 172.21.15.76:0/1463613065 learned_addr learned my addr 172.21.15.76:0/1463613065 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f477fe640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f48106690 0x7f3f48196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 -- 172.21.15.76:0/1463613065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f481005d0 msgr2=0x7f3f4819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f481005d0 0x7f3f4819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 -- 172.21.15.76:0/1463613065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f48106690 msgr2=0x7f3f48196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f48106690 0x7f3f48196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 -- 172.21.15.76:0/1463613065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f38009560 con 0x7f3f48100f90 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f477fe640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f48106690 0x7f3f48196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f46ffd640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f481005d0 0x7f3f4819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T15:54:08.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f467fc640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4819ccb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00eb90 tx=0x7f3f3c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:08.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f2ffff640 1 -- 172.21.15.76:0/1463613065 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0099b0 con 0x7f3f48100f90 2024-03-23T15:54:08.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f481975d0 con 0x7f3f48100f90 2024-03-23T15:54:08.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f2ffff640 1 -- 172.21.15.76:0/1463613065 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f3c00ee90 con 0x7f3f48100f90 2024-03-23T15:54:08.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.849+0000 7f3f2ffff640 1 -- 172.21.15.76:0/1463613065 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0107a0 con 0x7f3f48100f90 2024-03-23T15:54:08.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.850+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f481a3810 con 0x7f3f48100f90 2024-03-23T15:54:08.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.851+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f48101cf0 con 0x7f3f48100f90 2024-03-23T15:54:08.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.851+0000 7f3f2ffff640 1 -- 172.21.15.76:0/1463613065 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f3f3c005020 con 0x7f3f48100f90 2024-03-23T15:54:08.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.852+0000 7f3f2ffff640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3f300a2230 0x7f3f300a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:08.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.852+0000 7f3f46ffd640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3f300a2230 0x7f3f300a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:08.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.852+0000 7f3f2ffff640 1 -- 172.21.15.76:0/1463613065 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8615+0+0 (secure 0 0 0) 0x7f3f3c014070 con 0x7f3f48100f90 2024-03-23T15:54:08.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.852+0000 7f3f46ffd640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3f300a2230 0x7f3f300a46f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f48105730 tx=0x7f3f40005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:08.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:08.856+0000 7f3f2ffff640 1 -- 172.21.15.76:0/1463613065 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f3f3c0974c0 con 0x7f3f48100f90 2024-03-23T15:54:09.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:08 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/416523041' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T15:54:09.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:08 smithi192 ceph-mon[36653]: pgmap v335: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:54:09.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:08 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/2850955574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:54:09.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:08 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/416523041' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T15:54:09.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:08 smithi076 ceph-mon[32176]: pgmap v335: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:54:09.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:08 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/2850955574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:54:09.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.088+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f48197e50 con 0x7f3f48100f90 2024-03-23T15:54:09.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.089+0000 7f3f2ffff640 1 -- 172.21.15.76:0/1463613065 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11874 (secure 0 0 0) 0x7f3f3c09ab80 con 0x7f3f48100f90 2024-03-23T15:54:09.092 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:54:09.092 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1996137428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1996137428},{"type":"v1","addr":"172.21.15.76:6835","nonce":1996137428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3380265538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3380265538},{"type":"v1","addr":"172.21.15.76:6837","nonce":3380265538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/2542851051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":2542851051},{"type":"v1","addr":"172.21.15.76:6839","nonce":2542851051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6841/925396691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6840","nonce":925396691},{"type":"v1","addr":"172.21.15.76:6841","nonce":925396691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/2543151700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2543151700},{"type":"v1","addr":"172.21.15.183:6833","nonce":2543151700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/996338359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":996338359},{"type":"v1","addr":"172.21.15.192:6833","nonce":996338359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T15:41:17.990893+0000","modified":"2024-03-23T15:50:25.940983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24463,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1781188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1781188743},{"type":"v1","addr":"172.21.15.183:6835","nonce":1781188743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/1421961871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1421961871},{"type":"v1","addr":"172.21.15.192:6835","nonce":1421961871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/26718199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":26718199},{"type":"v1","addr":"172.21.15.192:6837","nonce":26718199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":60,"addr":"172.21.15.183:6837/3732793280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":3732793280},{"type":"v1","addr":"172.21.15.183:6837","nonce":3732793280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":57,"addr":"172.21.15.192:6839/2120704815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":2120704815},{"type":"v1","addr":"172.21.15.192:6839","nonce":2120704815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1020785546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1020785546},{"type":"v1","addr":"172.21.15.183:6839","nonce":1020785546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24449,24463]},"id":1}]} 2024-03-23T15:54:09.093 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 33 2024-03-23T15:54:09.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3f300a2230 msgr2=0x7f3f300a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3f300a2230 0x7f3f300a46f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f48105730 tx=0x7f3f40005f90 comp rx=0 tx=0).stop 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 msgr2=0x7f3f4819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4819ccb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00eb90 tx=0x7f3f3c00c6a0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 shutdown_connections 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f3f300a2230 0x7f3f300a46f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f48106690 0x7f3f48196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f48100f90 0x7f3f4819ccb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 --2- 172.21.15.76:0/1463613065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f481005d0 0x7f3f4819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 >> 172.21.15.76:0/1463613065 conn(0x7f3f48076990 msgr2=0x7f3f480fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 shutdown_connections 2024-03-23T15:54:09.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.091+0000 7f3f4d302640 1 -- 172.21.15.76:0/1463613065 wait complete. 2024-03-23T15:54:09.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T15:54:09.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/416523041' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T15:54:09.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:08 smithi183 ceph-mon[36296]: pgmap v335: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-23T15:54:09.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:08 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/2850955574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:54:09.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3606624842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 msgr2=0x7f6f38101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3606624842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38101b60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009930 tx=0x7f6f2802bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3606624842 shutdown_connections 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3606624842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f381020a0 0x7f6f3810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3606624842 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38101b60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3606624842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f381076f0 0x7f6f38107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3606624842 >> 172.21.15.76:0/3606624842 conn(0x7f6f38078ba0 msgr2=0x7f6f38078fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3606624842 shutdown_connections 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3606624842 wait complete. 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 Processor -- start 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.207+0000 7f6f3e3b5640 1 -- start start 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f381020a0 0x7f6f381982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f381076f0 0x7f6f3819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3819cc40 con 0x7f6f381016e0 2024-03-23T15:54:09.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3819cdb0 con 0x7f6f381020a0 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3819cf70 con 0x7f6f381076f0 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43328/0 (socket says 172.21.15.76:43328) 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 -- 172.21.15.76:0/3144694026 learned_addr learned my addr 172.21.15.76:0/3144694026 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f377fe640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f381020a0 0x7f6f381982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3c952640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f381076f0 0x7f6f3819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 -- 172.21.15.76:0/3144694026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f381076f0 msgr2=0x7f6f3819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f381076f0 0x7f6f3819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 -- 172.21.15.76:0/3144694026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f381020a0 msgr2=0x7f6f381982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f381020a0 0x7f6f381982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 -- 172.21.15.76:0/3144694026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f28009560 con 0x7f6f381016e0 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f37fff640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38197d80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9f0 tx=0x7f6f2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:09.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f357fa640 1 -- 172.21.15.76:0/3144694026 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c009720 con 0x7f6f381016e0 2024-03-23T15:54:09.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f357fa640 1 -- 172.21.15.76:0/3144694026 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c00cd80 con 0x7f6f381016e0 2024-03-23T15:54:09.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f357fa640 1 -- 172.21.15.76:0/3144694026 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c010690 con 0x7f6f381016e0 2024-03-23T15:54:09.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3819d250 con 0x7f6f381016e0 2024-03-23T15:54:09.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.208+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f381a4ad0 con 0x7f6f381016e0 2024-03-23T15:54:09.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.210+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f38068950 con 0x7f6f381016e0 2024-03-23T15:54:09.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.210+0000 7f6f357fa640 1 -- 172.21.15.76:0/3144694026 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f6f2c0046c0 con 0x7f6f381016e0 2024-03-23T15:54:09.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.211+0000 7f6f357fa640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6f200a2230 0x7f6f200a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.211+0000 7f6f377fe640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6f200a2230 0x7f6f200a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.211+0000 7f6f357fa640 1 -- 172.21.15.76:0/3144694026 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8615+0+0 (secure 0 0 0) 0x7f6f2c014070 con 0x7f6f381016e0 2024-03-23T15:54:09.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.211+0000 7f6f377fe640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6f200a2230 0x7f6f200a46f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f28005ab0 tx=0x7f6f28005fd0 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:09.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.215+0000 7f6f357fa640 1 -- 172.21.15.76:0/3144694026 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f6f2c097030 con 0x7f6f381016e0 2024-03-23T15:54:09.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.393+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f381990d0 con 0x7f6f381016e0 2024-03-23T15:54:09.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.394+0000 7f6f357fa640 1 -- 172.21.15.76:0/3144694026 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23264 (secure 0 0 0) 0x7f6f2c09a6f0 con 0x7f6f381016e0 2024-03-23T15:54:09.398 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-03-23T15:54:09.398 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":97,"fsid":"89c3e582-e92a-11ee-95cd-87774f69a715","created":"2024-03-23T15:33:18.792286+0000","modified":"2024-03-23T15:54:08.731491+0000","last_up_change":"2024-03-23T15:40:28.265537+0000","last_in_change":"2024-03-23T15:40:07.668488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T15:36:39.745272+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T15:41:16.242982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T15:41:17.182844+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ec188648-1f9e-47f7-a11c-31c3080d91dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6803","nonce":4179784307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6805","nonce":4179784307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6809","nonce":4179784307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":4179784307},{"type":"v1","addr":"172.21.15.76:6807","nonce":4179784307}]},"public_addr":"172.21.15.76:6803/4179784307","cluster_addr":"172.21.15.76:6805/4179784307","heartbeat_back_addr":"172.21.15.76:6809/4179784307","heartbeat_front_addr":"172.21.15.76:6807/4179784307","state":["exists","up"]},{"osd":1,"uuid":"99b709e5-af13-424e-9dc5-3b4c7e078d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6801","nonce":103132661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6803","nonce":103132661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6807","nonce":103132661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":103132661},{"type":"v1","addr":"172.21.15.183:6805","nonce":103132661}]},"public_addr":"172.21.15.183:6801/103132661","cluster_addr":"172.21.15.183:6803/103132661","heartbeat_back_addr":"172.21.15.183:6807/103132661","heartbeat_front_addr":"172.21.15.183:6805/103132661","state":["exists","up"]},{"osd":2,"uuid":"e53f5387-30ec-46d0-bd1e-0950c58b3101","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6801","nonce":4182864338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6803","nonce":4182864338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6807","nonce":4182864338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4182864338},{"type":"v1","addr":"172.21.15.192:6805","nonce":4182864338}]},"public_addr":"172.21.15.192:6801/4182864338","cluster_addr":"172.21.15.192:6803/4182864338","heartbeat_back_addr":"172.21.15.192:6807/4182864338","heartbeat_front_addr":"172.21.15.192:6805/4182864338","state":["exists","up"]},{"osd":3,"uuid":"70f911b5-5902-413d-9e35-98630a33251d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6811","nonce":2500918591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6813","nonce":2500918591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6817","nonce":2500918591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":2500918591},{"type":"v1","addr":"172.21.15.76:6815","nonce":2500918591}]},"public_addr":"172.21.15.76:6811/2500918591","cluster_addr":"172.21.15.76:6813/2500918591","heartbeat_back_addr":"172.21.15.76:6817/2500918591","heartbeat_front_addr":"172.21.15.76:6815/2500918591","state":["exists","up"]},{"osd":4,"uuid":"9c4ca544-320c-4f9f-ba0c-54737df4a1a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6809","nonce":3795301035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6811","nonce":3795301035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6815","nonce":3795301035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3795301035},{"type":"v1","addr":"172.21.15.183:6813","nonce":3795301035}]},"public_addr":"172.21.15.183:6809/3795301035","cluster_addr":"172.21.15.183:6811/3795301035","heartbeat_back_addr":"172.21.15.183:6815/3795301035","heartbeat_front_addr":"172.21.15.183:6813/3795301035","state":["exists","up"]},{"osd":5,"uuid":"ce29342e-6466-4e41-ba1e-2c8f10f4a559","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6809","nonce":2902288003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6811","nonce":2902288003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6815","nonce":2902288003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2902288003},{"type":"v1","addr":"172.21.15.192:6813","nonce":2902288003}]},"public_addr":"172.21.15.192:6809/2902288003","cluster_addr":"172.21.15.192:6811/2902288003","heartbeat_back_addr":"172.21.15.192:6815/2902288003","heartbeat_front_addr":"172.21.15.192:6813/2902288003","state":["exists","up"]},{"osd":6,"uuid":"3260ee11-29d2-493e-9bd1-156fd317542d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6819","nonce":2821822561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6821","nonce":2821822561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6825","nonce":2821822561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2821822561},{"type":"v1","addr":"172.21.15.76:6823","nonce":2821822561}]},"public_addr":"172.21.15.76:6819/2821822561","cluster_addr":"172.21.15.76:6821/2821822561","heartbeat_back_addr":"172.21.15.76:6825/2821822561","heartbeat_front_addr":"172.21.15.76:6823/2821822561","state":["exists","up"]},{"osd":7,"uuid":"d430ca1a-9ec0-43c2-9f12-01ca89b117fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6817","nonce":336323954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6819","nonce":336323954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6823","nonce":336323954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":336323954},{"type":"v1","addr":"172.21.15.183:6821","nonce":336323954}]},"public_addr":"172.21.15.183:6817/336323954","cluster_addr":"172.21.15.183:6819/336323954","heartbeat_back_addr":"172.21.15.183:6823/336323954","heartbeat_front_addr":"172.21.15.183:6821/336323954","state":["exists","up"]},{"osd":8,"uuid":"5b4a7d4f-99a1-4109-a142-427ef07c1333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6817","nonce":2740228504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6819","nonce":2740228504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6823","nonce":2740228504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2740228504},{"type":"v1","addr":"172.21.15.192:6821","nonce":2740228504}]},"public_addr":"172.21.15.192:6817/2740228504","cluster_addr":"172.21.15.192:6819/2740228504","heartbeat_back_addr":"172.21.15.192:6823/2740228504","heartbeat_front_addr":"172.21.15.192:6821/2740228504","state":["exists","up"]},{"osd":9,"uuid":"c93eb38c-a6a9-453f-8b9a-da479e25a337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6827","nonce":1143697448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6829","nonce":1143697448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6833","nonce":1143697448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1143697448},{"type":"v1","addr":"172.21.15.76:6831","nonce":1143697448}]},"public_addr":"172.21.15.76:6827/1143697448","cluster_addr":"172.21.15.76:6829/1143697448","heartbeat_back_addr":"172.21.15.76:6833/1143697448","heartbeat_front_addr":"172.21.15.76:6831/1143697448","state":["exists","up"]},{"osd":10,"uuid":"ef61a179-f812-4a25-8605-cf03e2d8a31c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6825","nonce":596098792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6827","nonce":596098792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6831","nonce":596098792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":596098792},{"type":"v1","addr":"172.21.15.183:6829","nonce":596098792}]},"public_addr":"172.21.15.183:6825/596098792","cluster_addr":"172.21.15.183:6827/596098792","heartbeat_back_addr":"172.21.15.183:6831/596098792","heartbeat_front_addr":"172.21.15.183:6829/596098792","state":["exists","up"]},{"osd":11,"uuid":"ab72b35e-6449-42d0-9a9b-7014e9586173","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6825","nonce":603123871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6827","nonce":603123871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6831","nonce":603123871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":603123871},{"type":"v1","addr":"172.21.15.192:6829","nonce":603123871}]},"public_addr":"172.21.15.192:6825/603123871","cluster_addr":"172.21.15.192:6827/603123871","heartbeat_back_addr":"172.21.15.192:6831/603123871","heartbeat_front_addr":"172.21.15.192:6829/603123871","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:35:48.616810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:12.551382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:36:36.989265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:01.789573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:26.505848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:37:51.399203+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:17.250966+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:38:42.641859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:08.512732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:39:34.778849+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:00.238488+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T15:40:26.299601+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:0/4023925236":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6801/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:6800/2479514030":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/1911361196":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/4249446235":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/3432469273":"2024-03-24T15:43:19.788306+0000","172.21.15.76:0/871302578":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/3336791879":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6801/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/2354876004":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/2171503933":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6800/36977200":"2024-03-24T15:34:15.395471+0000","172.21.15.76:0/4014456486":"2024-03-24T15:33:50.341648+0000","172.21.15.76:0/542720275":"2024-03-24T15:34:15.395471+0000","172.21.15.76:6800/3268079315":"2024-03-24T15:33:50.341648+0000","172.21.15.76:6801/3268079315":"2024-03-24T15:33:50.341648+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6f200a2230 msgr2=0x7f6f200a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6f200a2230 0x7f6f200a46f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f28005ab0 tx=0x7f6f28005fd0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 msgr2=0x7f6f38197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38197d80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9f0 tx=0x7f6f2c00edf0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 shutdown_connections 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f6f200a2230 0x7f6f200a46f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f381076f0 0x7f6f3819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f381020a0 0x7f6f381982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 --2- 172.21.15.76:0/3144694026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f381016e0 0x7f6f38197d80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 >> 172.21.15.76:0/3144694026 conn(0x7f6f38078ba0 msgr2=0x7f6f380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:09.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 shutdown_connections 2024-03-23T15:54:09.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.396+0000 7f6f3e3b5640 1 -- 172.21.15.76:0/3144694026 wait complete. 2024-03-23T15:54:09.411 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T15:54:09.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 -- 172.21.15.76:0/3964330339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac081005c0 msgr2=0x7fac08100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3964330339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac081005c0 0x7fac08100a40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009930 tx=0x7fabf002bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:09.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 -- 172.21.15.76:0/3964330339 shutdown_connections 2024-03-23T15:54:09.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3964330339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac08100f80 0x7fac0810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3964330339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac081005c0 0x7fac08100a40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3964330339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08106660 0x7fac08106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 -- 172.21.15.76:0/3964330339 >> 172.21.15.76:0/3964330339 conn(0x7fac08076990 msgr2=0x7fac08076da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 -- 172.21.15.76:0/3964330339 shutdown_connections 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.511+0000 7fac0cc07640 1 -- 172.21.15.76:0/3964330339 wait complete. 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 Processor -- start 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 -- start start 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac081005c0 0x7fac08198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08100f80 0x7fac081990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08106660 0x7fac081a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac08199900 con 0x7fac08106660 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac081a4120 con 0x7fac08100f80 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0cc07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac081a4260 con 0x7fac081005c0 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08106660 0x7fac081a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08106660 0x7fac081a3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43346/0 (socket says 172.21.15.76:43346) 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac0659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac081005c0 0x7fac08198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 -- 172.21.15.76:0/3813891783 learned_addr learned my addr 172.21.15.76:0/3813891783 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fabfffff640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08100f80 0x7fac081990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 -- 172.21.15.76:0/3813891783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac081005c0 msgr2=0x7fac08198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac081005c0 0x7fac08198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 -- 172.21.15.76:0/3813891783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08100f80 msgr2=0x7fac081990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08100f80 0x7fac081990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.512+0000 7fac06d9d640 1 -- 172.21.15.76:0/3813891783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf0009560 con 0x7fac08106660 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.513+0000 7fac06d9d640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08106660 0x7fac081a3be0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fac0000eb10 tx=0x7fac0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.513+0000 7fabff7fe640 1 -- 172.21.15.76:0/3813891783 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00004b20 con 0x7fac08106660 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.513+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac081a44c0 con 0x7fac08106660 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.513+0000 7fabff7fe640 1 -- 172.21.15.76:0/3813891783 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac00004c80 con 0x7fac08106660 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.513+0000 7fabff7fe640 1 -- 172.21.15.76:0/3813891783 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00010670 con 0x7fac08106660 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.513+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a49f0 con 0x7fac08106660 2024-03-23T15:54:09.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.514+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac08101d00 con 0x7fac08106660 2024-03-23T15:54:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.515+0000 7fabff7fe640 1 -- 172.21.15.76:0/3813891783 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fac00005020 con 0x7fac08106660 2024-03-23T15:54:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.515+0000 7fabff7fe640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fabf40a2230 0x7fabf40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.515+0000 7fac0659c640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fabf40a2230 0x7fabf40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.515+0000 7fabff7fe640 1 -- 172.21.15.76:0/3813891783 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8615+0+0 (secure 0 0 0) 0x7fac00014070 con 0x7fac08106660 2024-03-23T15:54:09.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.516+0000 7fac0659c640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fabf40a2230 0x7fabf40a46f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac08101700 tx=0x7fabf8005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:09.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.520+0000 7fabff7fe640 1 -- 172.21.15.76:0/3813891783 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fac00097280 con 0x7fac08106660 2024-03-23T15:54:09.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.733+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fac08110760 con 0x7fac08106660 2024-03-23T15:54:09.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.735+0000 7fabff7fe640 1 -- 172.21.15.76:0/3813891783 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7fac0009a940 con 0x7fac08106660 2024-03-23T15:54:09.738 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fabf40a2230 msgr2=0x7fabf40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fabf40a2230 0x7fabf40a46f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac08101700 tx=0x7fabf8005f90 comp rx=0 tx=0).stop 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08106660 msgr2=0x7fac081a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08106660 0x7fac081a3be0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fac0000eb10 tx=0x7fac0000c2d0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 shutdown_connections 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fabf40a2230 0x7fabf40a46f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08106660 0x7fac081a3be0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08100f80 0x7fac081990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 --2- 172.21.15.76:0/3813891783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac081005c0 0x7fac08198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 >> 172.21.15.76:0/3813891783 conn(0x7fac08076990 msgr2=0x7fac080fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 shutdown_connections 2024-03-23T15:54:09.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.737+0000 7fac0cc07640 1 -- 172.21.15.76:0/3813891783 wait complete. 2024-03-23T15:54:09.753 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T15:54:09.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 -- 172.21.15.76:0/1062336673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a81005c0 msgr2=0x7f10a8100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 --2- 172.21.15.76:0/1062336673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a81005c0 0x7f10a8100a40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009930 tx=0x7f10a402bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:09.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 -- 172.21.15.76:0/1062336673 shutdown_connections 2024-03-23T15:54:09.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 --2- 172.21.15.76:0/1062336673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a8100f80 0x7f10a810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 --2- 172.21.15.76:0/1062336673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a81005c0 0x7f10a8100a40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 --2- 172.21.15.76:0/1062336673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8106660 0x7f10a8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 -- 172.21.15.76:0/1062336673 >> 172.21.15.76:0/1062336673 conn(0x7f10a8076990 msgr2=0x7f10a8076da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 -- 172.21.15.76:0/1062336673 shutdown_connections 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.856+0000 7f10af363640 1 -- 172.21.15.76:0/1062336673 wait complete. 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 Processor -- start 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 -- start start 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a81005c0 0x7f10a8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a8100f80 0x7f10a8199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8106660 0x7f10a81a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81998b0 con 0x7f10a8100f80 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81a40b0 con 0x7f10a8106660 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10af363640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10a81a41f0 con 0x7f10a81005c0 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a8100f80 0x7f10a8199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a8100f80 0x7f10a8199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43354/0 (socket says 172.21.15.76:43354) 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 -- 172.21.15.76:0/3276440366 learned_addr learned my addr 172.21.15.76:0/3276440366 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ad900640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8106660 0x7f10a81a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ad0ff640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a81005c0 0x7f10a8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 -- 172.21.15.76:0/3276440366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a81005c0 msgr2=0x7f10a8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a81005c0 0x7f10a8198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 -- 172.21.15.76:0/3276440366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8106660 msgr2=0x7f10a81a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8106660 0x7f10a81a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 -- 172.21.15.76:0/3276440366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a4009560 con 0x7f10a8100f80 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.857+0000 7f10ac8fe640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a8100f80 0x7f10a8199030 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009430 tx=0x7f10a402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.858+0000 7f109e7fc640 1 -- 172.21.15.76:0/3276440366 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4004350 con 0x7f10a8100f80 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.858+0000 7f109e7fc640 1 -- 172.21.15.76:0/3276440366 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a40044b0 con 0x7f10a8100f80 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.858+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a81a43f0 con 0x7f10a8100f80 2024-03-23T15:54:09.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.858+0000 7f109e7fc640 1 -- 172.21.15.76:0/3276440366 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4003da0 con 0x7f10a8100f80 2024-03-23T15:54:09.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.858+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a81a48f0 con 0x7f10a8100f80 2024-03-23T15:54:09.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.859+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a8101d00 con 0x7f10a8100f80 2024-03-23T15:54:09.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.860+0000 7f109e7fc640 1 -- 172.21.15.76:0/3276440366 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f10a4004df0 con 0x7f10a8100f80 2024-03-23T15:54:09.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.860+0000 7f109e7fc640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f10940a22b0 0x7f10940a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:09.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.860+0000 7f10ad0ff640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f10940a22b0 0x7f10940a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:09.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.860+0000 7f109e7fc640 1 -- 172.21.15.76:0/3276440366 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8829+0+0 (secure 0 0 0) 0x7f10a40f0910 con 0x7f10a8100f80 2024-03-23T15:54:09.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.861+0000 7f10ad0ff640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f10940a22b0 0x7f10940a4770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10a8101700 tx=0x7f1098005f90 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:09.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:09.864+0000 7f109e7fc640 1 -- 172.21.15.76:0/3276440366 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f10a40b9580 con 0x7f10a8100f80 2024-03-23T15:54:10.070 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c[67641]: ignoring --setuser ceph since I am not root 2024-03-23T15:54:10.070 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c[67641]: ignoring --setgroup ceph since I am not root 2024-03-23T15:54:10.070 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mds[67645]: -- 172.21.15.192:0/2353897759 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8918d01a0 con 0x55d8918a6800 2024-03-23T15:54:10.070 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c[67641]: starting mds.c at 2024-03-23T15:54:10.071 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f[67316]: ignoring --setuser ceph since I am not root 2024-03-23T15:54:10.071 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f[67316]: ignoring --setgroup ceph since I am not root 2024-03-23T15:54:10.071 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mds[67320]: -- 172.21.15.192:0/4090330335 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a7e80c1a0 con 0x564a7e7e2c00 2024-03-23T15:54:10.071 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f[67316]: starting mds.f at 2024-03-23T15:54:10.071 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: ignoring --setuser ceph since I am not root 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: ignoring --setgroup ceph since I am not root 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mds[67949]: -- 172.21.15.192:0/2141443726 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561736081a0 con 0x5561735de800 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: starting mds.i at 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: osdmap e97: 12 total, 12 up, 12 in 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/1463613065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3144694026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3813891783' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T15:54:10.072 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T15:54:10.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T15:54:10.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-03-23T15:54:10.073 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:09 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3813891783' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:54:09.734+0000 7fa1f670f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: osdmap e97: 12 total, 12 up, 12 in 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/1463613065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3144694026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3813891783' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-03-23T15:54:10.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:09 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3813891783' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T15:54:10.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.078+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f10a8068950 con 0x7f10a8100f80 2024-03-23T15:54:10.134 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: ignoring --setuser ceph since I am not root 2024-03-23T15:54:10.134 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: ignoring --setgroup ceph since I am not root 2024-03-23T15:54:10.134 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mds[67011]: -- 172.21.15.183:0/1211110135 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee3a4ac1a0 con 0x55ee3a482800 2024-03-23T15:54:10.134 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: starting mds.b at 2024-03-23T15:54:10.135 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: ignoring --setuser ceph since I am not root 2024-03-23T15:54:10.135 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: ignoring --setgroup ceph since I am not root 2024-03-23T15:54:10.135 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mds[66702]: -- 172.21.15.183:0/177426084 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bba4c6e1a0 con 0x55bba4c45000 2024-03-23T15:54:10.135 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: starting mds.h at 2024-03-23T15:54:10.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: osdmap e97: 12 total, 12 up, 12 in 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/1463613065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3144694026' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3813891783' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3813891783' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-23T15:54:10.136 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e[66393]: ignoring --setuser ceph since I am not root 2024-03-23T15:54:10.137 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e[66393]: ignoring --setgroup ceph since I am not root 2024-03-23T15:54:10.137 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-mds[66397]: -- 172.21.15.183:0/1035998644 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559269eb81a0 con 0x559269e8e800 2024-03-23T15:54:10.137 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:54:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e[66393]: starting mds.e at 2024-03-23T15:54:10.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.748+0000 7f109e7fc640 1 -- 172.21.15.76:0/3276440366 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f10a40bcc40 con 0x7f10a8100f80 2024-03-23T15:54:10.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f10940a22b0 msgr2=0x7f10940a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:10.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f10940a22b0 0x7f10940a4770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10a8101700 tx=0x7f1098005f90 comp rx=0 tx=0).stop 2024-03-23T15:54:10.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a8100f80 msgr2=0x7f10a8199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:10.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a8100f80 0x7f10a8199030 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009430 tx=0x7f10a402c6c0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 shutdown_connections 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f10940a22b0 0x7f10940a4770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8106660 0x7f10a81a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10a8100f80 0x7f10a8199030 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 --2- 172.21.15.76:0/3276440366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a81005c0 0x7f10a8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 >> 172.21.15.76:0/3276440366 conn(0x7f10a8076990 msgr2=0x7f10a80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 shutdown_connections 2024-03-23T15:54:10.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.752+0000 7f10af363640 1 -- 172.21.15.76:0/3276440366 wait complete. 2024-03-23T15:54:10.770 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T15:54:10.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.881+0000 7fafbf815640 1 -- 172.21.15.76:0/2694548375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb8106670 msgr2=0x7fafb8106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:10.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.881+0000 7fafbf815640 1 --2- 172.21.15.76:0/2694548375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb8106670 0x7fafb8106a50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009900 tx=0x7fafb402bc30 comp rx=0 tx=0).stop 2024-03-23T15:54:10.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.881+0000 7fafbf815640 1 -- 172.21.15.76:0/2694548375 shutdown_connections 2024-03-23T15:54:10.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.881+0000 7fafbf815640 1 --2- 172.21.15.76:0/2694548375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8100f70 0x7fafb810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.881+0000 7fafbf815640 1 --2- 172.21.15.76:0/2694548375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb81005b0 0x7fafb8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.881+0000 7fafbf815640 1 --2- 172.21.15.76:0/2694548375 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb8106670 0x7fafb8106a50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.881+0000 7fafbf815640 1 -- 172.21.15.76:0/2694548375 >> 172.21.15.76:0/2694548375 conn(0x7fafb80777c0 msgr2=0x7fafb8077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 -- 172.21.15.76:0/2694548375 shutdown_connections 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 -- 172.21.15.76:0/2694548375 wait complete. 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 Processor -- start 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 -- start start 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb81005b0 0x7fafb8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8100f70 0x7fafb8198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8106670 0x7fafb81a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81997c0 con 0x7fafb81005b0 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8199930 con 0x7fafb8100f70 2024-03-23T15:54:10.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.882+0000 7fafbf815640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafb81a40a0 con 0x7fafb8106670 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb81005b0 0x7fafb8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb81005b0 0x7fafb8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43380/0 (socket says 172.21.15.76:43380) 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 -- 172.21.15.76:0/36798516 learned_addr learned my addr 172.21.15.76:0/36798516 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbcdb0640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8100f70 0x7fafb8198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbddb2640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8106670 0x7fafb81a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 -- 172.21.15.76:0/36798516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8106670 msgr2=0x7fafb81a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8106670 0x7fafb81a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 -- 172.21.15.76:0/36798516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8100f70 msgr2=0x7fafb8198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8100f70 0x7fafb8198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 -- 172.21.15.76:0/36798516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb4009560 con 0x7fafb81005b0 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbd5b1640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb81005b0 0x7fafb8198a90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005940 tx=0x7fafb4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:10.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafae7fc640 1 -- 172.21.15.76:0/36798516 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004440 con 0x7fafb81005b0 2024-03-23T15:54:10.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafae7fc640 1 -- 172.21.15.76:0/36798516 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb40052d0 con 0x7fafb81005b0 2024-03-23T15:54:10.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafae7fc640 1 -- 172.21.15.76:0/36798516 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4003da0 con 0x7fafb81005b0 2024-03-23T15:54:10.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a4320 con 0x7fafb81005b0 2024-03-23T15:54:10.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.883+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a4880 con 0x7fafb81005b0 2024-03-23T15:54:10.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.884+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb8068500 con 0x7fafb81005b0 2024-03-23T15:54:10.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.886+0000 7fafae7fc640 1 -- 172.21.15.76:0/36798516 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7fafb4005440 con 0x7fafb81005b0 2024-03-23T15:54:10.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.887+0000 7fafae7fc640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fafa40a22b0 0x7fafa40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:10.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.887+0000 7fafbcdb0640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fafa40a22b0 0x7fafa40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:10.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.887+0000 7fafae7fc640 1 -- 172.21.15.76:0/36798516 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8829+0+0 (secure 0 0 0) 0x7fafb40f07a0 con 0x7fafb81005b0 2024-03-23T15:54:10.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.887+0000 7fafbcdb0640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fafa40a22b0 0x7fafa40a4770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafb8069370 tx=0x7fafa800b390 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:10.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:10.890+0000 7fafae7fc640 1 -- 172.21.15.76:0/36798516 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7fafb40b9410 con 0x7fafb81005b0 2024-03-23T15:54:11.044 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:10 smithi076 ceph-mon[32176]: osdmap e98: 12 total, 12 up, 12 in 2024-03-23T15:54:11.044 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:10 smithi076 ceph-mon[32176]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-23T15:54:11.044 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:10 smithi076 ceph-mon[32176]: pgmap v338: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-03-23T15:54:11.044 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:10 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3276440366' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:54:11.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:10 smithi192 ceph-mon[36653]: osdmap e98: 12 total, 12 up, 12 in 2024-03-23T15:54:11.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:10 smithi192 ceph-mon[36653]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-23T15:54:11.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:10 smithi192 ceph-mon[36653]: pgmap v338: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-03-23T15:54:11.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:10 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3276440366' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:54:11.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.070+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fafb8108a60 con 0x7fafb81005b0 2024-03-23T15:54:11.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:10 smithi183 ceph-mon[36296]: osdmap e98: 12 total, 12 up, 12 in 2024-03-23T15:54:11.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:10 smithi183 ceph-mon[36296]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-23T15:54:11.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:10 smithi183 ceph-mon[36296]: pgmap v338: 97 pgs: 97 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-03-23T15:54:11.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:10 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3276440366' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T15:54:11.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.750+0000 7fafae7fc640 1 -- 172.21.15.76:0/36798516 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7fafb40430d0 con 0x7fafb81005b0 2024-03-23T15:54:11.754 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-03-23T15:54:11.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fafa40a22b0 msgr2=0x7fafa40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:11.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fafa40a22b0 0x7fafa40a4770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafb8069370 tx=0x7fafa800b390 comp rx=0 tx=0).stop 2024-03-23T15:54:11.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb81005b0 msgr2=0x7fafb8198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:11.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb81005b0 0x7fafb8198a90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005940 tx=0x7fafb4004040 comp rx=0 tx=0).stop 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 shutdown_connections 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7fafa40a22b0 0x7fafa40a4770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb8106670 0x7fafb81a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafb8100f70 0x7fafb8198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 --2- 172.21.15.76:0/36798516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafb81005b0 0x7fafb8198a90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.753+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 >> 172.21.15.76:0/36798516 conn(0x7fafb80777c0 msgr2=0x7fafb80ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.754+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 shutdown_connections 2024-03-23T15:54:11.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.754+0000 7fafbf815640 1 -- 172.21.15.76:0/36798516 wait complete. 2024-03-23T15:54:11.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T15:54:11.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 -- 172.21.15.76:0/1592134223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848100f60 msgr2=0x7f484810d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:11.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 --2- 172.21.15.76:0/1592134223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848100f60 0x7f484810d420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4840009ca0 tx=0x7f484002c0f0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 -- 172.21.15.76:0/1592134223 shutdown_connections 2024-03-23T15:54:11.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 --2- 172.21.15.76:0/1592134223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848100f60 0x7f484810d420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 --2- 172.21.15.76:0/1592134223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48481005a0 0x7f4848100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 --2- 172.21.15.76:0/1592134223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4848106660 0x7f4848106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 -- 172.21.15.76:0/1592134223 >> 172.21.15.76:0/1592134223 conn(0x7f4848077710 msgr2=0x7f4848077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 -- 172.21.15.76:0/1592134223 shutdown_connections 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.878+0000 7f485058a640 1 -- 172.21.15.76:0/1592134223 wait complete. 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 Processor -- start 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 -- start start 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48481005a0 0x7f4848198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4848100f60 0x7f4848198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848106660 0x7f48481a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4848199770 con 0x7f4848106660 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48481998e0 con 0x7f48481005a0 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f485058a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f48481a4050 con 0x7f4848100f60 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f484eb27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848106660 0x7f48481a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f484eb27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848106660 0x7f48481a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43392/0 (socket says 172.21.15.76:43392) 2024-03-23T15:54:11.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.879+0000 7f484eb27640 1 -- 172.21.15.76:0/3404989973 learned_addr learned my addr 172.21.15.76:0/3404989973 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484db25640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4848100f60 0x7f4848198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484e326640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48481005a0 0x7f4848198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484eb27640 1 -- 172.21.15.76:0/3404989973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4848100f60 msgr2=0x7f4848198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484eb27640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4848100f60 0x7f4848198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484eb27640 1 -- 172.21.15.76:0/3404989973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48481005a0 msgr2=0x7f4848198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484eb27640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48481005a0 0x7f4848198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484eb27640 1 -- 172.21.15.76:0/3404989973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4840009900 con 0x7f4848106660 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484e326640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48481005a0 0x7f4848198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f484eb27640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848106660 0x7f48481a3ac0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f484002c700 tx=0x7f4840035d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f483f7fe640 1 -- 172.21.15.76:0/3404989973 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840032ab0 con 0x7f4848106660 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48481a42d0 con 0x7f4848106660 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f483f7fe640 1 -- 172.21.15.76:0/3404989973 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4840032ea0 con 0x7f4848106660 2024-03-23T15:54:11.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f483f7fe640 1 -- 172.21.15.76:0/3404989973 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840004880 con 0x7f4848106660 2024-03-23T15:54:11.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.880+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48481a47d0 con 0x7f4848106660 2024-03-23T15:54:11.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.881+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4848068500 con 0x7f4848106660 2024-03-23T15:54:11.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.882+0000 7f483f7fe640 1 -- 172.21.15.76:0/3404989973 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) ==== 136887+0+0 (secure 0 0 0) 0x7f484003b040 con 0x7f4848106660 2024-03-23T15:54:11.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.883+0000 7f483f7fe640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f48340a2230 0x7f48340a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T15:54:11.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.883+0000 7f484e326640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f48340a2230 0x7f48340a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T15:54:11.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.883+0000 7f483f7fe640 1 -- 172.21.15.76:0/3404989973 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8275+0+0 (secure 0 0 0) 0x7f48400f0af0 con 0x7f4848106660 2024-03-23T15:54:11.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.883+0000 7f484e326640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f48340a2230 0x7f48340a46f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f483800baa0 tx=0x7f4838005d20 comp rx=0 tx=0).ready entity=mgr.15306 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T15:54:11.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:11.887+0000 7f483f7fe640 1 -- 172.21.15.76:0/3404989973 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+184932 (secure 0 0 0) 0x7f48400b9980 con 0x7f4848106660 2024-03-23T15:54:12.014 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T15:54:12.014 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T15:54:12.014 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: Cluster is now healthy 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3276440366' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6837/4079908052,v1:172.21.15.183:6839/4079908052] up:boot 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6834/3563975876,v1:172.21.15.183:6835/3563975876] up:boot 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6838/2182965810,v1:172.21.15.192:6839/2182965810] up:boot 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6834/28197582,v1:172.21.15.192:6835/28197582] up:boot 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.183:6836/2107056336,v1:172.21.15.183:6838/2107056336] up:boot 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: mds.? [v2:172.21.15.192:6836/800057795,v1:172.21.15.192:6837/800057795] up:boot 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: fsmap 12 up:standby 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:54:12.015 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:54:12.016 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:11 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/36798516' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T15:54:12.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T15:54:12.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T15:54:12.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: Cluster is now healthy 2024-03-23T15:54:12.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3276440366' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T15:54:12.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6837/4079908052,v1:172.21.15.183:6839/4079908052] up:boot 2024-03-23T15:54:12.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6834/3563975876,v1:172.21.15.183:6835/3563975876] up:boot 2024-03-23T15:54:12.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6838/2182965810,v1:172.21.15.192:6839/2182965810] up:boot 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6834/28197582,v1:172.21.15.192:6835/28197582] up:boot 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.183:6836/2107056336,v1:172.21.15.183:6838/2107056336] up:boot 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: mds.? [v2:172.21.15.192:6836/800057795,v1:172.21.15.192:6837/800057795] up:boot 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: fsmap 12 up:standby 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:54:12.071 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:11 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/36798516' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T15:54:12.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.083+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4848108a50 con 0x7f4848106660 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: Cluster is now healthy 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3276440366' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6837/4079908052,v1:172.21.15.183:6839/4079908052] up:boot 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6834/3563975876,v1:172.21.15.183:6835/3563975876] up:boot 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6838/2182965810,v1:172.21.15.192:6839/2182965810] up:boot 2024-03-23T15:54:12.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6834/28197582,v1:172.21.15.192:6835/28197582] up:boot 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.183:6836/2107056336,v1:172.21.15.183:6838/2107056336] up:boot 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: mds.? [v2:172.21.15.192:6836/800057795,v1:172.21.15.192:6837/800057795] up:boot 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: fsmap 12 up:standby 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='mgr.15306 172.21.15.76:0/281522392' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-23T15:54:12.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:11 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/36798516' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T15:54:12.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.762+0000 7f483f7fe640 1 -- 172.21.15.76:0/3404989973 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f48400bd040 con 0x7f4848106660 2024-03-23T15:54:12.765 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f48340a2230 msgr2=0x7f48340a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f48340a2230 0x7f48340a46f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f483800baa0 tx=0x7f4838005d20 comp rx=0 tx=0).stop 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848106660 msgr2=0x7f48481a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848106660 0x7f48481a3ac0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f484002c700 tx=0x7f4840035d50 comp rx=0 tx=0).stop 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 shutdown_connections 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:6800/1113739493,v1:172.21.15.76:6801/1113739493] conn(0x7f48340a2230 0x7f48340a46f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4848106660 0x7f48481a3ac0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4848100f60 0x7f4848198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 --2- 172.21.15.76:0/3404989973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48481005a0 0x7f4848198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T15:54:12.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.765+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 >> 172.21.15.76:0/3404989973 conn(0x7f4848077710 msgr2=0x7f48480ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T15:54:12.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.766+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 shutdown_connections 2024-03-23T15:54:12.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-03-23T15:54:12.766+0000 7f485058a640 1 -- 172.21.15.76:0/3404989973 wait complete. 2024-03-23T15:54:12.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:54:12.818 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:54:12.852 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:54:12.886 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-23T15:54:12.887 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T15:54:12.902 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T15:54:12.918 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T15:54:12.943 INFO:tasks.cephadm:Stopping all daemons... 2024-03-23T15:54:12.943 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-23T15:54:12.943 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a 2024-03-23T15:54:13.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:12 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/36798516' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T15:54:13.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:12 smithi192 ceph-mon[36653]: osdmap e99: 12 total, 12 up, 12 in 2024-03-23T15:54:13.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:12 smithi192 ceph-mon[36653]: pgmap v340: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:54:13.070 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:12 smithi192 ceph-mon[36653]: from='client.? 172.21.15.76:0/3404989973' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T15:54:13.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:12 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/36798516' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T15:54:13.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:12 smithi076 ceph-mon[32176]: osdmap e99: 12 total, 12 up, 12 in 2024-03-23T15:54:13.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:12 smithi076 ceph-mon[32176]: pgmap v340: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:54:13.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:12 smithi076 ceph-mon[32176]: from='client.? 172.21.15.76:0/3404989973' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T15:54:13.076 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:12 smithi076 systemd[1]: Stopping Ceph mon.a for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:13.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:12 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/36798516' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-23T15:54:13.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:12 smithi183 ceph-mon[36296]: osdmap e99: 12 total, 12 up, 12 in 2024-03-23T15:54:13.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:12 smithi183 ceph-mon[36296]: pgmap v340: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-23T15:54:13.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:12 smithi183 ceph-mon[36296]: from='client.? 172.21.15.76:0/3404989973' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-23T15:54:13.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:54:13.246+0000 7fa1fbf1a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:13.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a[32172]: 2024-03-23T15:54:13.246+0000 7fa1fbf1a700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-23T15:54:13.576 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:13 smithi076 podman[90610]: 2024-03-23 15:54:13.471528039 +0000 UTC m=+0.348989206 container died c68c8b1c86ce1ee38c431a34ef451ed00d50c6a528970cf74c4d08a5fdc19731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-23T15:54:14.308 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:14 smithi076 podman[90610]: 2024-03-23 15:54:14.038182802 +0000 UTC m=+0.915643980 container cleanup c68c8b1c86ce1ee38c431a34ef451ed00d50c6a528970cf74c4d08a5fdc19731 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-23T15:54:14.309 INFO:journalctl@ceph.mon.a.smithi076.stdout:Mar 23 15:54:14 smithi076 bash[90610]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-a 2024-03-23T15:54:14.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.a.service' 2024-03-23T15:54:14.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:14.477 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-23T15:54:14.478 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-23T15:54:14.478 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.b 2024-03-23T15:54:14.815 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:14 smithi183 systemd[1]: Stopping Ceph mon.b for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:15.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-b[36292]: 2024-03-23T15:54:14.812+0000 7f33fe24c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:15.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-b[36292]: 2024-03-23T15:54:14.812+0000 7f33fe24c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-23T15:54:15.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:14 smithi183 podman[72446]: 2024-03-23 15:54:14.976476514 +0000 UTC m=+0.308987601 container died c1f8e62a41f7bbacb84cc672989830bbb84f9f77548926a07a1dfaa85cddbeb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-23T15:54:15.633 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:15 smithi183 podman[72446]: 2024-03-23 15:54:15.363275029 +0000 UTC m=+0.695786120 container cleanup c1f8e62a41f7bbacb84cc672989830bbb84f9f77548926a07a1dfaa85cddbeb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-23T15:54:15.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Mar 23 15:54:15 smithi183 bash[72446]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-b 2024-03-23T15:54:15.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.b.service' 2024-03-23T15:54:15.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:15.835 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-23T15:54:15.835 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-23T15:54:15.835 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.c 2024-03-23T15:54:16.154 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:15 smithi192 systemd[1]: Stopping Ceph mon.c for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:16.470 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c[36649]: 2024-03-23T15:54:16.149+0000 7f1d12477700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:16.470 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c[36649]: 2024-03-23T15:54:16.149+0000 7f1d12477700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-23T15:54:16.470 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:16 smithi192 podman[73628]: 2024-03-23 15:54:16.170171486 +0000 UTC m=+0.144454859 container died a2d3f72e129101dd95796bc9dbf9258aa483919e2c239e888eca591ff339ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-23T15:54:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:16 smithi192 podman[73628]: 2024-03-23 15:54:16.566642365 +0000 UTC m=+0.540925729 container cleanup a2d3f72e129101dd95796bc9dbf9258aa483919e2c239e888eca591ff339ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-23T15:54:16.820 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:16 smithi192 bash[73628]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c 2024-03-23T15:54:17.129 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.c.service' 2024-03-23T15:54:17.154 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:16 smithi192 podman[73641]: 2024-03-23 15:54:16.901781987 +0000 UTC m=+0.727277064 container remove a2d3f72e129101dd95796bc9dbf9258aa483919e2c239e888eca591ff339ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_BRANCH=HEAD) 2024-03-23T15:54:17.154 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:17 smithi192 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-23T15:54:17.154 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:17 smithi192 systemd[1]: Stopped Ceph mon.c for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:54:17.154 INFO:journalctl@ceph.mon.c.smithi192.stdout:Mar 23 15:54:17 smithi192 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mon.c.service: Consumed 10.969s CPU time. 2024-03-23T15:54:17.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:17.179 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-23T15:54:17.179 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-23T15:54:17.179 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.x 2024-03-23T15:54:17.487 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:54:17 smithi076 systemd[1]: Stopping Ceph mgr.x for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:17.826 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:54:17 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x[32411]: 2024-03-23T15:54:17.484+0000 7fc6d1331700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:17.826 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:54:17 smithi076 podman[90731]: 2024-03-23 15:54:17.515418531 +0000 UTC m=+0.143220755 container died 333621a8ab1baeb008f528fd543a8a3b5f91a73245dc4ef50e76d654cc0a2282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-23T15:54:18.237 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:54:17 smithi076 podman[90731]: 2024-03-23 15:54:17.935925734 +0000 UTC m=+0.563727957 container cleanup 333621a8ab1baeb008f528fd543a8a3b5f91a73245dc4ef50e76d654cc0a2282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-23T15:54:18.237 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:54:17 smithi076 bash[90731]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x 2024-03-23T15:54:18.489 INFO:journalctl@ceph.mgr.x.smithi076.stdout:Mar 23 15:54:18 smithi076 podman[90745]: 2024-03-23 15:54:18.234653016 +0000 UTC m=+0.714525393 container remove 333621a8ab1baeb008f528fd543a8a3b5f91a73245dc4ef50e76d654cc0a2282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:21b3bd672163092b4e542f772797fcba02d3d4de, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:54:18.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-23T15:54:18.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:18.545 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-23T15:54:18.545 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-23T15:54:18.545 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.y 2024-03-23T15:54:18.884 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:54:18 smithi183 systemd[1]: Stopping Ceph mgr.y for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:19.313 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:54:19 smithi183 podman[72567]: 2024-03-23 15:54:19.022553737 +0000 UTC m=+0.287080110 container died 31d95d411636bb14be63dd9bf412ee5cb2888401fb53d636e97c9d41cc93bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-23T15:54:19.634 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:54:19 smithi183 podman[72567]: 2024-03-23 15:54:19.418495716 +0000 UTC m=+0.683022088 container cleanup 31d95d411636bb14be63dd9bf412ee5cb2888401fb53d636e97c9d41cc93bfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-23T15:54:19.634 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Mar 23 15:54:19 smithi183 bash[72567]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-y 2024-03-23T15:54:19.890 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-23T15:54:19.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:19.927 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-23T15:54:19.927 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-23T15:54:19.927 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.z 2024-03-23T15:54:20.252 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:54:19 smithi192 systemd[1]: Stopping Ceph mgr.z for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:20.566 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:54:20 smithi192 podman[73751]: 2024-03-23 15:54:20.250269567 +0000 UTC m=+0.139508847 container died 5d993f699ea106b60f07b59ffeda4ba17734c817f87d9fdab101b6d63e22e6a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:54:20.820 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:54:20 smithi192 podman[73751]: 2024-03-23 15:54:20.672182134 +0000 UTC m=+0.561421411 container cleanup 5d993f699ea106b60f07b59ffeda4ba17734c817f87d9fdab101b6d63e22e6a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-23T15:54:20.820 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Mar 23 15:54:20 smithi192 bash[73751]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mgr-z 2024-03-23T15:54:21.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-23T15:54:21.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:21.182 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-23T15:54:21.182 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-23T15:54:21.182 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.0 2024-03-23T15:54:21.467 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:21 smithi076 systemd[1]: Stopping Ceph osd.0 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:21.826 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0[44534]: 2024-03-23T15:54:21.499+0000 7fee06a53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:21.826 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0[44534]: 2024-03-23T15:54:21.499+0000 7fee06a53700 -1 osd.0 100 *** Got signal Terminated *** 2024-03-23T15:54:21.826 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0[44534]: 2024-03-23T15:54:21.499+0000 7fee06a53700 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:54:27.294 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:27 smithi076 podman[90852]: 2024-03-23 15:54:27.009662462 +0000 UTC m=+5.637490773 container died 649067e388d810c29c1f9c5c0a84c03ddf44da321c0a528324b45f4d95d44d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-23T15:54:27.576 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:27 smithi076 podman[90852]: 2024-03-23 15:54:27.374904183 +0000 UTC m=+6.002732489 container cleanup 649067e388d810c29c1f9c5c0a84c03ddf44da321c0a528324b45f4d95d44d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-23T15:54:27.576 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:27 smithi076 bash[90852]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0 2024-03-23T15:54:27.995 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:27 smithi076 podman[90870]: 2024-03-23 15:54:27.668738753 +0000 UTC m=+0.657706648 container remove 649067e388d810c29c1f9c5c0a84c03ddf44da321c0a528324b45f4d95d44d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:54:28.326 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:27 smithi076 podman[90961]: 2024-03-23 15:54:27.895610504 +0000 UTC m=+0.017936073 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:54:28.326 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:28 smithi076 podman[90961]: 2024-03-23 15:54:28.028116139 +0000 UTC m=+0.150441682 container create 64f76661ed456fbac1ca2b433d9a9fae41208f94cc948e562e4d18f54ea01247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-23T15:54:28.623 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:28 smithi076 podman[90961]: 2024-03-23 15:54:28.325842993 +0000 UTC m=+0.448168550 container init 64f76661ed456fbac1ca2b433d9a9fae41208f94cc948e562e4d18f54ea01247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-03-23T15:54:28.623 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:28 smithi076 podman[90961]: 2024-03-23 15:54:28.328881139 +0000 UTC m=+0.451206689 container start 64f76661ed456fbac1ca2b433d9a9fae41208f94cc948e562e4d18f54ea01247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-23T15:54:28.623 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:28 smithi076 podman[90961]: 2024-03-23 15:54:28.422783899 +0000 UTC m=+0.545109469 container attach 64f76661ed456fbac1ca2b433d9a9fae41208f94cc948e562e4d18f54ea01247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True) 2024-03-23T15:54:28.886 INFO:journalctl@ceph.osd.0.smithi076.stdout:Mar 23 15:54:28 smithi076 podman[90961]: 2024-03-23 15:54:28.621509455 +0000 UTC m=+0.743834991 container died 64f76661ed456fbac1ca2b433d9a9fae41208f94cc948e562e4d18f54ea01247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-23T15:54:29.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.0.service' 2024-03-23T15:54:29.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:29.262 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-23T15:54:29.262 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-23T15:54:29.266 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.1 2024-03-23T15:54:29.617 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:29 smithi183 systemd[1]: Stopping Ceph osd.1 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:29.884 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1[40908]: 2024-03-23T15:54:29.614+0000 7f55b8465700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:29.884 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1[40908]: 2024-03-23T15:54:29.614+0000 7f55b8465700 -1 osd.1 100 *** Got signal Terminated *** 2024-03-23T15:54:29.884 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1[40908]: 2024-03-23T15:54:29.614+0000 7f55b8465700 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:54:35.384 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:35 smithi183 podman[72690]: 2024-03-23 15:54:35.117900949 +0000 UTC m=+5.662191958 container died 3a58df9a51ad7f9e5ae78ed8a90a548eba80987381242692f8d7dc3a47c43e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-03-23T15:54:35.776 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:35 smithi183 podman[72690]: 2024-03-23 15:54:35.51405095 +0000 UTC m=+6.058341952 container cleanup 3a58df9a51ad7f9e5ae78ed8a90a548eba80987381242692f8d7dc3a47c43e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-23T15:54:35.776 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:35 smithi183 bash[72690]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1 2024-03-23T15:54:36.129 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:35 smithi183 podman[72718]: 2024-03-23 15:54:35.773753723 +0000 UTC m=+0.653520889 container remove 3a58df9a51ad7f9e5ae78ed8a90a548eba80987381242692f8d7dc3a47c43e28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:54:36.383 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:36 smithi183 podman[72811]: 2024-03-23 15:54:36.030844015 +0000 UTC m=+0.017569463 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:54:36.384 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:36 smithi183 podman[72811]: 2024-03-23 15:54:36.15685565 +0000 UTC m=+0.143581087 container create 54ea1a3f6be50d472b98b5bb63e6fcc26dd4bce2ff9b9185a43a804f5ac8ca23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-23T15:54:36.727 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:36 smithi183 podman[72811]: 2024-03-23 15:54:36.435935539 +0000 UTC m=+0.422660975 container init 54ea1a3f6be50d472b98b5bb63e6fcc26dd4bce2ff9b9185a43a804f5ac8ca23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-23T15:54:36.728 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:36 smithi183 podman[72811]: 2024-03-23 15:54:36.439118968 +0000 UTC m=+0.425844412 container start 54ea1a3f6be50d472b98b5bb63e6fcc26dd4bce2ff9b9185a43a804f5ac8ca23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-23T15:54:36.728 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:36 smithi183 podman[72811]: 2024-03-23 15:54:36.535871337 +0000 UTC m=+0.522596790 container attach 54ea1a3f6be50d472b98b5bb63e6fcc26dd4bce2ff9b9185a43a804f5ac8ca23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-23T15:54:37.033 INFO:journalctl@ceph.osd.1.smithi183.stdout:Mar 23 15:54:36 smithi183 podman[72811]: 2024-03-23 15:54:36.72553625 +0000 UTC m=+0.712261686 container died 54ea1a3f6be50d472b98b5bb63e6fcc26dd4bce2ff9b9185a43a804f5ac8ca23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:54:37.368 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.1.service' 2024-03-23T15:54:37.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:37.412 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-23T15:54:37.413 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-23T15:54:37.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.2 2024-03-23T15:54:37.767 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:37 smithi192 systemd[1]: Stopping Ceph osd.2 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:38.070 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2[41811]: 2024-03-23T15:54:37.763+0000 7f00a4f11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:38.070 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2[41811]: 2024-03-23T15:54:37.763+0000 7f00a4f11700 -1 osd.2 100 *** Got signal Terminated *** 2024-03-23T15:54:38.070 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2[41811]: 2024-03-23T15:54:37.763+0000 7f00a4f11700 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:54:43.570 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:43 smithi192 podman[73892]: 2024-03-23 15:54:43.270425848 +0000 UTC m=+5.668975670 container died 429ad4021fce9de204c228e035fd9bf946f90230debcec8b6072ae49d4997951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:54:43.930 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:43 smithi192 podman[73892]: 2024-03-23 15:54:43.685169024 +0000 UTC m=+6.083718791 container cleanup 429ad4021fce9de204c228e035fd9bf946f90230debcec8b6072ae49d4997951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:54:43.930 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:43 smithi192 bash[73892]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2 2024-03-23T15:54:44.242 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:43 smithi192 podman[73909]: 2024-03-23 15:54:43.928270601 +0000 UTC m=+0.654632230 container remove 429ad4021fce9de204c228e035fd9bf946f90230debcec8b6072ae49d4997951 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-23T15:54:44.509 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:44 smithi192 podman[74001]: 2024-03-23 15:54:44.142821786 +0000 UTC m=+0.017613582 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:54:44.509 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:44 smithi192 podman[74001]: 2024-03-23 15:54:44.245857605 +0000 UTC m=+0.120649415 container create 267556a04fd2971bed65621b14744e4e73cddd7c5297647b030258c12318262d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:54:44.799 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:44 smithi192 podman[74001]: 2024-03-23 15:54:44.506891375 +0000 UTC m=+0.381683171 container init 267556a04fd2971bed65621b14744e4e73cddd7c5297647b030258c12318262d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-03-23T15:54:44.799 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:44 smithi192 podman[74001]: 2024-03-23 15:54:44.510119704 +0000 UTC m=+0.384911498 container start 267556a04fd2971bed65621b14744e4e73cddd7c5297647b030258c12318262d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-23T15:54:44.799 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:44 smithi192 podman[74001]: 2024-03-23 15:54:44.598740325 +0000 UTC m=+0.473532124 container attach 267556a04fd2971bed65621b14744e4e73cddd7c5297647b030258c12318262d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-23T15:54:45.070 INFO:journalctl@ceph.osd.2.smithi192.stdout:Mar 23 15:54:44 smithi192 podman[74001]: 2024-03-23 15:54:44.797292739 +0000 UTC m=+0.672084535 container died 267556a04fd2971bed65621b14744e4e73cddd7c5297647b030258c12318262d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:54:45.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.2.service' 2024-03-23T15:54:45.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:45.536 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-23T15:54:45.536 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-23T15:54:45.536 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.3 2024-03-23T15:54:45.826 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:45 smithi076 systemd[1]: Stopping Ceph osd.3 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:46.326 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3[51477]: 2024-03-23T15:54:45.904+0000 7f2f795d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:46.326 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3[51477]: 2024-03-23T15:54:45.904+0000 7f2f795d5700 -1 osd.3 100 *** Got signal Terminated *** 2024-03-23T15:54:46.326 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:45 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3[51477]: 2024-03-23T15:54:45.904+0000 7f2f795d5700 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:54:49.820 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:49.327+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:50.076 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:49 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3[51477]: 2024-03-23T15:54:49.806+0000 7f2f71bc1700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.003489+0000 front 2024-03-23T15:54:25.003442+0000 (oldest deadline 2024-03-23T15:54:49.703696+0000) 2024-03-23T15:54:50.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:49 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:49.733+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:50.569 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:50.282+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:51.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:50.684+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:51.076 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:50 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3[51477]: 2024-03-23T15:54:50.800+0000 7f2f71bc1700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.003489+0000 front 2024-03-23T15:54:25.003442+0000 (oldest deadline 2024-03-23T15:54:49.703696+0000) 2024-03-23T15:54:51.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:50 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:50.812+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:51.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:50 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:50.753+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:51.569 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:51.248+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:51.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:51.303+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:51.633 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:51 smithi076 podman[91065]: 2024-03-23 15:54:51.343479742 +0000 UTC m=+5.617969012 container died 0b560c074823e679cdc3d84a56f9b23be7d12b8879456e53ab28ed3891e9aa88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True) 2024-03-23T15:54:52.043 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:51 smithi076 podman[91065]: 2024-03-23 15:54:51.730608541 +0000 UTC m=+6.005097803 container cleanup 0b560c074823e679cdc3d84a56f9b23be7d12b8879456e53ab28ed3891e9aa88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-23T15:54:52.043 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:51 smithi076 bash[91065]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3 2024-03-23T15:54:52.043 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:51 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:51.766+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:52.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:51.703+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:52.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:51 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:51.741+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:52.326 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:52 smithi076 podman[91084]: 2024-03-23 15:54:52.040854273 +0000 UTC m=+0.699605823 container remove 0b560c074823e679cdc3d84a56f9b23be7d12b8879456e53ab28ed3891e9aa88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:54:52.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:52.300+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:52.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:52.227+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:52.605 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:52 smithi076 podman[91176]: 2024-03-23 15:54:52.341739501 +0000 UTC m=+0.108968355 container create b2c505ba4f9d95386900a5882149a7e96777d585be941afddc52fab071f48f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:54:52.605 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:52 smithi076 podman[91176]: 2024-03-23 15:54:52.250529822 +0000 UTC m=+0.017758643 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:54:52.605 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:52 smithi076 podman[91176]: 2024-03-23 15:54:52.603045945 +0000 UTC m=+0.370274766 container init b2c505ba4f9d95386900a5882149a7e96777d585be941afddc52fab071f48f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-23T15:54:52.900 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:52 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:52.816+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:52.900 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:52 smithi076 podman[91176]: 2024-03-23 15:54:52.606233025 +0000 UTC m=+0.373461833 container start b2c505ba4f9d95386900a5882149a7e96777d585be941afddc52fab071f48f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-23T15:54:52.900 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:52 smithi076 podman[91176]: 2024-03-23 15:54:52.719721178 +0000 UTC m=+0.486949992 container attach b2c505ba4f9d95386900a5882149a7e96777d585be941afddc52fab071f48f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-23T15:54:53.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:52.682+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:53.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:52 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:52.718+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:53.242 INFO:journalctl@ceph.osd.3.smithi076.stdout:Mar 23 15:54:52 smithi076 podman[91176]: 2024-03-23 15:54:52.8983243 +0000 UTC m=+0.665553109 container died b2c505ba4f9d95386900a5882149a7e96777d585be941afddc52fab071f48f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-23T15:54:53.242 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:52 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:52.921+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:54:53.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:53.312+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:53.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:53.246+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:53.617 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.3.service' 2024-03-23T15:54:53.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:54:53.661 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-23T15:54:53.661 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-23T15:54:53.662 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.4 2024-03-23T15:54:53.977 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:53 smithi183 systemd[1]: Stopping Ceph osd.4 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:54:53.978 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:53 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:53.738+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:54.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:53.700+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:54.326 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:53 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:53.885+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:54:54.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:53 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:53.843+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:54.383 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:53 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4[47459]: 2024-03-23T15:54:53.973+0000 7fd0e69b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:54:54.383 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:53 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4[47459]: 2024-03-23T15:54:53.973+0000 7fd0e69b9700 -1 osd.4 100 *** Got signal Terminated *** 2024-03-23T15:54:54.383 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:53 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4[47459]: 2024-03-23T15:54:53.973+0000 7fd0e69b9700 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:54:54.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:54.267+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:54.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:54.215+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:55.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:54.735+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:55.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:54.835+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:54:55.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:54 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:54.830+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:55.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:54 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:54.693+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:55.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:55 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:55.228+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:55.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:55 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:55.252+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:56.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:55 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:55.696+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:56.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:55 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:55.802+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:54:56.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:55 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:55.832+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:56.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:55 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:55.695+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:56.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:55 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:55.695+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:54:56.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:56 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:56.232+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:56.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:56 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:56.264+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:57.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:56 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:56.716+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:57.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:56 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:56.791+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:54:57.077 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:56 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:56.882+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:57.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:56 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:56.719+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:57.134 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:56 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:56.719+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:54:57.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:57 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:57.262+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:57.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:57 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:57.295+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:58.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:57 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:57.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:58.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:57 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:57.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:54:58.133 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:57 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4[47459]: 2024-03-23T15:54:57.815+0000 7fd0e2fad700 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.946545+0000 front 2024-03-23T15:54:32.946516+0000 (oldest deadline 2024-03-23T15:54:57.646717+0000) 2024-03-23T15:54:58.134 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:57 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:57.753+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:58.134 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:57 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:57.753+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:54:58.326 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:57 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:57.826+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:54:58.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:57 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:57.918+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:58.345 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:58 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:58.266+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:58.346 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:58 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:58.242+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:58.346 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:58 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:58.242+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:54:59.035 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:58 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4[47459]: 2024-03-23T15:54:58.789+0000 7fd0e2fad700 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.946545+0000 front 2024-03-23T15:54:32.946516+0000 (oldest deadline 2024-03-23T15:54:57.646717+0000) 2024-03-23T15:54:59.035 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:58 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:58.753+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:54:59.035 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:58 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:58.753+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:54:59.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:58 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:58.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:54:59.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:58 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:58.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:54:59.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:58 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:58.783+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:54:59.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:58 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:58.783+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:54:59.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:58 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:58.907+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:54:59.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:54:59 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:54:59.265+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:54:59.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:59 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:59.258+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:54:59.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:54:59 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:54:59.258+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:54:59.633 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:59 smithi183 podman[72917]: 2024-03-23 15:54:59.432363356 +0000 UTC m=+5.579408370 container died 93f534254e16df110412c926a8b266e7e8f155505d4685300a4e0df19bc74900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5) 2024-03-23T15:54:59.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:54:59 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:54:59.370+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:00.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:59 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:59.654+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:00.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:54:59 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:54:59.654+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:00.075 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:59 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:59.740+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:00.075 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:54:59 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:54:59.740+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:00.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:54:59 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:54:59.920+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:00.095 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:59 smithi183 podman[72917]: 2024-03-23 15:54:59.858398402 +0000 UTC m=+6.005443415 container cleanup 93f534254e16df110412c926a8b266e7e8f155505d4685300a4e0df19bc74900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-23T15:55:00.095 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:54:59 smithi183 bash[72917]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4 2024-03-23T15:55:00.095 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:59 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:59.784+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:00.095 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:54:59 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:54:59.784+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:00.384 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:55:00 smithi183 podman[72938]: 2024-03-23 15:55:00.093039203 +0000 UTC m=+0.658773571 container remove 93f534254e16df110412c926a8b266e7e8f155505d4685300a4e0df19bc74900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-23T15:55:00.384 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:00 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:00.334+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:00.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:00 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:00.230+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:00.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:00 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:00.305+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:55:00.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:00 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:00.305+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:55:00.790 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:55:00 smithi183 podman[73035]: 2024-03-23 15:55:00.422938891 +0000 UTC m=+0.017670968 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:00.790 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:55:00 smithi183 podman[73035]: 2024-03-23 15:55:00.55230175 +0000 UTC m=+0.147033815 container create fcf8d3c9c3aa3819e3405410b4f173697e7cd91248d68099e568402864899859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-23T15:55:01.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:00 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:00.610+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:01.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:00 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:00.610+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:01.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:00.731+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:01.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:00.731+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:01.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:00.884+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:01.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:00 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:00.884+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:01.085 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:55:00 smithi183 podman[73035]: 2024-03-23 15:55:00.788128074 +0000 UTC m=+0.382860158 container init fcf8d3c9c3aa3819e3405410b4f173697e7cd91248d68099e568402864899859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-03-23T15:55:01.085 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:55:00 smithi183 podman[73035]: 2024-03-23 15:55:00.791255289 +0000 UTC m=+0.385987348 container start fcf8d3c9c3aa3819e3405410b4f173697e7cd91248d68099e568402864899859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-23T15:55:01.086 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:55:00 smithi183 podman[73035]: 2024-03-23 15:55:00.888042842 +0000 UTC m=+0.482774909 container attach fcf8d3c9c3aa3819e3405410b4f173697e7cd91248d68099e568402864899859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:55:01.086 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:00 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:00.797+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:01.086 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:00 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:00.797+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:01.383 INFO:journalctl@ceph.osd.4.smithi183.stdout:Mar 23 15:55:01 smithi183 podman[73035]: 2024-03-23 15:55:01.082866485 +0000 UTC m=+0.677598561 container died fcf8d3c9c3aa3819e3405410b4f173697e7cd91248d68099e568402864899859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True) 2024-03-23T15:55:01.383 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:01 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:01.312+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:01.569 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:01 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:01.196+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:01.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:01 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:01.273+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:55:01.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:01 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:01.273+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:55:01.676 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.4.service' 2024-03-23T15:55:01.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:01.720 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-23T15:55:01.720 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-23T15:55:01.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.5 2024-03-23T15:55:02.025 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:01 smithi192 systemd[1]: Stopping Ceph osd.5 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:02.026 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:01 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:01.599+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:02.026 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:01 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:01.599+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:02.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:01 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:01.764+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:02.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:01 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:01.764+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:02.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:01 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:01.899+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:02.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:01 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:01.899+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:02.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:01 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:01.832+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:02.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:01 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:01.832+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:02.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:02.243+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:02.320 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:02.022+0000 7fbe1ad50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:02.320 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:02.022+0000 7fbe1ad50700 -1 osd.5 100 *** Got signal Terminated *** 2024-03-23T15:55:02.320 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:02.022+0000 7fbe1ad50700 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:55:02.601 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:02.321+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:55:02.602 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:02.321+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:55:02.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:02 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:02.313+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:03.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:02.597+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:03.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:02 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:02.597+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:03.075 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:02 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:02.768+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:03.075 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:02 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:02.768+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:03.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:02 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:02.931+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:03.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:02 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:02.931+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:03.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:02 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:02.865+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:03.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:02 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:02.865+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:03.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:03 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:03.200+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:03.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:03 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:03.282+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:55:03.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:03 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:03.282+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:55:03.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:03 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:03.318+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:04.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:03 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:03.577+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:04.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:03 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:03.577+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:04.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:03 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:03.803+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:04.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:03 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:03.803+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:04.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:03 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:03.981+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:04.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:03 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:03.981+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:04.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:03 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:03.836+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:04.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:03 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:03.836+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:04.568 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:04 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:04.289+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:55:04.568 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:04 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:04.289+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:55:04.568 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:04 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:04.289+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.029635+0000 front 2024-03-23T15:54:38.029654+0000 (oldest deadline 2024-03-23T15:55:03.329746+0000) 2024-03-23T15:55:04.568 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:04 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:04.157+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:04.569 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:04 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:04.157+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:04.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:04 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:04.274+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:04.819 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:04 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:04.565+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:04.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:04 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:04.565+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:05.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:04 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:04.786+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:05.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:04 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:04.786+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:05.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:04 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:04.786+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:05.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:04 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:04.994+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:05.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:04 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:04.994+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:05.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:04 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:04.876+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:05.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:04 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:04.876+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:05.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:05 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:05.135+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:05.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:05 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:05.135+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:05.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:05 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:05.262+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:55:05.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:05 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:05.262+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:55:05.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:05 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:05.262+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.029635+0000 front 2024-03-23T15:54:38.029654+0000 (oldest deadline 2024-03-23T15:55:03.329746+0000) 2024-03-23T15:55:05.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:05 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:05.295+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:05.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:05 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:05.295+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:06.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:05 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:05.601+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:06.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:05 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:05.601+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:06.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:05 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:05.776+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:06.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:05 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:05.776+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:06.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:05 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:05.776+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:06.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:05 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:05.995+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:06.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:05 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:05.995+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:06.270 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:05 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:05.925+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:06.270 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:05 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:05.925+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:06.270 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:05 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:05.925+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:06.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:06.086+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:06.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:06.086+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:06.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:06.269+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:25.327281+0000 front 2024-03-23T15:54:25.327310+0000 (oldest deadline 2024-03-23T15:54:50.627286+0000) 2024-03-23T15:55:06.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:06.269+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:34.529453+0000 front 2024-03-23T15:54:34.529536+0000 (oldest deadline 2024-03-23T15:54:58.029385+0000) 2024-03-23T15:55:06.570 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5[48478]: 2024-03-23T15:55:06.269+0000 7fbe1333c700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.029635+0000 front 2024-03-23T15:54:38.029654+0000 (oldest deadline 2024-03-23T15:55:03.329746+0000) 2024-03-23T15:55:06.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:06 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:06.267+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:06.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:06 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:06.267+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:07.022 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:06 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:06.747+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:07.023 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:06 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:06.747+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:07.023 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:06 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:06.747+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:07.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:06.638+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:07.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:06.638+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:07.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:06 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:06.638+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:07.262 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:06 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:06.901+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:07.263 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:06 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:06.901+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:07.263 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:06 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:06.901+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:07.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:07.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:07.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:07.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:07.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:07.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:07.460 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:07 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:07.076+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:07.460 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:07 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:07.076+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:07.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:07 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:07.260+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:07.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:07 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:07.260+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:07.802 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:07 smithi192 podman[74131]: 2024-03-23 15:55:07.458202753 +0000 UTC m=+5.544973580 container died 522fadbeed6c7cf50acd7916f2141d52f46b0ee1901a8de47f2802393eeba664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-23T15:55:07.803 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:07 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:07.599+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:07.803 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:07 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:07.599+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:07.803 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:07 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:07.599+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:08.070 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:07 smithi192 podman[74131]: 2024-03-23 15:55:07.890581426 +0000 UTC m=+5.977352278 container cleanup 522fadbeed6c7cf50acd7916f2141d52f46b0ee1901a8de47f2802393eeba664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True) 2024-03-23T15:55:08.070 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:07 smithi192 bash[74131]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5 2024-03-23T15:55:08.075 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:07.776+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:08.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:07.776+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:08.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:07.776+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:08.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:07.996+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:08.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:07.996+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:08.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:07 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:07.996+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:08.289 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:07 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:07.934+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:08.289 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:07 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:07.934+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:08.289 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:07 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:07.934+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:08.550 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:08 smithi192 podman[74148]: 2024-03-23 15:55:08.125266721 +0000 UTC m=+0.658367173 container remove 522fadbeed6c7cf50acd7916f2141d52f46b0ee1901a8de47f2802393eeba664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:08.551 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:08 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:08.109+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:08.551 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:08 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:08.109+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:08.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:08 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:08.286+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:08.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:08 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:08.286+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:08.814 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:08 smithi192 podman[74240]: 2024-03-23 15:55:08.451003302 +0000 UTC m=+0.017358556 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:08.815 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:08 smithi192 podman[74240]: 2024-03-23 15:55:08.567748497 +0000 UTC m=+0.134103775 container create 599541abe14f6240457c8ac5ef9f3524c5f2a526259e3993d6bbfb69474f265f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-23T15:55:08.815 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:08 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:08.585+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:08.815 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:08 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:08.585+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:08.815 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:08 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:08.585+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:09.070 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:08 smithi192 podman[74240]: 2024-03-23 15:55:08.812522218 +0000 UTC m=+0.378877453 container init 599541abe14f6240457c8ac5ef9f3524c5f2a526259e3993d6bbfb69474f265f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:55:09.071 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:08 smithi192 podman[74240]: 2024-03-23 15:55:08.815671533 +0000 UTC m=+0.382026781 container start 599541abe14f6240457c8ac5ef9f3524c5f2a526259e3993d6bbfb69474f265f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-23T15:55:09.071 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:08 smithi192 podman[74240]: 2024-03-23 15:55:08.903621567 +0000 UTC m=+0.469976825 container attach 599541abe14f6240457c8ac5ef9f3524c5f2a526259e3993d6bbfb69474f265f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-03-23T15:55:09.071 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:09.060+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:09.071 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:09.060+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:09.318 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:08 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:08.979+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:09.318 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:08 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:08.979+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:09.318 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:08 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:08.979+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:09.326 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:08 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:08.825+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:09.326 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:08 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:08.825+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:09.326 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:08 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:08.825+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:09.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:09.028+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:09.327 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:09.028+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:09.327 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:09.028+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:09.417 INFO:journalctl@ceph.osd.5.smithi192.stdout:Mar 23 15:55:09 smithi192 podman[74240]: 2024-03-23 15:55:09.103965643 +0000 UTC m=+0.670320889 container died 599541abe14f6240457c8ac5ef9f3524c5f2a526259e3993d6bbfb69474f265f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-23T15:55:09.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:09.315+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:09.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:09.315+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:09.735 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:09.601+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:09.735 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:09.601+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:09.735 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:09 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:09.601+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:09.749 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.5.service' 2024-03-23T15:55:09.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:09.789 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-23T15:55:09.789 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-23T15:55:09.789 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.6 2024-03-23T15:55:10.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:09.811+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:10.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:09.811+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:10.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:09.811+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:10.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:09 smithi076 systemd[1]: Stopping Ceph osd.6 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:10.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:09.980+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:10.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:09.980+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:10.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:09 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:09.980+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:10.290 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:09.989+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:10.290 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:09.989+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:10.290 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:09 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:09.989+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:10.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:10 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:10.037+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:10.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:10 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:10.037+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:10.576 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:10.090+0000 7f10c8446700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:10.576 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:10.090+0000 7f10c8446700 -1 osd.6 100 *** Got signal Terminated *** 2024-03-23T15:55:10.576 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:10.090+0000 7f10c8446700 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:55:10.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:10 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:10.287+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:10.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:10 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:10.287+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:11.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:10 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:10.623+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:11.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:10 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:10.623+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:11.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:10 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:10.623+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:11.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:10.775+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:11.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:10.775+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:11.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:10.775+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:11.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:10.974+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:11.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:10.974+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:11.077 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:10 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:10.974+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:11.326 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:10 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:10.973+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:11.326 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:10 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:10.973+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:11.326 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:10 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:10.973+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:11.569 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:11 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:11.071+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:11.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:11 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:11.071+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:11.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:11 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:11.323+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:11.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:11 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:11.323+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:12.009 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:11 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:11.755+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:12.009 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:11 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:11.755+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:12.010 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:11 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:11.755+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:12.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:11 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:11.592+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:12.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:11 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:11.592+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:12.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:11 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:11.593+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:12.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:12.006+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:12.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:12.006+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:12.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:12.006+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:12.355 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:11 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:11.947+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:12.355 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:11 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:11.947+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:12.355 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:11 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:11.947+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:12.569 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:12 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:12.089+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:12.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:12 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:12.089+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:12.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:12.352+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:12.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:12.352+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:12.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:12.352+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:13.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:12 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:12.617+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:13.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:12 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:12.617+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:13.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:12 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:12.617+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:13.075 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:13.010+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:13.075 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:13.010+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:13.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:13.010+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:13.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:12.773+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:13.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:12.773+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:13.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:12 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:12.773+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:13.335 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:12.969+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:13.335 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:12.969+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:13.335 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:12.969+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:13.335 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:12 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:12.969+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:13.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:13.124+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:13.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:13.124+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:13.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:13.124+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:13.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:13.332+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:13.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:13.332+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:13.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:13.332+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:14.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:13.613+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:14.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:13.613+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:14.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:13 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:13.613+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:14.075 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:14.001+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:14.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:14.001+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:14.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:14.001+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:14.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:13.754+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:14.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:13.754+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:14.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:13 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:13.754+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:14.378 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:13.997+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:14.378 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:13.997+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:14.379 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:13.997+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:14.379 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:13 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:13.997+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:14.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:14.075+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:14.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:14.075+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:14.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:14.075+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:14.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:14.376+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:14.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:14.376+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:14.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:14.376+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:15.033 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:14.567+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:15.033 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:14.567+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:15.033 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:14 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:14.567+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:15.075 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:15.010+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:15.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:15.010+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:15.076 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:15.010+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:15.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:14.774+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:22.822586+0000 front 2024-03-23T15:54:22.822873+0000 (oldest deadline 2024-03-23T15:54:52.823109+0000) 2024-03-23T15:55:15.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:14.774+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.823690+0000 front 2024-03-23T15:54:32.823765+0000 (oldest deadline 2024-03-23T15:54:58.123585+0000) 2024-03-23T15:55:15.076 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:14 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6[58061]: 2024-03-23T15:55:14.774+0000 7f10c0a32700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.024679+0000 front 2024-03-23T15:54:41.024699+0000 (oldest deadline 2024-03-23T15:55:03.924597+0000) 2024-03-23T15:55:15.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:15 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:15.030+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:15.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:15 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:15.030+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:15.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:15 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:15.030+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:15.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:14.957+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:15.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:14.957+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:15.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:14.957+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:15.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:14 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:14.957+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:15.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:15 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:15.535+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:15.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:15 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:15.535+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:15.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:15 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:15.535+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:15.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:15.410+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:15.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:15.410+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:15.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:15.410+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:15.949 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:15 smithi076 podman[91295]: 2024-03-23 15:55:15.653230458 +0000 UTC m=+5.673171246 container died f8ca7c6d09b3edeb173a8f033c98f1cb74464e3a6798095d306f01de274ae2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-23T15:55:16.249 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:16 smithi076 podman[91295]: 2024-03-23 15:55:16.052917484 +0000 UTC m=+6.072858258 container cleanup f8ca7c6d09b3edeb173a8f033c98f1cb74464e3a6798095d306f01de274ae2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:16.250 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:16 smithi076 bash[91295]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6 2024-03-23T15:55:16.251 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:15.973+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:16.251 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:15.973+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:16.251 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:15.973+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:16.251 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:15 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:15.973+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:16.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:15.937+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:16.385 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:15.937+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:16.385 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:15.937+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:16.385 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:15.937+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:16.510 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:16.070+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:16.511 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:16.070+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:16.511 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:16.070+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:16.575 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:16 smithi076 podman[91322]: 2024-03-23 15:55:16.246389375 +0000 UTC m=+0.593259144 container remove f8ca7c6d09b3edeb173a8f033c98f1cb74464e3a6798095d306f01de274ae2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-23T15:55:16.819 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:16.507+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:16.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:16.507+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:16.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:16 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:16.507+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:16.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:16.404+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:16.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:16.404+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:16.884 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:16.404+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:16.968 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:16 smithi076 podman[91413]: 2024-03-23 15:55:16.565081 +0000 UTC m=+0.018492650 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:16.969 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:16 smithi076 podman[91413]: 2024-03-23 15:55:16.705968716 +0000 UTC m=+0.159380357 container create 9234136a80717f153900984dd7e597d06dcd0f2b175df44bf743e3d736d5e7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T15:55:16.969 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:16 smithi076 podman[91413]: 2024-03-23 15:55:16.967062108 +0000 UTC m=+0.420473749 container init 9234136a80717f153900984dd7e597d06dcd0f2b175df44bf743e3d736d5e7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True) 2024-03-23T15:55:17.260 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:16 smithi076 podman[91413]: 2024-03-23 15:55:16.970421663 +0000 UTC m=+0.423833308 container start 9234136a80717f153900984dd7e597d06dcd0f2b175df44bf743e3d736d5e7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD) 2024-03-23T15:55:17.260 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:17 smithi076 podman[91413]: 2024-03-23 15:55:17.069603188 +0000 UTC m=+0.523014842 container attach 9234136a80717f153900984dd7e597d06dcd0f2b175df44bf743e3d736d5e7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:17.260 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:16.968+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:17.260 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:16.968+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:17.261 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:16.968+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:17.261 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:16 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:16.968+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:17.319 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:17.036+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:17.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:17.036+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:17.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:17.036+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:17.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:16.950+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:17.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:16.950+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:17.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:16.950+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:17.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:16 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:16.950+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:17.547 INFO:journalctl@ceph.osd.6.smithi076.stdout:Mar 23 15:55:17 smithi076 podman[91413]: 2024-03-23 15:55:17.258701476 +0000 UTC m=+0.712113112 container died 9234136a80717f153900984dd7e597d06dcd0f2b175df44bf743e3d736d5e7a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:55:17.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:17.371+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:17.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:17.371+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:17.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:17.371+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:17.819 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:17.461+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:17.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:17.461+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:17.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:17 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:17.461+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:17.877 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.6.service' 2024-03-23T15:55:17.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:17.918 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-23T15:55:17.918 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-23T15:55:17.918 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.7 2024-03-23T15:55:18.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:17 smithi183 systemd[1]: Stopping Ceph osd.7 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:18.238 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:17.978+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:18.238 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:17.978+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:18.238 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:17.978+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:18.238 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:17 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:17.978+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:18.319 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:18 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:18.011+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:18.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:18 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:18.011+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:18.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:18 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:18.011+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:18.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:18.000+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:18.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:18.000+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:18.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:18.000+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:18.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:18 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:18.000+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:18.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:18.235+0000 7fad67d75700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:18.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:18.235+0000 7fad67d75700 -1 osd.7 100 *** Got signal Terminated *** 2024-03-23T15:55:18.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:18.235+0000 7fad67d75700 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:55:18.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:18.405+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:18.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:18.405+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:18.634 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:18.405+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:18.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:18 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:18.479+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:18.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:18 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:18.479+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:18.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:18 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:18.479+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:19.319 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:19.060+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:19.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:19.060+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:19.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:19.060+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:19.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:19 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:19.033+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:19.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:19 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:19.033+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:19.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:19 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:19.033+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:19.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:19 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:19.033+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:19.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:18.962+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:19.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:18.962+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:19.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:18.962+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:19.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:18.962+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:19.822 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:19.510+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:19.822 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:19.510+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:19.822 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:19 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:19.510+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:19.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:19.403+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:19.884 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:19.403+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:19.884 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:19.403+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:20.319 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:20 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:20.035+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:20.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:20 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:20.035+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:20.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:20 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:20.035+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:20.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:20.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:20.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:20.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:20.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:20.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:20.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:20.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:20.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:20 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:20.019+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:20.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:19.998+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:20.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:19.998+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:20.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:19.998+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:20.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:19 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:19.998+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:20.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:20 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:20.552+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:20.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:20 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:20.552+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:20.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:20 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:20.552+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:20.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:20.390+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:20.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:20.390+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:20.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:20.390+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:21.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:21.059+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:21.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:21.059+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:21.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:21.059+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:21.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:21.059+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:21.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:21 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:21.059+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:21.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:20.987+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:21.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:20.987+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:21.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:20.987+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:21.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:20.987+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:21.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:20 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:20.987+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:21.524 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:21.069+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:21.524 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:21.069+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:21.524 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:21.069+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:21.524 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:21.069+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:21.819 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:21.521+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:21.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:21.521+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:21.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:21.521+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:21.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:21 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:21.521+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:21.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:21 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:21.387+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:21.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:21 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:21.387+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:21.884 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:21 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:21.387+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:21.884 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:21 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:21.387+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:55.738286+0000 front 2024-03-23T15:54:55.738261+0000 (oldest deadline 2024-03-23T15:55:21.038369+0000) 2024-03-23T15:55:22.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:22.026+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:22.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:22.026+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:22.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:22.026+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:22.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:22.026+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:22.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:22 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:22.045+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:22.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:22 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:22.045+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:22.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:22 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:22.045+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:22.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:22 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:22.045+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:22.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:22 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:22.045+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:22.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:22.002+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:22.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:22.002+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:22.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:22.002+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:22.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:22.002+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:22.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:22.002+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:22.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:22.500+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:22.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:22.500+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:22.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:22.500+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:22.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:22 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:22.500+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:22.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:22.430+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:32.935190+0000 front 2024-03-23T15:54:32.935166+0000 (oldest deadline 2024-03-23T15:54:58.835329+0000) 2024-03-23T15:55:22.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:22.430+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.136363+0000 front 2024-03-23T15:54:41.136175+0000 (oldest deadline 2024-03-23T15:55:05.236262+0000) 2024-03-23T15:55:22.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:22.430+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:46.937422+0000 front 2024-03-23T15:54:46.937383+0000 (oldest deadline 2024-03-23T15:55:11.637373+0000) 2024-03-23T15:55:22.883 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7[54310]: 2024-03-23T15:55:22.430+0000 7fad60361700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:55.738286+0000 front 2024-03-23T15:54:55.738261+0000 (oldest deadline 2024-03-23T15:55:21.038369+0000) 2024-03-23T15:55:23.319 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:23.023+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:23.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:23.023+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:23.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:23.023+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:23.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:23.023+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:23.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:23.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:23.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:23.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:23.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:23.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:23.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:23.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:23.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:23 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:23.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:23.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:23 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:23.030+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:23.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:23 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:23.030+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:23.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:23 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:23.030+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:23.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:23 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:23.030+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:23.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:23 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:23.030+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:23.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:23.513+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:23.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:23.513+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:23.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:23.513+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:23.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:23 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:23.513+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:24.071 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:23 smithi183 podman[73142]: 2024-03-23 15:55:23.801451069 +0000 UTC m=+5.698067633 container died 277ef7ab467f95072703d61f4561b70c73e5ffc5e276d675421831dfaca44e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-23T15:55:24.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:24.043+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:24.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:24.043+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:24.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:24.043+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:24.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:24.043+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:24.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:24 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:24.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:24.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:24 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:24.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:24.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:24 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:24.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:24.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:24 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:24.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:24.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:24 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:24.040+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:24.383 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:24 smithi183 podman[73142]: 2024-03-23 15:55:24.168076496 +0000 UTC m=+6.064693052 container cleanup 277ef7ab467f95072703d61f4561b70c73e5ffc5e276d675421831dfaca44e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-23T15:55:24.384 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:24 smithi183 bash[73142]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7 2024-03-23T15:55:24.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:24 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:24.071+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:24.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:24 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:24.071+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:24.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:24 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:24.071+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:24.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:24 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:24.071+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:24.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:24 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:24.071+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:24.785 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:24 smithi183 podman[73159]: 2024-03-23 15:55:24.444512752 +0000 UTC m=+0.645123659 container remove 277ef7ab467f95072703d61f4561b70c73e5ffc5e276d675421831dfaca44e44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-23T15:55:24.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:24.529+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:24.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:24.529+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:24.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:24.529+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:24.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:24 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:24.529+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:25.049 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:24 smithi183 podman[73250]: 2024-03-23 15:55:24.686539138 +0000 UTC m=+0.018296821 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:25.049 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:24 smithi183 podman[73250]: 2024-03-23 15:55:24.820414625 +0000 UTC m=+0.152172294 container create a9b90d0eb8bb24ea8307b4ebb95a55d68fe346003fff045828889aca217d1e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-23T15:55:25.320 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 podman[73250]: 2024-03-23 15:55:25.047721987 +0000 UTC m=+0.379479651 container init a9b90d0eb8bb24ea8307b4ebb95a55d68fe346003fff045828889aca217d1e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-23T15:55:25.321 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 podman[73250]: 2024-03-23 15:55:25.051140174 +0000 UTC m=+0.382897849 container start a9b90d0eb8bb24ea8307b4ebb95a55d68fe346003fff045828889aca217d1e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-03-23T15:55:25.321 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 podman[73250]: 2024-03-23 15:55:25.172835661 +0000 UTC m=+0.504593333 container attach a9b90d0eb8bb24ea8307b4ebb95a55d68fe346003fff045828889aca217d1e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:55:25.321 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:25 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:25.116+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:25.321 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:25 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:25.116+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:25.321 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:25 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:25.116+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:25.321 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:25 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:25.116+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:25.321 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:25 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:25.116+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:25.557 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:25.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:25.557 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:25.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:25.557 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:25.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:25.557 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:25.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:25.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:25 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:25.076+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:25.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:25 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:25.076+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:25.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:25 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:25.076+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:25.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:25 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:25.076+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:25.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:25 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:25.076+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:25.633 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 podman[73250]: 2024-03-23 15:55:25.362637338 +0000 UTC m=+0.694395022 container died a9b90d0eb8bb24ea8307b4ebb95a55d68fe346003fff045828889aca217d1e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2) 2024-03-23T15:55:25.819 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:25.554+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:25.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:25.554+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:25.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:25.554+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:25.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:25 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:25.554+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:25.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.7.service' 2024-03-23T15:55:26.004 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 podman[73277]: 2024-03-23 15:55:25.959513589 +0000 UTC m=+0.591621650 container remove a9b90d0eb8bb24ea8307b4ebb95a55d68fe346003fff045828889aca217d1e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:26.004 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.7.service: Deactivated successfully. 2024-03-23T15:55:26.004 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 systemd[1]: Stopped Ceph osd.7 for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:55:26.004 INFO:journalctl@ceph.osd.7.smithi183.stdout:Mar 23 15:55:25 smithi183 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.7.service: Consumed 48.140s CPU time. 2024-03-23T15:55:26.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:26.019 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-23T15:55:26.019 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-23T15:55:26.019 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.8 2024-03-23T15:55:26.266 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:26 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:26.112+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:26.266 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:26 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:26.112+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:26.266 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:26 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:26.112+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:26.266 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:26 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:26.112+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:26.266 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:26 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:26.112+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:26.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:26.062+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:26.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:26.062+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:26.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:26.062+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:26.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:26.062+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:26.320 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 systemd[1]: Stopping Ceph osd.8 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:26.361 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:26 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:26.102+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:26.361 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:26 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:26.102+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:26.361 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:26 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:26.102+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:26.361 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:26 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:26.102+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:26.361 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:26 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:26.102+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:26.820 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:26.334+0000 7f5f6867a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:26.820 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:26.334+0000 7f5f6867a700 -1 osd.8 100 *** Got signal Terminated *** 2024-03-23T15:55:26.820 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:26.334+0000 7f5f6867a700 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:55:26.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:26.544+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:26.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:26.544+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:26.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:26.544+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:26.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:26 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:26.544+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:27.530 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:27.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:27.530 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:27.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:27.530 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:27.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:27.530 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:27.078+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:27.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:27 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:27.103+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:27.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:27 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:27.103+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:27.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:27 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:27.103+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:27.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:27 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:27.103+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:27.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:27 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:27.103+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:27.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:27.134+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:27.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:27.134+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:27.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:27.134+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:27.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:27.134+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:27.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:27.134+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:27.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:27.527+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:27.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:27.527+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:27.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:27.527+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:27.799 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:27 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:27.527+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:28.566 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:28.089+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:28.566 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:28.089+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:28.566 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:28.089+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:28.567 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:28.089+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:28.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:28 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:28.133+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:28.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:28 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:28.133+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:28.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:28 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:28.133+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:28.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:28 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:28.133+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:28.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:28 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:28.133+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:28.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:28 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:28.157+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:28.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:28 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:28.157+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:28.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:28 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:28.157+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:28.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:28 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:28.157+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:28.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:28 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:28.157+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:28.819 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:28.563+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:28.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:28.563+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:28.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:28.563+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:28.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:28 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:28.563+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:29.534 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:29.156+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:29.535 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:29.156+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:29.535 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:29.156+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:29.535 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:29.156+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:29.535 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:29.156+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:29.535 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:29 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:29.156+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:29.569 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:29.134+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:29.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:29.134+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:29.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:29.134+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:29.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:29.134+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:29.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:29.134+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.052971+0000 front 2024-03-23T15:55:06.052988+0000 (oldest deadline 2024-03-23T15:55:28.353198+0000) 2024-03-23T15:55:29.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:29 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:29.172+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:29.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:29 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:29.172+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:29.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:29 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:29.172+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:29.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:29 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:29.172+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:29.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:29 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:29.172+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:30.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:29.595+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:30.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:29.595+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:30.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:29.595+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:30.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:29 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:29.595+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:30.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:30.111+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:30.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:30.111+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:30.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:30.111+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:30.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:30.111+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:30.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:30.111+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:30.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:30 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:30.111+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:30.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:30.174+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:30.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:30.174+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:30.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:30.174+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:30.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:30.174+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:30.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:30.174+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.052971+0000 front 2024-03-23T15:55:06.052988+0000 (oldest deadline 2024-03-23T15:55:28.353198+0000) 2024-03-23T15:55:30.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:30 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:30.185+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:30.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:30 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:30.185+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:30.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:30 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:30.185+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:30.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:30 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:30.185+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:30.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:30 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:30.185+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:31.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:30.622+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:31.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:30.622+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:31.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:30.622+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:31.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:30 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:30.622+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:31.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:31.188+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:23.247665+0000 front 2024-03-23T15:54:23.247678+0000 (oldest deadline 2024-03-23T15:54:49.147621+0000) 2024-03-23T15:55:31.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:31.188+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:38.549153+0000 front 2024-03-23T15:54:38.549253+0000 (oldest deadline 2024-03-23T15:55:03.849359+0000) 2024-03-23T15:55:31.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:31.188+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.250982+0000 front 2024-03-23T15:54:50.251984+0000 (oldest deadline 2024-03-23T15:55:12.551026+0000) 2024-03-23T15:55:31.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:31.188+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.852179+0000 front 2024-03-23T15:54:57.852122+0000 (oldest deadline 2024-03-23T15:55:20.752046+0000) 2024-03-23T15:55:31.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8[55518]: 2024-03-23T15:55:31.188+0000 7f5f60c66700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.052971+0000 front 2024-03-23T15:55:06.052988+0000 (oldest deadline 2024-03-23T15:55:28.353198+0000) 2024-03-23T15:55:31.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:31 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:31.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:31.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:31 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:31.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:31.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:31 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:31.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:31.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:31 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:31.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:31.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:31 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:31.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:31.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:31 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:31.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:31.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:31.160+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:31.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:31.160+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:31.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:31.160+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:31.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:31.160+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:31.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:31.160+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:31.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:31 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:31.160+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:32.070 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:31 smithi192 podman[74344]: 2024-03-23 15:55:31.865231048 +0000 UTC m=+5.659436892 container died 6bbdaa138d9012b48fd1d046020f1f7605e19daea69b66fc1eed2cf48f321e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-23T15:55:32.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:31.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:32.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:31.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:32.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:31.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:32.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:31 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:31.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:32.412 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:32 smithi192 podman[74344]: 2024-03-23 15:55:32.20959879 +0000 UTC m=+6.003804626 container cleanup 6bbdaa138d9012b48fd1d046020f1f7605e19daea69b66fc1eed2cf48f321e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-23T15:55:32.413 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:32 smithi192 bash[74344]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8 2024-03-23T15:55:32.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:32 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:32.198+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:32.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:32 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:32.198+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:32.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:32 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:32.198+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:32.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:32 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:32.198+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:32.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:32 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:32.198+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:32.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:32 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:32.198+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:32.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:32.143+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:32.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:32.143+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:32.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:32.143+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:32.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:32.143+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:32.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:32.143+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:32.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:32 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:32.143+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:32.820 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:32 smithi192 podman[74373]: 2024-03-23 15:55:32.410709398 +0000 UTC m=+0.542096398 container remove 6bbdaa138d9012b48fd1d046020f1f7605e19daea69b66fc1eed2cf48f321e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319) 2024-03-23T15:55:32.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:32.631+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:32.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:32.631+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:32.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:32.631+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:32.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:32.631+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:32.820 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:32.631+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:33.149 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:32 smithi192 podman[74461]: 2024-03-23 15:55:32.720773077 +0000 UTC m=+0.017649834 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:33.150 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:32 smithi192 podman[74461]: 2024-03-23 15:55:32.862024825 +0000 UTC m=+0.158901602 container create 0128a24b9419a0f8a7e0a8eef8e7bbb5c561fb08ff194dc4f3964d68ed9e95ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-23T15:55:33.150 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:33 smithi192 podman[74461]: 2024-03-23 15:55:33.064284045 +0000 UTC m=+0.361160795 container init 0128a24b9419a0f8a7e0a8eef8e7bbb5c561fb08ff194dc4f3964d68ed9e95ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, CEPH_POINT_RELEASE=) 2024-03-23T15:55:33.150 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:33 smithi192 podman[74461]: 2024-03-23 15:55:33.067622259 +0000 UTC m=+0.364499010 container start 0128a24b9419a0f8a7e0a8eef8e7bbb5c561fb08ff194dc4f3964d68ed9e95ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:55:33.150 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:33 smithi192 podman[74461]: 2024-03-23 15:55:33.147538084 +0000 UTC m=+0.444414850 container attach 0128a24b9419a0f8a7e0a8eef8e7bbb5c561fb08ff194dc4f3964d68ed9e95ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:55:33.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:33.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:33.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:33.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:33.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:33.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:33.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:33.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:33.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:33.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:33.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:33 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:33.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:33.570 INFO:journalctl@ceph.osd.8.smithi192.stdout:Mar 23 15:55:33 smithi192 podman[74461]: 2024-03-23 15:55:33.355631395 +0000 UTC m=+0.652508151 container died 0128a24b9419a0f8a7e0a8eef8e7bbb5c561fb08ff194dc4f3964d68ed9e95ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:55:33.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:33.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:33.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:33.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:33.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:33.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:33.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:33.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:33.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:33.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:33.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:33 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:33.226+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:33.970 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:33 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:33.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:33.970 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:33 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:33.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:33.970 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:33 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:33.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:33.970 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:33 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:33.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:33.970 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:33 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:33.665+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:33.997 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.8.service' 2024-03-23T15:55:34.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:34.040 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-23T15:55:34.040 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-23T15:55:34.041 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.9 2024-03-23T15:55:34.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 systemd[1]: Stopping Ceph osd.9 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:34.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.190+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:34.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.190+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:34.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.190+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:34.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.190+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:34.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.190+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:34.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.190+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:34.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:34.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:34.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:34.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:34.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:34.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:34.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:34.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:34.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:34.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:34.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:34 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:34.104+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:34.825 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.356+0000 7f649f468700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:34.826 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.356+0000 7f649f468700 -1 osd.9 100 *** Got signal Terminated *** 2024-03-23T15:55:34.826 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:34 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:34.356+0000 7f649f468700 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:55:35.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:34 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:34.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:35.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:34 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:34.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:35.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:34 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:34.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:35.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:34 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:34.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:35.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:34 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:34.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:35.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:35 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:35.074+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:35.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:35 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:35.074+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:35.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:35 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:35.074+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:35.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:35 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:35.074+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:35.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:35 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:35.074+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:35.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:35 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:35.074+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:35.147+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:35.147+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:35.147+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:35.147+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:35.147+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:35.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:35 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:35.147+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:36.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:35 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:35.690+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:36.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:35 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:35.690+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:36.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:35 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:35.690+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:36.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:35 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:35.690+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:36.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:35 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:35.690+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:36.334 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:36 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:36.041+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:36.334 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:36 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:36.041+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:36.334 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:36 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:36.041+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:36.334 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:36 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:36.041+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:36.335 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:36 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:36.041+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:36.335 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:36 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:36.041+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:36.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:36.195+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:36.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:36.195+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:36.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:36.195+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:36.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:36.195+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:36.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:36.195+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:36.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:36 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:36.195+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:37.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:36 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:36.680+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:37.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:36 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:36.680+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:37.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:36 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:36.680+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:37.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:36 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:36.680+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:37.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:36 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:36.680+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:37.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:37 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:37.037+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:37.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:37 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:37.037+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:37.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:37 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:37.037+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:37.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:37 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:37.037+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:37.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:37 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:37.037+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:37.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:37 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:37.037+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:37.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:37.158+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:37.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:37.158+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:37.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:37.158+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:37.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:37.158+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:37.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:37.158+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:37.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:37 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:37.158+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:38.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:37.697+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:38.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:37.697+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:38.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:37.697+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:38.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:37.697+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:38.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:37 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:37.697+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:38.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:38 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:38.056+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:38.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:38 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:38.056+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:38.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:38 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:38.056+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:38.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:38 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:38.056+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:38.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:38 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:38.056+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:38.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:38 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:38.056+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:38.575 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:38 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:38.143+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:38.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:38 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:38.143+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:38.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:38 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:38.143+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:38.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:38 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:38.143+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:38.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:38 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:38.143+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:38.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:38 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:38.143+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:39.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:38 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:38.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:39.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:38 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:38.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:39.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:38 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:38.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:39.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:38 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:38.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:39.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:38 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:38.669+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:39.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:39 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:39.076+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:39.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:39 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:39.076+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:39.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:39 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:39.076+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:39.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:39 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:39.076+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:39.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:39 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:39.076+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:39.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:39 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:39.076+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:39.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:39 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:39.076+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:39.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:39.164+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.545873+0000 front 2024-03-23T15:54:24.545966+0000 (oldest deadline 2024-03-23T15:54:50.446073+0000) 2024-03-23T15:55:39.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:39.164+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.247856+0000 front 2024-03-23T15:54:33.247908+0000 (oldest deadline 2024-03-23T15:55:00.848609+0000) 2024-03-23T15:55:39.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:39.164+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:40.849354+0000 front 2024-03-23T15:54:40.849255+0000 (oldest deadline 2024-03-23T15:55:06.749042+0000) 2024-03-23T15:55:39.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:39.164+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:50.650589+0000 front 2024-03-23T15:54:50.650623+0000 (oldest deadline 2024-03-23T15:55:15.950618+0000) 2024-03-23T15:55:39.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:39.164+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:57.551975+0000 front 2024-03-23T15:54:57.552062+0000 (oldest deadline 2024-03-23T15:55:19.852200+0000) 2024-03-23T15:55:39.576 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:39 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9[65118]: 2024-03-23T15:55:39.164+0000 7f649ba5c700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.153434+0000 front 2024-03-23T15:55:05.153380+0000 (oldest deadline 2024-03-23T15:55:30.353948+0000) 2024-03-23T15:55:40.003 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:39 smithi076 podman[91519]: 2024-03-23 15:55:39.704929961 +0000 UTC m=+5.467687468 container died 497764e16f6ccd5ca5bbff7dc2169a7bde65822291799af72131e7e7cfb99fdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-23T15:55:40.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:39 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:39.626+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:40.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:39 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:39.626+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:40.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:39 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:39.626+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:40.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:39 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:39.626+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:40.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:39 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:39.626+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:40.071 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:39 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:39.627+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:40.325 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 podman[91519]: 2024-03-23 15:55:40.091222031 +0000 UTC m=+5.853979552 container cleanup 497764e16f6ccd5ca5bbff7dc2169a7bde65822291799af72131e7e7cfb99fdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-23T15:55:40.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 bash[91519]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9 2024-03-23T15:55:40.387 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:40 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:40.110+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:40.387 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:40 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:40.110+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:40.387 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:40 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:40.110+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:40.387 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:40 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:40.110+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:40.387 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:40 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:40.110+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:40.387 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:40 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:40.110+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:40.387 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:40 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:40.110+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:40.664 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 podman[91535]: 2024-03-23 15:55:40.33421879 +0000 UTC m=+0.624308346 container remove 497764e16f6ccd5ca5bbff7dc2169a7bde65822291799af72131e7e7cfb99fdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS) 2024-03-23T15:55:40.981 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 podman[91627]: 2024-03-23 15:55:40.564468608 +0000 UTC m=+0.018427396 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:40.981 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 podman[91627]: 2024-03-23 15:55:40.668617437 +0000 UTC m=+0.122576197 container create 98a6021b4de963ed2068a27777e28b52496c8849e955b7ea92d4a0cd2de57e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:40.981 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 podman[91627]: 2024-03-23 15:55:40.905189214 +0000 UTC m=+0.359147982 container init 98a6021b4de963ed2068a27777e28b52496c8849e955b7ea92d4a0cd2de57e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:55:40.982 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 podman[91627]: 2024-03-23 15:55:40.908553005 +0000 UTC m=+0.362511769 container start 98a6021b4de963ed2068a27777e28b52496c8849e955b7ea92d4a0cd2de57e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T15:55:40.982 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:40 smithi076 podman[91627]: 2024-03-23 15:55:40.979576116 +0000 UTC m=+0.433534899 container attach 98a6021b4de963ed2068a27777e28b52496c8849e955b7ea92d4a0cd2de57e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-23T15:55:41.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:40 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:40.672+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:41.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:40 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:40.672+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:41.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:40 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:40.672+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:41.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:40 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:40.672+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:41.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:40 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:40.672+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:41.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:40 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:40.672+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:41.326 INFO:journalctl@ceph.osd.9.smithi076.stdout:Mar 23 15:55:41 smithi076 podman[91627]: 2024-03-23 15:55:41.199193899 +0000 UTC m=+0.653152662 container died 98a6021b4de963ed2068a27777e28b52496c8849e955b7ea92d4a0cd2de57e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-23T15:55:41.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:41.061+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:41.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:41.061+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:41.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:41.061+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:41.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:41.061+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:41.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:41.061+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:41.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:41.061+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:41.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:41.061+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:41.766 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.9.service' 2024-03-23T15:55:41.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:41.807 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-23T15:55:41.807 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-23T15:55:41.807 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.10 2024-03-23T15:55:42.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:41.666+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:42.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:41.666+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:42.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:41.666+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:42.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:41.666+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:42.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:41.666+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:42.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:41 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:41.666+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:42.122 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:41 smithi183 systemd[1]: Stopping Ceph osd.10 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:42.122 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.080+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:42.122 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.080+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:42.122 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.080+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:42.123 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.080+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:42.123 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.080+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:42.123 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.080+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:42.123 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.080+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:42.123 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.119+0000 7fb22c98a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:42.123 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.119+0000 7fb22c98a700 -1 osd.10 100 *** Got signal Terminated *** 2024-03-23T15:55:42.123 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:42 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:42.119+0000 7fb22c98a700 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:55:43.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:42 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:42.695+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:43.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:42 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:42.695+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:43.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:42 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:42.695+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:43.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:42 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:42.695+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:43.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:42 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:42.695+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:43.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:42 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:42.695+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:43.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:43.083+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:43.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:43.083+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:43.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:43.083+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:43.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:43.083+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:43.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:43.083+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:43.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:43.083+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:43.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:43 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:43.083+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:44.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:43.711+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:44.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:43.711+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:44.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:43.711+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:44.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:43.711+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:44.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:43.711+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:44.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:43.711+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:44.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:44.078+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:44.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:44.078+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:44.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:44.078+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:44.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:44.078+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:44.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:44.078+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:44.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:44.078+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:44.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:44 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:44.078+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:45.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:44 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:44.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:45.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:44 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:44.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:45.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:44 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:44.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:45.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:44 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:44.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:45.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:44 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:44.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:45.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:44 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:44.707+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:45.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:45 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:45.107+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:45.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:45 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:45.107+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:45.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:45 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:45.107+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:45.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:45 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:45.107+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:45.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:45 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:45.107+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:45.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:45 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:45.107+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:45.384 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:45 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:45.107+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:46.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:45 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:45.752+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:46.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:45 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:45.752+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:46.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:45 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:45.752+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:46.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:45 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:45.752+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:46.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:45 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:45.752+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:46.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:45 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:45.752+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:46.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:45 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:45.752+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:46.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:46 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:46.131+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:24.732501+0000 front 2024-03-23T15:54:24.732675+0000 (oldest deadline 2024-03-23T15:54:48.832413+0000) 2024-03-23T15:55:46.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:46 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:46.131+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:29.933364+0000 front 2024-03-23T15:54:29.933439+0000 (oldest deadline 2024-03-23T15:54:55.233570+0000) 2024-03-23T15:55:46.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:46 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:46.131+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.134599+0000 front 2024-03-23T15:54:41.134545+0000 (oldest deadline 2024-03-23T15:55:05.234743+0000) 2024-03-23T15:55:46.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:46 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:46.131+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6814 osd.3 since back 2024-03-23T15:54:48.636189+0000 front 2024-03-23T15:54:48.636262+0000 (oldest deadline 2024-03-23T15:55:12.136356+0000) 2024-03-23T15:55:46.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:46 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:46.131+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:58.438296+0000 front 2024-03-23T15:54:58.438369+0000 (oldest deadline 2024-03-23T15:55:20.138390+0000) 2024-03-23T15:55:46.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:46 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:46.131+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:05.439747+0000 front 2024-03-23T15:55:05.439807+0000 (oldest deadline 2024-03-23T15:55:28.339537+0000) 2024-03-23T15:55:46.634 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:46 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10[61335]: 2024-03-23T15:55:46.131+0000 7fb224f76700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:13.640601+0000 front 2024-03-23T15:55:13.640570+0000 (oldest deadline 2024-03-23T15:55:38.340413+0000) 2024-03-23T15:55:47.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:46 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:46.742+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:47.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:46 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:46.742+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:47.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:46 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:46.742+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:47.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:46 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:46.742+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:47.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:46 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:46.742+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:47.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:46 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:46.742+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:47.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:46 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:46.742+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:47.872 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:47 smithi183 podman[73367]: 2024-03-23 15:55:47.609326825 +0000 UTC m=+5.611133030 container died 89ab8420319b622bc40bd2601af82e5bba17fcb8fc01e3525fa0bb516794f167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-23T15:55:48.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:47 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:47.753+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:48.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:47 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:47.753+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:48.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:47 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:47.753+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:48.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:47 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:47.753+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:48.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:47 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:47.753+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:48.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:47 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:47.753+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:48.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:47 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:47.753+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:48.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 podman[73367]: 2024-03-23 15:55:48.094058319 +0000 UTC m=+6.095864518 container cleanup 89ab8420319b622bc40bd2601af82e5bba17fcb8fc01e3525fa0bb516794f167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-23T15:55:48.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 bash[73367]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10 2024-03-23T15:55:48.633 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 podman[73384]: 2024-03-23 15:55:48.336700301 +0000 UTC m=+0.726259587 container remove 89ab8420319b622bc40bd2601af82e5bba17fcb8fc01e3525fa0bb516794f167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:48.933 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 podman[73477]: 2024-03-23 15:55:48.555251257 +0000 UTC m=+0.017955826 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:48.933 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 podman[73477]: 2024-03-23 15:55:48.662259689 +0000 UTC m=+0.124964251 container create 9b3fee93728dfad85b4d5da6976e3751fcedce0bc669a20d4b44b8f2904aaafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:48.933 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 podman[73477]: 2024-03-23 15:55:48.856046082 +0000 UTC m=+0.318750635 container init 9b3fee93728dfad85b4d5da6976e3751fcedce0bc669a20d4b44b8f2904aaafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-23T15:55:48.933 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 podman[73477]: 2024-03-23 15:55:48.859369755 +0000 UTC m=+0.322074320 container start 9b3fee93728dfad85b4d5da6976e3751fcedce0bc669a20d4b44b8f2904aaafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:55:48.933 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:48 smithi183 podman[73477]: 2024-03-23 15:55:48.930851698 +0000 UTC m=+0.393556266 container attach 9b3fee93728dfad85b4d5da6976e3751fcedce0bc669a20d4b44b8f2904aaafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-23T15:55:49.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:48.719+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:49.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:48.719+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:49.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:48.719+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:49.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:48.719+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:49.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:48.719+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:49.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:48.719+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:49.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:48 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:48.719+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:49.383 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:49 smithi183 podman[73477]: 2024-03-23 15:55:49.149519357 +0000 UTC m=+0.612223925 container died 9b3fee93728dfad85b4d5da6976e3751fcedce0bc669a20d4b44b8f2904aaafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-23T15:55:49.727 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:49 smithi183 podman[73503]: 2024-03-23 15:55:49.708980019 +0000 UTC m=+0.551124814 container remove 9b3fee93728dfad85b4d5da6976e3751fcedce0bc669a20d4b44b8f2904aaafe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-23T15:55:49.728 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:49 smithi183 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.10.service: Deactivated successfully. 2024-03-23T15:55:49.728 INFO:journalctl@ceph.osd.10.smithi183.stdout:Mar 23 15:55:49 smithi183 systemd[1]: Stopped Ceph osd.10 for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:55:49.730 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.10.service' 2024-03-23T15:55:49.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:49.770 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-23T15:55:49.771 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-23T15:55:49.771 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.11 2024-03-23T15:55:50.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:49.689+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:49.689+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:49.689+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:49.689+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:49.689+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:49.689+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:49.689+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:49 smithi192 systemd[1]: Stopping Ceph osd.11 for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.061+0000 7f6c8f38e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.061+0000 7f6c8f38e700 -1 osd.11 100 *** Got signal Terminated *** 2024-03-23T15:55:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.061+0000 7f6c8f38e700 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T15:55:51.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.739+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:51.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.739+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:51.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.739+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:51.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.739+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:51.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.739+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:51.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.739+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:51.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:50 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:50.739+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:52.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:51.785+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:52.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:51.785+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:52.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:51.785+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:52.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:51.785+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:52.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:51.785+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:52.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:51.785+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:52.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:51 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:51.785+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:53.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:52.811+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:53.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:52.811+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:53.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:52.811+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:53.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:52.811+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:53.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:52.811+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:53.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:52.811+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:53.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:52 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:52.811+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:54.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:53 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:53.771+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-03-23T15:55:29.660670+0000 front 2024-03-23T15:55:29.660689+0000 (oldest deadline 2024-03-23T15:55:53.160709+0000) 2024-03-23T15:55:55.069 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6806 osd.0 since back 2024-03-23T15:54:26.350413+0000 front 2024-03-23T15:54:26.350187+0000 (oldest deadline 2024-03-23T15:54:49.850308+0000) 2024-03-23T15:55:55.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-03-23T15:54:33.252155+0000 front 2024-03-23T15:54:33.251855+0000 (oldest deadline 2024-03-23T15:54:56.752013+0000) 2024-03-23T15:55:55.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-03-23T15:54:41.453215+0000 front 2024-03-23T15:54:41.453043+0000 (oldest deadline 2024-03-23T15:55:06.153297+0000) 2024-03-23T15:55:55.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-03-23T15:54:56.456208+0000 front 2024-03-23T15:54:56.456098+0000 (oldest deadline 2024-03-23T15:55:21.156007+0000) 2024-03-23T15:55:55.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-03-23T15:55:06.357482+0000 front 2024-03-23T15:55:06.357531+0000 (oldest deadline 2024-03-23T15:55:32.257632+0000) 2024-03-23T15:55:55.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.76:6822 osd.6 since back 2024-03-23T15:55:14.459098+0000 front 2024-03-23T15:55:14.459082+0000 (oldest deadline 2024-03-23T15:55:39.158812+0000) 2024-03-23T15:55:55.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-03-23T15:55:22.659991+0000 front 2024-03-23T15:55:22.660016+0000 (oldest deadline 2024-03-23T15:55:45.559854+0000) 2024-03-23T15:55:55.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:54 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11[62469]: 2024-03-23T15:55:54.793+0000 7f6c8797a700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-03-23T15:55:29.660670+0000 front 2024-03-23T15:55:29.660689+0000 (oldest deadline 2024-03-23T15:55:53.160709+0000) 2024-03-23T15:55:55.696 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:55 smithi192 podman[74568]: 2024-03-23 15:55:55.398503803 +0000 UTC m=+5.440763402 container died 5d337b0fe268dfd0ccd198d358c33383894369d82e126e569eccdbd3310e5e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-03-23T15:55:55.697 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:55 smithi192 podman[74568]: 2024-03-23 15:55:55.694680676 +0000 UTC m=+5.736940265 container cleanup 5d337b0fe268dfd0ccd198d358c33383894369d82e126e569eccdbd3310e5e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-23T15:55:56.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:55 smithi192 bash[74568]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11 2024-03-23T15:55:56.070 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:55 smithi192 podman[74585]: 2024-03-23 15:55:55.920721586 +0000 UTC m=+0.516677030 container remove 5d337b0fe268dfd0ccd198d358c33383894369d82e126e569eccdbd3310e5e2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-23T15:55:56.525 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:56 smithi192 podman[74678]: 2024-03-23 15:55:56.148103754 +0000 UTC m=+0.017870933 image pull 553463a90fd5dcbd654ed9debc69e27bdd9d55bf450e60a53eb3e730689e5e12 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919 2024-03-23T15:55:56.525 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:56 smithi192 podman[74678]: 2024-03-23 15:55:56.246509119 +0000 UTC m=+0.116276327 container create 85086d5d70b046d231a0b2639cda2aa5974994e225ab4239cbab9b3b8ddda215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-23T15:55:56.525 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:56 smithi192 podman[74678]: 2024-03-23 15:55:56.457061337 +0000 UTC m=+0.326828526 container init 85086d5d70b046d231a0b2639cda2aa5974994e225ab4239cbab9b3b8ddda215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:55:56.526 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:56 smithi192 podman[74678]: 2024-03-23 15:55:56.460238594 +0000 UTC m=+0.330005766 container start 85086d5d70b046d231a0b2639cda2aa5974994e225ab4239cbab9b3b8ddda215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-23T15:55:56.526 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:56 smithi192 podman[74678]: 2024-03-23 15:55:56.523517936 +0000 UTC m=+0.393285136 container attach 85086d5d70b046d231a0b2639cda2aa5974994e225ab4239cbab9b3b8ddda215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-23T15:55:56.819 INFO:journalctl@ceph.osd.11.smithi192.stdout:Mar 23 15:55:56 smithi192 podman[74678]: 2024-03-23 15:55:56.753190703 +0000 UTC m=+0.622957890 container died 85086d5d70b046d231a0b2639cda2aa5974994e225ab4239cbab9b3b8ddda215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-23T15:55:57.314 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@osd.11.service' 2024-03-23T15:55:57.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:55:57.353 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-23T15:55:57.353 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-23T15:55:57.353 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.a 2024-03-23T15:55:57.660 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:55:57 smithi076 systemd[1]: Stopping Ceph mds.a for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:55:58.076 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:55:57 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a[71809]: 2024-03-23T15:55:57.657+0000 7f51fe7fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:55:58.076 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:55:57 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a[71809]: 2024-03-23T15:55:57.657+0000 7f51fe7fa700 -1 mds.a *** got signal Terminated *** 2024-03-23T15:56:01.953 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:56:01 smithi076 podman[91748]: 2024-03-23 15:56:01.653627287 +0000 UTC m=+4.098080208 container died c4dd412540ede56784f888a68f71239b7787b3ac0b11f05ab24933945b4f876e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:56:02.259 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:56:02 smithi076 podman[91748]: 2024-03-23 15:56:02.031476098 +0000 UTC m=+4.475929027 container cleanup c4dd412540ede56784f888a68f71239b7787b3ac0b11f05ab24933945b4f876e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD) 2024-03-23T15:56:02.260 INFO:journalctl@ceph.mds.a.smithi076.stdout:Mar 23 15:56:02 smithi076 bash[91748]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-a 2024-03-23T15:56:02.389 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.a.service' 2024-03-23T15:56:02.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:02.427 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-23T15:56:02.427 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-23T15:56:02.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.d 2024-03-23T15:56:02.780 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:56:02 smithi076 systemd[1]: Stopping Ceph mds.d for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:03.075 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:56:02 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d[70853]: 2024-03-23T15:56:02.778+0000 7f84553b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:03.076 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:56:02 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d[70853]: 2024-03-23T15:56:02.778+0000 7f84553b5700 -1 mds.d *** got signal Terminated *** 2024-03-23T15:56:05.825 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:56:05 smithi076 podman[91868]: 2024-03-23 15:56:05.541089668 +0000 UTC m=+2.887943216 container died 8383b805fa59f8a3bf7a9bf7f363eb1cf258dea52fb6df429cd885cda0c95657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-23T15:56:06.129 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:56:05 smithi076 podman[91868]: 2024-03-23 15:56:05.844637438 +0000 UTC m=+3.191490988 container cleanup 8383b805fa59f8a3bf7a9bf7f363eb1cf258dea52fb6df429cd885cda0c95657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-23T15:56:06.129 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:56:05 smithi076 bash[91868]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d 2024-03-23T15:56:06.129 INFO:journalctl@ceph.mds.d.smithi076.stdout:Mar 23 15:56:06 smithi076 podman[91882]: 2024-03-23 15:56:06.046027911 +0000 UTC m=+0.504455654 container remove 8383b805fa59f8a3bf7a9bf7f363eb1cf258dea52fb6df429cd885cda0c95657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-23T15:56:06.152 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.d.service' 2024-03-23T15:56:06.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:06.187 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-23T15:56:06.187 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-23T15:56:06.187 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.g 2024-03-23T15:56:06.508 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:56:06 smithi076 systemd[1]: Stopping Ceph mds.g for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:06.825 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:56:06 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g[71449]: 2024-03-23T15:56:06.505+0000 7fcc4bd8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:06.825 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:56:06 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g[71449]: 2024-03-23T15:56:06.505+0000 7fcc4bd8b700 -1 mds.g *** got signal Terminated *** 2024-03-23T15:56:10.802 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:56:10 smithi076 podman[91989]: 2024-03-23 15:56:10.512737883 +0000 UTC m=+4.112431829 container died 4776c633b3457427a871e89c3e71b553d2ce639bb3eaecfd7c234ca076e9a90f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-23T15:56:11.075 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:56:10 smithi076 podman[91989]: 2024-03-23 15:56:10.890566585 +0000 UTC m=+4.490260530 container cleanup 4776c633b3457427a871e89c3e71b553d2ce639bb3eaecfd7c234ca076e9a90f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-23T15:56:11.075 INFO:journalctl@ceph.mds.g.smithi076.stdout:Mar 23 15:56:10 smithi076 bash[91989]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-g 2024-03-23T15:56:11.316 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.g.service' 2024-03-23T15:56:11.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:11.350 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-23T15:56:11.350 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-23T15:56:11.350 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.j 2024-03-23T15:56:11.825 INFO:journalctl@ceph.mds.j.smithi076.stdout:Mar 23 15:56:11 smithi076 systemd[1]: Stopping Ceph mds.j for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:11.826 INFO:journalctl@ceph.mds.j.smithi076.stdout:Mar 23 15:56:11 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-j[70502]: 2024-03-23T15:56:11.652+0000 7fe637e83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:11.826 INFO:journalctl@ceph.mds.j.smithi076.stdout:Mar 23 15:56:11 smithi076 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-j[70502]: 2024-03-23T15:56:11.652+0000 7fe637e83700 -1 mds.j *** got signal Terminated *** 2024-03-23T15:56:14.956 INFO:journalctl@ceph.mds.j.smithi076.stdout:Mar 23 15:56:14 smithi076 podman[92113]: 2024-03-23 15:56:14.685947451 +0000 UTC m=+3.121055101 container died 71b8f665b914c6dba6687b7fd8ead913203381c67ade374ea2e14d42e95619ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-23T15:56:15.323 INFO:journalctl@ceph.mds.j.smithi076.stdout:Mar 23 15:56:15 smithi076 podman[92113]: 2024-03-23 15:56:15.053730217 +0000 UTC m=+3.488837930 container cleanup 71b8f665b914c6dba6687b7fd8ead913203381c67ade374ea2e14d42e95619ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-23T15:56:15.323 INFO:journalctl@ceph.mds.j.smithi076.stdout:Mar 23 15:56:15 smithi076 bash[92113]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-j 2024-03-23T15:56:15.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.j.service' 2024-03-23T15:56:15.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:15.512 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-23T15:56:15.512 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-23T15:56:15.512 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.b 2024-03-23T15:56:15.861 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:56:15 smithi183 systemd[1]: Stopping Ceph mds.b for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:16.125 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:56:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: 2024-03-23T15:56:15.857+0000 7f0ed0398700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:16.125 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:56:15 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b[67007]: 2024-03-23T15:56:15.857+0000 7f0ed0398700 -1 mds.b *** got signal Terminated *** 2024-03-23T15:56:18.087 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:56:17 smithi183 podman[73583]: 2024-03-23 15:56:17.801746995 +0000 UTC m=+2.106179734 container died cf5580cbf4f26d629fb0d661b5a211b8b9b47175633dd6ff7c99dfca57c0adee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-23T15:56:18.383 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:56:18 smithi183 podman[73583]: 2024-03-23 15:56:18.199920902 +0000 UTC m=+2.504353649 container cleanup cf5580cbf4f26d629fb0d661b5a211b8b9b47175633dd6ff7c99dfca57c0adee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-23T15:56:18.383 INFO:journalctl@ceph.mds.b.smithi183.stdout:Mar 23 15:56:18 smithi183 bash[73583]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-b 2024-03-23T15:56:18.570 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.b.service' 2024-03-23T15:56:18.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:18.609 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-23T15:56:18.609 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-23T15:56:18.609 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.e 2024-03-23T15:56:19.133 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:18 smithi183 systemd[1]: Stopping Ceph mds.e for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:19.133 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e[66393]: 2024-03-23T15:56:18.935+0000 7fc56a813700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:19.133 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:18 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e[66393]: 2024-03-23T15:56:18.935+0000 7fc56a813700 -1 mds.e *** got signal Terminated *** 2024-03-23T15:56:22.089 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:21 smithi183 podman[73715]: 2024-03-23 15:56:21.797093073 +0000 UTC m=+2.969567140 container died 94452d8f2e84d0c3d9d5d7cd4ad277154377aa75484dc136a0541096f27d20f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-23T15:56:22.348 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:22 smithi183 podman[73715]: 2024-03-23 15:56:22.161219479 +0000 UTC m=+3.333693546 container cleanup 94452d8f2e84d0c3d9d5d7cd4ad277154377aa75484dc136a0541096f27d20f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-23T15:56:22.348 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:22 smithi183 bash[73715]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e 2024-03-23T15:56:22.575 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.e.service' 2024-03-23T15:56:22.599 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:22 smithi183 podman[73729]: 2024-03-23 15:56:22.346297042 +0000 UTC m=+0.546171121 container remove 94452d8f2e84d0c3d9d5d7cd4ad277154377aa75484dc136a0541096f27d20f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-03-23T15:56:22.599 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:22 smithi183 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.e.service: Deactivated successfully. 2024-03-23T15:56:22.599 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:22 smithi183 systemd[1]: Stopped Ceph mds.e for 89c3e582-e92a-11ee-95cd-87774f69a715. 2024-03-23T15:56:22.599 INFO:journalctl@ceph.mds.e.smithi183.stdout:Mar 23 15:56:22 smithi183 systemd[1]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.e.service: Consumed 3.111s CPU time. 2024-03-23T15:56:22.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:22.611 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-23T15:56:22.611 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-23T15:56:22.611 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.h 2024-03-23T15:56:22.883 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:56:22 smithi183 systemd[1]: Stopping Ceph mds.h for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:23.383 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:56:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: 2024-03-23T15:56:22.932+0000 7fe5d3220700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:23.383 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:56:22 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h[66698]: 2024-03-23T15:56:22.932+0000 7fe5d3220700 -1 mds.h *** got signal Terminated *** 2024-03-23T15:56:26.133 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:56:25 smithi183 podman[73835]: 2024-03-23 15:56:25.810212309 +0000 UTC m=+2.985890777 container died be9a99f2bdc163e76bc13b55d979ec33c44974cd3ce3ad94ae848a8886c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.vendor=CentOS) 2024-03-23T15:56:26.547 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:56:26 smithi183 podman[73835]: 2024-03-23 15:56:26.273267227 +0000 UTC m=+3.448945701 container cleanup be9a99f2bdc163e76bc13b55d979ec33c44974cd3ce3ad94ae848a8886c511d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-23T15:56:26.548 INFO:journalctl@ceph.mds.h.smithi183.stdout:Mar 23 15:56:26 smithi183 bash[73835]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-h 2024-03-23T15:56:26.693 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.h.service' 2024-03-23T15:56:26.727 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:26.728 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-23T15:56:26.728 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-23T15:56:26.728 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.k 2024-03-23T15:56:27.112 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:56:26 smithi183 systemd[1]: Stopping Ceph mds.k for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:27.112 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:56:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k[66073]: 2024-03-23T15:56:27.051+0000 7f0ad9c4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:27.113 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:56:27 smithi183 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k[66073]: 2024-03-23T15:56:27.051+0000 7f0ad9c4e700 -1 mds.k *** got signal Terminated *** 2024-03-23T15:56:31.371 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:56:31 smithi183 podman[73957]: 2024-03-23 15:56:31.057527418 +0000 UTC m=+4.117627657 container died db71d6822185d8474544eb54843b40ea26f8834303d12f1cd43d4684e5dcb20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-23T15:56:31.633 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:56:31 smithi183 podman[73957]: 2024-03-23 15:56:31.444033049 +0000 UTC m=+4.504133281 container cleanup db71d6822185d8474544eb54843b40ea26f8834303d12f1cd43d4684e5dcb20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-23T15:56:31.633 INFO:journalctl@ceph.mds.k.smithi183.stdout:Mar 23 15:56:31 smithi183 bash[73957]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-k 2024-03-23T15:56:31.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.k.service' 2024-03-23T15:56:31.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:31.831 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-23T15:56:31.831 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-23T15:56:31.831 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.c 2024-03-23T15:56:32.175 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:56:31 smithi192 systemd[1]: Stopping Ceph mds.c for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:32.436 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:56:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c[67641]: 2024-03-23T15:56:32.172+0000 7f50adb92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:32.436 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:56:32 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c[67641]: 2024-03-23T15:56:32.172+0000 7f50adb92700 -1 mds.c *** got signal Terminated *** 2024-03-23T15:56:34.069 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:56:33 smithi192 podman[74794]: 2024-03-23 15:56:33.814475214 +0000 UTC m=+1.792066718 container died 982978143faa59a4208ff6d61bc090a2a3f13acb6fff350ad3888bd8698201e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-23T15:56:34.438 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:56:34 smithi192 podman[74794]: 2024-03-23 15:56:34.168348671 +0000 UTC m=+2.145940161 container cleanup 982978143faa59a4208ff6d61bc090a2a3f13acb6fff350ad3888bd8698201e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-23T15:56:34.438 INFO:journalctl@ceph.mds.c.smithi192.stdout:Mar 23 15:56:34 smithi192 bash[74794]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-c 2024-03-23T15:56:34.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.c.service' 2024-03-23T15:56:34.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:34.631 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-23T15:56:34.631 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-23T15:56:34.631 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.f 2024-03-23T15:56:34.958 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:56:34 smithi192 systemd[1]: Stopping Ceph mds.f for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:35.319 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:56:34 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f[67316]: 2024-03-23T15:56:34.955+0000 7f7db2c94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:35.319 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:56:34 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f[67316]: 2024-03-23T15:56:34.955+0000 7f7db2c94700 -1 mds.f *** got signal Terminated *** 2024-03-23T15:56:38.069 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:56:37 smithi192 podman[74927]: 2024-03-23 15:56:37.814763162 +0000 UTC m=+2.967609249 container died f049503cec02a660a40df821b6126ee6d5896530baa99e37eb11c6e455164bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6) 2024-03-23T15:56:38.441 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:56:38 smithi192 podman[74927]: 2024-03-23 15:56:38.171881448 +0000 UTC m=+3.324727490 container cleanup f049503cec02a660a40df821b6126ee6d5896530baa99e37eb11c6e455164bab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:56:38.441 INFO:journalctl@ceph.mds.f.smithi192.stdout:Mar 23 15:56:38 smithi192 bash[74927]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-f 2024-03-23T15:56:38.599 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.f.service' 2024-03-23T15:56:38.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:38.635 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-23T15:56:38.635 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-23T15:56:38.635 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.i 2024-03-23T15:56:39.069 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:56:38 smithi192 systemd[1]: Stopping Ceph mds.i for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:39.069 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:56:38 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:56:38.940+0000 7f420b9a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:39.070 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:56:38 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i[67945]: 2024-03-23T15:56:38.941+0000 7f420b9a4700 -1 mds.i *** got signal Terminated *** 2024-03-23T15:56:42.069 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:56:41 smithi192 podman[75047]: 2024-03-23 15:56:41.80129974 +0000 UTC m=+2.957321935 container died 0bc7bf21b48ab05ace1937d2f5ec0b5e7147d0909390ee3f9232b639f00b62ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-23T15:56:42.528 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:56:42 smithi192 podman[75047]: 2024-03-23 15:56:42.258832725 +0000 UTC m=+3.414854926 container cleanup 0bc7bf21b48ab05ace1937d2f5ec0b5e7147d0909390ee3f9232b639f00b62ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-23T15:56:42.528 INFO:journalctl@ceph.mds.i.smithi192.stdout:Mar 23 15:56:42 smithi192 bash[75047]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-i 2024-03-23T15:56:42.677 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.i.service' 2024-03-23T15:56:42.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:42.711 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-23T15:56:42.711 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-23T15:56:42.711 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.l 2024-03-23T15:56:43.047 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:56:42 smithi192 systemd[1]: Stopping Ceph mds.l for 89c3e582-e92a-11ee-95cd-87774f69a715... 2024-03-23T15:56:43.319 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:56:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l[67008]: 2024-03-23T15:56:43.045+0000 7fdd9170a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T15:56:43.319 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:56:43 smithi192 ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l[67008]: 2024-03-23T15:56:43.045+0000 7fdd9170a700 -1 mds.l *** got signal Terminated *** 2024-03-23T15:56:45.256 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:56:44 smithi192 podman[75169]: 2024-03-23 15:56:44.950343411 +0000 UTC m=+2.020588491 container died ce914a2b35931c7a40e9071f7232c1920adc79e1dba73544194e563e41cce427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-23T15:56:45.565 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:56:45 smithi192 podman[75169]: 2024-03-23 15:56:45.345909264 +0000 UTC m=+2.416154324 container cleanup ce914a2b35931c7a40e9071f7232c1920adc79e1dba73544194e563e41cce427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a32537e3857c015cff859e834d65089d866c82026571595cd97228205b5b8919, name=ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240323.003144-squid-21b3bd6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-23T15:56:45.565 INFO:journalctl@ceph.mds.l.smithi192.stdout:Mar 23 15:56:45 smithi192 bash[75169]: ceph-89c3e582-e92a-11ee-95cd-87774f69a715-mds-l 2024-03-23T15:56:45.689 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-89c3e582-e92a-11ee-95cd-87774f69a715@mds.l.service' 2024-03-23T15:56:45.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T15:56:45.724 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-23T15:56:45.724 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T15:56:45.912 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:56:48.359 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T15:56:48.550 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:56:51.192 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T15:56:51.381 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:56:53.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:56:53.757 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:56:53.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T15:56:53.821 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-23T15:56:53.824 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi076/crash 2024-03-23T15:56:53.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash -- . 2024-03-23T15:56:53.852 INFO:teuthology.orchestra.run.smithi076.stderr:tar: /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T15:56:53.853 INFO:teuthology.orchestra.run.smithi076.stderr:tar: Error is not recoverable: exiting now 2024-03-23T15:56:53.855 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi183/crash 2024-03-23T15:56:53.856 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash -- . 2024-03-23T15:56:53.893 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T15:56:53.893 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-03-23T15:56:53.895 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi192/crash 2024-03-23T15:56:53.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash -- . 2024-03-23T15:56:53.937 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T15:56:53.937 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-03-23T15:56:53.938 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-23T15:56:53.939 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-23T15:56:53.973 INFO:tasks.cephadm:Compressing logs... 2024-03-23T15:56:53.974 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T15:56:54.016 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T15:56:54.018 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T15:56:54.041 INFO:teuthology.orchestra.run.smithi076.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T15:56:54.041 INFO:teuthology.orchestra.run.smithi076.stderr:: No such file or directory 2024-03-23T15:56:54.041 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-23T15:56:54.043 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-23T15:56:54.044 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T15:56:54.044 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-23T15:56:54.044 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-23T15:56:54.044 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log: /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-23T15:56:54.044 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.j.log: -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-23T15:56:54.044 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-23T15:56:54.045 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T15:56:54.045 INFO:teuthology.orchestra.run.smithi192.stderr:: No such file or directory 2024-03-23T15:56:54.045 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-23T15:56:54.050 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T15:56:54.050 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T15:56:54.050 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-23T15:56:54.050 INFO:teuthology.orchestra.run.smithi076.stderr: 94.7% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-23T15:56:54.050 INFO:teuthology.orchestra.run.smithi076.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.b.log: /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.4.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T15:56:54.057 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-23T15:56:54.058 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-23T15:56:54.058 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-23T15:56:54.058 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-23T15:56:54.058 INFO:teuthology.orchestra.run.smithi183.stderr: 90.2%/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.7.log: -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T15:56:54.058 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-23T15:56:54.058 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-23T15:56:54.058 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.11.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-23T15:56:54.060 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log: 94.8% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.2.log: 87.2% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T15:56:54.061 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.c.log: 91.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-23T15:56:54.062 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log 2024-03-23T15:56:54.062 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T15:56:54.062 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log: 83.7% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T15:56:54.062 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-23T15:56:54.062 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log: 87.2% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T15:56:54.063 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.y.log: 91.8% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-23T15:56:54.064 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log 2024-03-23T15:56:54.064 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-23T15:56:54.065 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-23T15:56:54.069 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-23T15:56:54.069 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-23T15:56:54.069 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.d.log: 90.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T15:56:54.070 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-23T15:56:54.070 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log: 87.2% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-23T15:56:54.070 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-23T15:56:54.073 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-23T15:56:54.073 INFO:teuthology.orchestra.run.smithi076.stderr: 94.5% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-23T15:56:54.083 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T15:56:54.095 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-23T15:56:54.399 INFO:teuthology.orchestra.run.smithi076.stderr: 90.2% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-23T15:56:54.516 INFO:teuthology.orchestra.run.smithi192.stderr: 91.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-23T15:56:54.520 INFO:teuthology.orchestra.run.smithi183.stderr: 92.1% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-23T15:56:54.618 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-23T15:56:54.767 INFO:teuthology.orchestra.run.smithi192.stderr: 94.0% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-23T15:56:54.929 INFO:teuthology.orchestra.run.smithi076.stderr: 90.4% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-23T15:56:58.585 INFO:teuthology.orchestra.run.smithi183.stderr: 93.7% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-23T15:56:59.152 INFO:teuthology.orchestra.run.smithi076.stderr: 92.0% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-23T15:56:59.830 INFO:teuthology.orchestra.run.smithi192.stderr: 93.6% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-23T15:57:00.324 INFO:teuthology.orchestra.run.smithi192.stderr: 94.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-23T15:57:01.329 INFO:teuthology.orchestra.run.smithi192.stderr: 92.0% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-23T15:57:01.400 INFO:teuthology.orchestra.run.smithi183.stderr: 92.0% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-23T15:57:01.940 INFO:teuthology.orchestra.run.smithi076.stderr: 91.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-23T15:57:01.999 INFO:teuthology.orchestra.run.smithi076.stderr: 91.8% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-23T15:57:02.258 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-23T15:57:02.891 INFO:teuthology.orchestra.run.smithi192.stderr: 92.1% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-23T15:57:03.063 INFO:teuthology.orchestra.run.smithi076.stderr: 91.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-23T15:57:03.065 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-03-23T15:57:03.065 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m9.037s 2024-03-23T15:57:03.065 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m30.251s 2024-03-23T15:57:03.065 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.968s 2024-03-23T15:57:03.104 INFO:teuthology.orchestra.run.smithi192.stderr: 91.7% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-23T15:57:03.323 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-23T15:57:04.716 INFO:teuthology.orchestra.run.smithi192.stderr: 91.7% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-23T15:57:04.719 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-03-23T15:57:04.719 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m10.686s 2024-03-23T15:57:04.719 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m47.558s 2024-03-23T15:57:04.719 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m1.464s 2024-03-23T15:57:04.996 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-23T15:58:24.945 INFO:teuthology.orchestra.run.smithi183.stderr: 93.6% -- replaced with /var/log/ceph/89c3e582-e92a-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-23T15:58:24.947 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-03-23T15:58:24.947 INFO:teuthology.orchestra.run.smithi183.stderr:real 1m30.914s 2024-03-23T15:58:24.947 INFO:teuthology.orchestra.run.smithi183.stderr:user 2m7.949s 2024-03-23T15:58:24.947 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m4.059s 2024-03-23T15:58:24.947 INFO:tasks.cephadm:Archiving logs... 2024-03-23T15:58:24.948 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi076/log 2024-03-23T15:58:24.949 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T15:58:30.168 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi183/log 2024-03-23T15:58:30.169 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T15:58:51.068 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi192/log 2024-03-23T15:58:51.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T15:58:57.612 INFO:tasks.cephadm:Removing cluster... 2024-03-23T15:58:57.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 --force 2024-03-23T15:58:57.802 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:58:58.103 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 --force 2024-03-23T15:58:58.296 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:58:58.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 89c3e582-e92a-11ee-95cd-87774f69a715 --force 2024-03-23T15:58:58.957 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 89c3e582-e92a-11ee-95cd-87774f69a715 2024-03-23T15:58:59.264 INFO:tasks.cephadm:Removing cephadm ... 2024-03-23T15:58:59.264 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T15:58:59.282 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T15:58:59.299 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T15:58:59.317 INFO:tasks.cephadm:Teardown complete 2024-03-23T15:58:59.317 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T15:58:59.331 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T15:58:59.331 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T15:58:59.334 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T15:58:59.341 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T15:58:59.445 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T15:58:59.456 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T15:58:59.456 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T15:58:59.459 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T15:58:59.461 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T15:58:59.472 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-03-23T15:58:59.473 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-03-23T15:58:59.474 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-03-23T15:58:59.476 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T15:58:59.477 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-03-23T15:58:59.477 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +3170us[+3170us] +/- 101ms 2024-03-23T15:58:59.477 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -1052us[-1030us] +/- 73ms 2024-03-23T15:58:59.477 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -1088us[-1088us] +/- 66ms 2024-03-23T15:58:59.477 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:58:59.478 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T15:58:59.478 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-03-23T15:58:59.478 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +3171us[+3225us] +/- 101ms 2024-03-23T15:58:59.479 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -1043us[ -989us] +/- 73ms 2024-03-23T15:58:59.479 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -1064us[-1009us] +/- 66ms 2024-03-23T15:58:59.479 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:58:59.480 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T15:58:59.480 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-03-23T15:58:59.480 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +3179us[+3179us] +/- 101ms 2024-03-23T15:58:59.480 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -1042us[-1002us] +/- 73ms 2024-03-23T15:58:59.480 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -1060us[-1020us] +/- 66ms 2024-03-23T15:58:59.480 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T15:58:59.480 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T15:58:59.559 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T15:58:59.559 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T15:58:59.574 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T15:58:59.659 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T15:58:59.731 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T15:58:59.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T15:58:59.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:58:59.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T15:58:59.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:58:59.864 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-23T15:58:59.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:58:59.894 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T15:58:59.908 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T15:58:59.922 INFO:teuthology.task.internal:Duration was 1997.248955 seconds 2024-03-23T15:58:59.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T15:58:59.935 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T15:58:59.936 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T15:58:59.938 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T15:58:59.940 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T15:58:59.980 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T15:58:59.981 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T15:58:59.983 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T15:59:00.413 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T15:59:00.413 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-03-23T15:59:00.413 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T15:59:00.434 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-03-23T15:59:00.434 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T15:59:00.487 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-03-23T15:59:00.487 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T15:59:00.508 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T15:59:00.508 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T15:59:00.510 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T15:59:00.530 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T15:59:00.577 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T15:59:00.578 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T15:59:00.580 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T15:59:00.599 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T15:59:00.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T15:59:00.970 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T15:59:00.971 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T15:59:00.998 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T15:59:01.025 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T15:59:01.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T15:59:01.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T15:59:01.067 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T15:59:01.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T15:59:01.090 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-03-23T15:59:01.092 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-03-23T15:59:01.117 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-03-23T15:59:01.152 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T15:59:01.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:59:01.167 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T15:59:01.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:59:01.181 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T15:59:01.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T15:59:01.207 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T15:59:01.220 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T15:59:01.222 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi076 2024-03-23T15:59:01.222 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T15:59:01.268 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi183 2024-03-23T15:59:01.268 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T15:59:01.307 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448/remote/smithi192 2024-03-23T15:59:01.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T15:59:01.346 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T15:59:01.346 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T15:59:01.348 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T15:59:01.350 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T15:59:01.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T15:59:01.411 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T15:59:01.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T15:59:01.423 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T15:59:01.423 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T15:59:01.425 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T15:59:01.427 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T15:59:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: 265309 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 15:59 /home/ubuntu/cephtest 2024-03-23T15:59:01.439 INFO:teuthology.orchestra.run.smithi183.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 15:59 /home/ubuntu/cephtest 2024-03-23T15:59:01.450 INFO:teuthology.orchestra.run.smithi192.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 15:59 /home/ubuntu/cephtest 2024-03-23T15:59:01.451 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T15:59:01.464 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T15:59:01.489 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T15:59:01.517 DEBUG:teuthology.nuke:shortname: smithi076 2024-03-23T15:59:01.517 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T15:59:01.546 DEBUG:teuthology.nuke:shortname: smithi183 2024-03-23T15:59:01.546 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T15:59:01.551 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 15:16:03.538390', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMx8IIWK56fikEo9ZprYtOVI1zpIm3NUtHMXlwNOqNiXuGpae7usAivni1APB+pFqGQqFcd8HEmCP45W4JU36E='} 2024-03-23T15:59:01.569 DEBUG:teuthology.nuke:shortname: smithi192 2024-03-23T15:59:01.570 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T15:59:01.573 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 15:16:03.539448', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/fFKzerWxP+Si6gH89i627Ly6MfnNTRjCc4sYQ3jQ0LgyClsRX5Y/VUf2tJGG21DEjLQDwJemJQt8EuJhC0hY='} 2024-03-23T15:59:01.602 INFO:teuthology.orchestra.console.smithi076:Power off 2024-03-23T15:59:01.603 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T15:59:01.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618448', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 15:16:03.540336', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmYcB6Xaa/ZMTcukp3RTyLUzGVkjjFGUVVNpJ/cchpy6nMeqsRbkWdNPLH7bKHfPDAaMP/JQUzVruu58q60J8A='} 2024-03-23T15:59:01.633 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2024-03-23T15:59:01.633 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:01.650 INFO:teuthology.orchestra.console.smithi183:Power off 2024-03-23T15:59:01.650 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T15:59:01.663 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-03-23T15:59:01.674 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2024-03-23T15:59:01.674 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:01.692 INFO:teuthology.orchestra.console.smithi192:Power off 2024-03-23T15:59:01.692 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T15:59:01.701 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-03-23T15:59:01.713 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-03-23T15:59:01.713 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:01.736 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-03-23T15:59:05.664 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:05.703 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:05.737 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:05.777 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-03-23T15:59:05.814 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-03-23T15:59:05.848 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-03-23T15:59:09.782 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:09.815 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:09.850 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:09.893 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-03-23T15:59:09.926 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-03-23T15:59:09.960 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-03-23T15:59:10.027 INFO:teuthology.orchestra.console.smithi183:Power off completed 2024-03-23T15:59:10.061 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-03-23T15:59:13.895 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T15:59:14.006 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2024-03-23T15:59:14.106 INFO:teuthology.orchestra.console.smithi076:Power off completed 2024-03-23T15:59:14.211 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} duration: 1997.2489545345306 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-03-23T15:59:14.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T15:59:14.301 INFO:teuthology.run:FAIL